Microblog: A very long article Wikipedia article on the orientation of toilet paper [Jun 7th, 22:52] [R]

Tuesday, May 4th, 2010

Shell Scripts and Large Files

Categories: [ IT | TV/Leffakone ]

I use a shell script wrapper around MPlayer for recording TV programs in my Leffakone, the problem being that when TV reception is too bad, MPlayer crashes or freezes, so the wrapper restarts it when needed. Since MPlayer doesn't output anything while recording, I use a loop that checks every second that the recorded file is actually growing. Whem it stops growing (and there is still some recording time left), it means that MPlayer stopped working and that it needs to be restarted. Until now, I did this in a loop like this: 
while [ $(stat -c %s $filename) -gt $last_size ]
do
  last_size=$(stat -c %s $filename)
  sleep 1
done

I then noticed that all recordings exceeding one hour (there aren't that many, that's why it took so long to notice it) where cut in two even though the picture doesn't show hints of bad reception. More peculiar, the size of the first part is slightly over 2 GB in size (therefore the problem is not related to a 2 GB size limit, right?). An experiment conducted yesterday show that it actually is: the test program (used here in its |[| form) handling the -gt comparison actually doesn't like values greater that 2ˆ31, which caused the loop to be interrupted and the recording to be split into two parts.

The solution is then (for Bash at least) to use the following syntax: 

while [[ `stat -c %s $filename` -gt $last_size ]]
which seems to be working with values greater than 2ˆ31 (and than 2ˆ32, I just checked).

[ Posted on May 4th, 2010 at 08:53 | 1 comment | ]

Trackback Address

https://weber.fi.eu.org/blog/Informatique/shell_scripts_and_large_files.trackback

Comments

I noticed that with the old Bash 2.05 running on Leffakone, the [[ form is also afflicted by the 2^31 bug (Bash 3.2 works correctly in that respect, though). The solution was to use an external perl script for watching the file grow.

Comment #1, Matthieu Weber (Jyväskylä, Finland), May 22nd, 2010 at 15:36

Add comments

You can use the following HTML tags: <p>, <br>, <em> <strong>, <pre>. URLs starting with http:// will automatically be turned into hyperlinks.

(optional)
(optional)


Save my Name and URL/Email for next time

18 / 3 =