race condition in GNU "tail -f"
Ken Raeburn <[email protected]> Thu, 15 May 2003 00:21:12 -0400
| Newsgroups | gmane.comp.gnu.textutils.bugs |
|---|---|
| Message-ID | <[email protected]> |
It appears that there is a small window between when GNU "tail -f" reads the last bytes of the file (adjusting the current offset of the file descriptor to the then-current end of the file) and when it calls fstat() to get the size of the file. The fstat size is saved away as the current size of the file. The tail_forever loop calls fstat and checks the return size against the saved current size of the file. Any data added during that window will not be displayed until *another* change is made to the file size, normally by appending more data. At that point, the data added during the window and the data added later will be read and displayed, and the combined size added to tail's notion of the current file size. But the earlier fstat call reported a size that would've included the data added during the window, so that data has now been counted twice. So, the next time fstat is called to check for new data, if the file hasn't been modified, or if the added data is smaller than the data added during the first window, the file will appear to have shrunk; tail will complain and adjust the computed current size back to the real current size, without displaying any output, so some data might never be seen. If the file has been modified by adding more data than was added during the initial window, I believe it'll all be displayed, the computed current size updated and still out of sync, so the out-of-sync state can probably be maintained until the file stops growing fast enough. Since tail is just reading and displaying a few lines, normally, and the file is already open and actively being accessed, and especially if the file is on local disk, one would think that this theoretical race condition wouldn't ever cause a problem. And even if it did, a little missed data probably wouldn't ever be noticed. As it happens, I'm working on a project that has an expect script which fires up "tail -f" on a log file, then appends a cookie value to the log file, and waits for the cookie to show up in the tail output so we know when tail has reached the end of the file. We've hit this window quite a number of times; enough that it makes our testing unreliable on our slower systems. I've got some workarounds to try, but tail still needs fixing... My suggestion would be to use lseek to get the current file position as the initial computed file size for the tail_forever loop, and ignore the size returned by fstat. Or, check if they're different and read some more from the file until caught up. Ken