2

My knowledge of how file buffers work is fairly weak, so forgive the simple mindedness of this question. I have a piece of code in Python that waits for a file to appear, then reads it.

while 1: try: f = open('file.txt') f.close() print "Message received." break except: time.sleep(0.3)

Where file.txt is a file written by another program. However, I've a suspicion that Python is reading the file and then closing the handle before the file file.txt is completely written (that is, before I even call close() in the program that is writing the file). Should this even be possible?

If it is the case, is there a way I can detect (in the reading program listed above) whether or not the buffers have been flushed before I call f.close()?

Improve this question
1
  • In this case, no, because my goal in this little program is to check if the file exists, and is fully written. But would f.read() solve my problem? (the problem of unflushed buffers) Commented Oct 13, 2011 at 21:39

4 Answers 4

Reset to default
3

Don't use a while-loop for this purpose. There is a better way, though the details depend on your OS.

Linux has inotify. You can write a simple pyinotify script to watch the directory and react to IN_CLOSE_WRITE events (which occur only after the file has been closed).

OSX (FSEvents) and Windows have similar facilities.

Improve this answer
Sign up to request clarification or add additional context in comments.

1 Comment

Thanks, this was what I was looking for. I've used pyinotify to monitor for FS changes, but it didn't occur to me to use it for this very simple task.
1

Python will not stop you from reading a file that another process has not completed writing.

If I'm waiting for a file from another process, I usually have that process write to a different file name. Once that process has finished writing it closes and renames the file to the file name i'm expecting. That way I don't get a partial file.

Improve this answer

Comments

1

To check whether a file exists, use os.path.exists or maybe os.path.isfile if you require it to be a "regular" file.

On Unix, a directory entry will be created when the file is opened for write/append. But bytes written to a file will not actually show up there (from the point of view of a reader) until I/O is flushed. There is no normal way of asking whether there is "pending" I/O on a file; at least not on Unix.

Improve this answer

Comments

0

In general Linux has no problem whatsoever with one process opening a file for read while another process has it open for writing. Think about doing things like tail -f $SOME_FILE to follow updates to a file in real time; that could not possibly work if attempting to open the file for reading had to block and wait until the writer closed the file.

Improve this answer

Comments

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.