33

Suppose I read (cat) a file while another process is rewriting its contents. Is the output predictable? What would happen?

Improve this question
2

3 Answers 3

Reset to default
31

That depends on what the writer does.

If the writer overwrites the existing file, then the reader will see the new content when the writer overtakes the reader, if ever. If the writer and the reader proceed at variable speeds, the reader may alternatively see old and new content.

If the writer truncates the file before it starts to write, the reader will run against the end of the file at that point.

If the writer creates a new file then moves the new file to the old name, the reader will keep reading from the old file. If an opened file is moved or removed, the processes that have the file opened keep reading from that same file. If the file is removed, it actually remains on the disk (but with no way to open it again) until the last process has closed it.

Unix systems tend not to have mandatory locks. If an application wants to ensure that its writer component and its reader component don't step on each other's toes, it's up to the developer to use proper locking. There are a few exceptions where a file that's open by the kernel may be protected from writing by user applications, for example a loop-mounted filesystem image or an executable that's being executed on some unix variants.

Improve this answer
4
  • Gilles, does your explanation also apply in ftp/sftp scenarios? Say a process starts reading a transmitted ftp file while another version of the same file is overwriting it due to a new transmission. Commented Aug 4, 2014 at 20:27
  • @1_CR Yes. In that case, the writer and the reader are the ftpd processes. Commented Aug 4, 2014 at 21:46
  • what's the meaning of overtake here? Commented Oct 10, 2016 at 7:02
  • @VictorChoy Standard meaning: to start behind/after someone else and at some point move ahead of them. Commented Oct 10, 2016 at 17:50
23

It's a classic race condition, so the outcome is unpredictable by definition.

Among others, it depends on

  • fopen(3) or open(2) write modes,
  • how/if the writer is buffering its output,
  • how the reader is reading the file,
  • the speed difference between the reader and writer,
  • the time difference between the read and writer's start.
  • And of course, on modern multi-core machines, things are complicated even more by other factors lower down (e.g. process scheduling).

If you need to be able to read a file while it's being rewritten, then you can make the writer make a transient copy of the file, modify that, then copy it back to the original file. This the way rsync does this, for instance. There are a number of ways to implement this, but no free lunch. Each method has its own shortcomings and repercussions.

Improve this answer
1
  • 1
    This answer is far more comprehensive than mine. Deleting my answer. Commented Jun 26, 2012 at 14:22
0

Previous responders have more comprehensive explanations than this, but here's a trick that definitely works as well, pretty much doing exactly what he wants:

$ tail -f <filename>

Will show you the end of the file as it's being written. Handy if you want to pipe STDERR to a file but still see it in another terminal window, for example.

Improve this answer

You must log in to answer this question.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.