3

I'm writing continuously to a file, and I want to tail it from another shell. But it seems that as long as the file is open for writing,

tail -f filename

can't show all the new lines until the file is closed again.

Unfortunately, opening and closing the file many times a second slows my script down to the point where data is received faster than it can be written, so I can't close the file during the process.

The writing is being done by a python program:

ser = serial.Serial('/dev/ttyACM2',9600) f = open("filename", "a+") while True: s = ser.readline() f.write(s + "\r\n")

Isn't there a way to view the contents of a file that is in the process of being written to?

Improve this question
3
  • Do you mean tail -f <file>? Or better: tail -F <file>, the uppercase F will follow even if the file does not exist yet or is logrotated etc. Commented Feb 22, 2019 at 16:27
  • 1
    This is either a matter of not using tail -f on the file, or of the program writing to the file doing buffering (either explicitly or, more likely, implicitly). How are you writing to the file? Buffering of output is done for performance reasons, but the data would not be written until the buffer is full. Sometimes buffering can be avoided. Commented Feb 22, 2019 at 16:28
  • 1
    Seconding the thought that this is most likely a buffering issue. Commented Feb 22, 2019 at 16:29

1 Answer 1

Reset to default
5

If you can't see it with tail, it's not in the file.

It might be in the write buffer of the Python program, though. The customary behavior when writing to files is that the C library buffers outputs until a full block is written (e.g. 4096 bytes or such) instead of sending all the writes to the OS immediately. This saves on system call overhead. This is similar in Python. (I'm not sure if it uses the C library, or if it does similar buffering itself, but the end result is the same.)

For output to the terminal, the default is line buffering: the buffer is written out when a newline is written. Of course, any buffers are also flushed if the file is closed, but that's indeed not very efficient.

To fix that, either call f.flush() after each f.write(), use disable the buffering altogether. There are some ways to do that in this question on SO: Disable output buffering

Improve this answer
1
  • 1
    Sorry it took so long. f.flush() indeed did the trick; I just didn't discover it until now, because I was accessing the text file over SSHFS which apparently buffers a lot. So now I have a new problem - or rather the same problem, but now I need a new solution :-) Thanks! Commented Dec 30, 2020 at 22:43

You must log in to answer this question.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.