How to make fprintf() writes immediately

Question

One way to write into a file is by using fprintf(). However, this function does not write the results into a file immediately. It rather seems to write everything at once when the program is terminated or finished.

My question is the following: I have a program that takes very long time to run (4-5 hours for big dataset). During this time, I want to see the intermediate results so that I don't have to wait for 5 hours. My university uses Sun Grid Engine for job submission. As most of you know, you have to wait until your job finishes to see your final results. Thus, I want to be able to write the intermediate results into a text file and see the updated results as the program is processing (Similarly if I am using printf).

How can I modify fprintf() to write anything I want immediately to the target file?

See also What are the rules of automatic flushing stdout buffer in C?. — chux
– chux, Commented Jul 27, 2017 at 2:27

dbush · Accepted Answer · 2017-07-27 02:03:28Z

You can use the fflush function after each write to flush the output buffer to disk.

fprintf(fileptr, "writing to file\n"); fflush(fileptr);

If you're on a POSIX system (i.e. Linux, BSD, etc), and you really want to be sure the file is written to disk, i.e. you want to flush the kernel buffers as well as the userspace buffers, also use fsync:

fsync(fileno(fileptr));

But fflush should be sufficient. Don't bother with fsync unless you find that you need to.

anka · Accepted Answer · 2017-07-27 04:07:10Z

Maybe you can set FILE pointer _IONBF mode. Then you cloud not use fflush or fsync. FILE *pFilePointor = fopen(...); setvbuf(pFilePointor, NULL, _IONBF, 0);

fprintf(...) fprintf(...)

Tasty · Accepted Answer · 2024-11-17 22:54:34Z

Set the buffering mode on the file stream.

setbuf(logFile, NULL);

https://learn.microsoft.com/en-us/cpp/c-runtime-library/reference/setbuf?view=msvc-170

Updated: https://learn.microsoft.com/en-us/cpp/c-runtime-library/reference/setvbuf?view=msvc-170

Kamal Palei · Accepted Answer · 2017-07-27 03:23:30Z

fflush
This works on FILE *. For your case it looks more appropriate. Please note fflush(NULL) will update all opened files / streams and my be CPU intensive. You may like to use/avoid fflush(NULL) for performance reason.

fsync
This works on int descriptor. It not only updates file/stream, also metadata. It can work even in system crash / reboot cases as well. You can check man page for more details.

Personally I use fflush, and it works fine for me (in Ubuntu / Linux).

Collectives™ on Stack Overflow

How to make fprintf() writes immediately

4 Answers 4

3 Comments

Comments

Comments

Comments

Linked

Hot Network Questions

Collectives™ on Stack Overflow

4 Answers 4

3 Comments

Comments

Comments

Comments

Linked

Related