How to force file flushing

Question

Suppose that I have the following code:

#include <chrono> #include <fstream> #include <thread> int main() { std::ofstream f("test.log"); int i = 0; while (true) { f << i++; f.flush(); std::this_thread::sleep_for(std::chrono::milliseconds(100)); } }

(note that I have a flush call after each write operation)

I noticed that this application doesn't update "last modified time" and "size" attributes of the "test.log" file unless I do a right-click on this file or open it.

I guess that this is due to an internal bufferization (system doesn't want to make such time-consuming operations as an actual I/O to disk unless forced to do so). Am I right?

I need to write an application that should watch for changes in log files created by other applications (I can't change them). At first, I thought about FileSystemWatcher class in C# but I noticed that it has the same behavior (it doesn't fire a corresponding event unless file was closed in a source application or was forced to update by right-clicking that file in Windows Explorer). What can I do then? Call WinAPI functions like GetFileAttributes for every file that I want to look for as often as I can?

I think you're suffering from standard windows buffering, especially common on network drives. In that no it doesnt notice immediately, you are semi doomed to wait till windows acknowledges the write to disk - this maybe also caused by disk buffering maybe, in that although it doesnt make a lot of sense, because windows buffers the write, if windows hasnt actually committed it yet, other portions dont see it — BugFinder
– BugFinder, Commented Jul 27, 2016 at 8:34
@BugFinder I see. But I have a problem to solve anyway. I need to find a correct way to force those flushes made by other applications — FrozenHeart
– FrozenHeart, Commented Jul 27, 2016 at 8:36
I noticed that this application doesn't update "last modified time" and "size" attributes of the "test.log" file unless I do a right-click on this file or open it. That sounds more like a problem with the GUI application you're using to view and use the mouse to open the file with. — Andrew Henle
– Andrew Henle, Commented Jul 27, 2016 at 8:38
Try to write such that application. Something like a program that get the status of the file once a second. Probably this program would detect the change. — EFenix
– EFenix, Commented Jul 27, 2016 at 8:39

Ben · Accepted Answer · 2016-07-27 08:58:11Z

There are two separate things here. First, the last modified time on the file MFT record (inode equivalent) is updated every time you write to it.

However the information returned by FindFirstFile and friends is not from the file, it is from information cached in the directory entry. This cache is updated whenever a file is closed which was opened through that directory entry. This is the information displayed by most applications, such as Windows Explorer, and the command prompt DIR command.

If you want to know when a file was updated you need to do the equivalent of a Unix stat operation which reads the MFT record (inode). This requires opening a handle to the file, calling GetFileInformationByHandle and closing the handle again.

The second thing is that there is a good reason not to do this. If a program is writing to a file, it may be partway through the writing process. Therefore the file may be in an invalid (corrupt) state. To ensure that the file is in a valid state you should wait until the file has been closed. This is how you know that the file is now ready to look at.

Once the writing program has finished writing to the file, the directory entry will be updated and FileSystemWatcher will show the file.

If you are absolutely sure you want to see notifications of files which are still in the process of being written, then you can look into the USN change journal as an option. I don't know if this is kept more up to date than the directory entries, you will have to investigate that.

I need to watch for file changes in speific directory. At this moment, I can't rely on FileSystemWatcher class from C#. What can I do then? Manually iterate over all files in the directory, open it and watch for changes?
So, flushing a file does actually writes any buffered content to the disk, am I right? Because I'm not sure whether Windows Explorer / Total Commander just display the cached file info (such as last modify time and size) or the system actually flushes an internal buffer only when I right-click / open that file / close it in the application
Flushing a file writes the content to the disk. It just doesn't update the directory entry.
Note that you don't need to do an open-getfileinformationbyhandle-close loop, you can call GetFileInformationByHandle repeatedly on the same handle. (You'll want to specify an access mask of 0 when opening the file, otherwise your handle and the other program's handle will interfere with one another.)

Collectives™ on Stack Overflow

How to force file flushing

1 Answer 1

4 Comments

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

4 Comments

Related