Memory-Mapped File is Faster on Huge Sequential Read? Why?

Question

I used the code below to measure the performance difference between reading large, sequential reads of a memory-mapped file, as compared to just calling ReadFile:

HANDLE hFile = CreateFile(_T("D:\\LARGE_ENOUGH_FILE"), FILE_READ_DATA, FILE_SHARE_READ | FILE_SHARE_WRITE, NULL, OPEN_EXISTING, FILE_FLAG_NO_BUFFERING, NULL); __try { const size_t TO_READ = 32 * 1024 * 1024; char sum = 0; #if TEST_READ_FILE DWORD start = GetTickCount(); char* p = (char*)malloc(TO_READ); DWORD nw; ReadFile(hFile, p, TO_READ, &nw, NULL); #else HANDLE hMapping = CreateFileMapping(hFile, NULL, PAGE_READONLY, 0, 0, NULL); const char* const p = (const char*)MapViewOfFile(hMapping, FILE_MAP_READ, 0, 0, 0); DWORD start = GetTickCount(); #endif for (size_t i = 0; i < TO_READ; i++) { sum += p[i]; // Do something kind of trivial... } DWORD end = GetTickCount(); _tprintf(_T("Elapsed: %u"), end - start); } __finally { CloseHandle(hFile); }

(I just changed the value of TEST_READ_FILE to change the test.)

To my surprise, ReadFile was slower by ~20%! Why?

Not really... how can I be sure? I've turned off the file system cache but I can't do much about the disk cache... — user541686
– user541686, Commented Mar 10, 2011 at 8:21
@Thilo: There's no "order" to the tests -- notice that only one of them happens at every run, and I alternated the runs. How would rebooting change anything? — user541686
– user541686, Commented Mar 10, 2011 at 8:28
Rebooting (as in power-cycle) should clear the cache. By order I mean the order in which you run the two test programs. — Thilo
– Thilo, Commented Mar 10, 2011 at 8:31

Ben Voigt · Accepted Answer · 2011-03-10 13:15:56Z

8

FILE_FLAG_NO_BUFFERING cripples ReadFile. The memory-mapped file is free to use whatever read-ahead algorithm it wants, and you've forbidden ReadFile to do the same. You've turned off caching only in the ReadFile version. Memory-mapped files can't work without file cache.

answered Mar 10, 2011 at 13:15

Ben Voigt

286k45 gold badges444 silver badges764 bronze badges

Sign up to request clarification or add additional context in comments.

6 Comments

user541686 Over a year ago

@Ben: Wait, why wouldn't the memory-mapped file respect the flag?

Ben Voigt Over a year ago

@Mehrdad: How could it? MapViewOfFile maps pages from the file cache into your process memory space using the MMU.

user541686 Over a year ago

@Ben: I had no idea how it worked, but I imagined that it caused a page fault whenever a page was accessed, so the kernel could take control and fill the page with the data -- so there didn't actually have to be any caching.

Ben Voigt Over a year ago

You should also use FILE_FLAG_SEQUENTIAL_SCAN for large sequential reads.

Ben Voigt Over a year ago

@Mehrdad: It only does that the first time a page is accessed. Re-reading each page 4000 times would be crazy. And it's the cache manager which fills that page with data and decides when to free it... this is how memory-mapped files stay consistent between processes, because the same physical page is mapped by the cache manager into multiple processes. Your process doesn't own the memory.

|

Collectives™ on Stack Overflow

Memory-Mapped File is Faster on Huge Sequential Read? Why?

1 Answer 1

6 Comments

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

6 Comments

Linked

Related