1

Cant really find any specifics on this, heres all I know about mmf's in windows:

  1. Creating a memory mapped file in windows adds nothing to the apparent amount of memory a program uses
  2. Creating a view to that file consumes memory equivalent to the view size

This looks rather backwards to me, since for one, I know that the mmf itself actually has memory...somewhere. If I write something in a mmf and destroy the view, the data is still there. Meanwhile, why does the view take any memory at all? Its just a pointer, no?

Then theres the weirdness with whats actually in the ram and whats on the disk. In large mmf's with a distributed looking access pattern, sometimes the speed is there and sometimes its not. I'm guessing some of it gets sometimes stored in the file if one is tied to it or the paging file but really, I have no clue.

Anyways, the problem that drove me to investigate this is that I have a ~2gb file that I want multiple programs to share. I can't create a 2gb view in each of them since I'm just "out of memory" so I have to create/destroy smaller ones. This creates a lot of overhead due to additional offset calculations and the creation of the view itself. Can anybody explain to me why it is like this?

Improve this question
3
  • Your assuming the the OS will make several copies of the memory mapping, one for each program. This may not be the case. You should investigate further by searching the MSDN site. Commented Oct 9, 2015 at 20:27
  • Be aware that the OS may not read the entire 2GB file into memory when you create the memory map. Most likely, the OS will read in a chunk of the file into memory. Commented Oct 9, 2015 at 20:31
  • There are a lot of different ways to measure memory usage. Which one are you talking about? The availability of physical memory shouldn't affect the use of memory-mapped files, unless the system is severely overloaded. Commented Oct 9, 2015 at 20:53

3 Answers 3

Reset to default
7

On a demand-paged virtual memory operating system like Windows, the view of an MMF occupies address space. Just numbers to the processor, one for each 4096 bytes. You only start using RAM until you actually use the view. Reading or writing data. At which point you trigger a page fault and force the OS to map the virtual memory page to physical memory. The "demand-paged" part.

You can't get a single chunk of 2 GB of address space in a 32-bit process since there would not be room for anything else. The limit is the largest hole in the address space between other allocations for code and data, usually hovers around ~650 megabytes, give or take. You'll need to target x64. Or build an x86 program that's linked with /LARGEADDRESSAWARE and runs on a 64-bit operating system. A backdoor which is getting to be pretty pointless these days.

Improve this answer
Sign up to request clarification or add additional context in comments.

Comments

0

The thing in memory mapped file is that it lets you manipulate its data without I/O calls. Because of this behavior, when you access the file, windows loads it to the physical memory, so it can be manipulated in it rather than on the disk. You can read more about this in here: http://blogs.msdn.com/b/khen1234/archive/2006/01/30/519483.aspx

Improve this answer

2 Comments

I don't think so: "...data without I/O calls." There must be I/O calls to fetch the data from the file. Memory mapping means the OS handles the reading and buffering for you. There are still I/O calls; with memory mapping, the OS makes the calls.
There are I/O calls of course, because, as you said, the file is need to be fetched, but what I meant is that not every manipulation of the file leads to an I/O call.
0

Anyways, the problem that drove me to investigate this is that I have a ~2gb file that I want multiple programs to share. I can't create a 2gb view in each of them since I'm just "out of memory" so I have to create/destroy smaller ones.

The most likely cause is that the programs are 32-bit. 32-bit programs (by default) only have 2GB of address space so you can't map a 2GB file in a single view. If you rebuild them in 64-bit mode, the problem should go away.

Improve this answer

Comments

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.