0

I am unable to understand how files are managed in memory mapped I/O. As normal If we open a file using open or fopen, it returns fd or file pointer respectively. After this open where the file resides for processing. It is in memory(copy of the file which is in hard disk) or not? If it is not in memory where the data is fetch by consequent read or write system call or It fetchs data from the hard disk for each time of calling read or write. Otherwise the copy of the file is stored in memory and the file is accessed by process for furthur manipulation and once the process is completed the file is copied to hard disk. In the above concepts which scenario is worked ?

The following is the definition given for memory mapped i/o in Advanced Programming in Unix Environment(2nd Edition) book:

Memory-mapped I/O lets us map a file on disk into a buffer in memory so that, when we fetch bytes from the buffer, the corresponding bytes of the file are read. Similarly, when we store data in the buffer, the corresponding bytes are automatically written to the file. This lets us perform I/O without using read or write.

what is mapping a file into memory? And here, they defined the memory is placed in between stack and heap. In this memory, what type of data is present after mapping a file. It contains copy of the file or the address of the file which resides in hard disk. And how the above scenario becomes true.

Does anyone explain the working mechanism of memory mapped I/O and mmap functionality?

Improve this question
4
  • 1
    Drivers...................................... Commented Dec 14, 2015 at 14:02
  • which means? I am using ubuntu 12.04 Commented Dec 14, 2015 at 14:03
  • Read about page cache & linuxatemyram.com & virtual address space Commented Dec 14, 2015 at 14:09
  • To the original poster, some general advice: When you have a question like this that is more advanced DO NOT POST cross post to a broad group like "C" as you have. There are too many yahoos who follow those topics who are trying to run up SO points and close questions they do not understand and just vote to close the question instead. Commented Dec 17, 2015 at 18:26

3 Answers 3

Reset to default
3

Normally when you open a file, the system sets up some bookkeeping structures (metadata) but does not need to read any part of the actual data of the file. When you call read(), the system loads a chunk of the file into (virtual) memory which you allocated for the purpose.

When you memory-map a file, the system again sets up bookkeeping, and also sets up a (virtual) memory "mapping" which means a range of valid addresses which, if used, will reflect reads (or writes) of the underlying file. It does not mean the entire file needs to be read at once, because it can be "paged in" on demand, i.e. the system can give you an address range to use, then wait for you to actually use it before loading any data there. This "page faulting" is supported by a hardware device called the Memory Management Unit, or MMU. The same system is used when you run an executable file--the system can simply map it into virtual memory and read pages (chunks) from disk only as needed.

Improve this answer
Sign up to request clarification or add additional context in comments.

3 Comments

What is the range of valid address. Is the address of file which is in hard disk ?
No, it is a range of addresses in "virtual memory" which is a per-process concept, meaning the number only has meaning to the process whose "virtual memory" it is. One process might see 0x2000 as one object, and another might see the same virtual address 0x2000 as another thing entirely. Files on disk do not have "addresses" in the same flat/linear sense--it's the operating system's job to hide this fact from you.
I like to liken it to the inverse of a "swap file" :)
1

It is in memory(copy of the file which is in hard disk) or not?

According to Computer Programming and Utilization, When you open file with fopen its content are loaded into memory. (Partially or wholly).

If it is not in memory where the data is fetch by consequent read or write system call

When you fwrite some data, it is eventually copied into the kernel which will then write it to disk (or wherever) after buffering. In general, no part of a file needs to be loaded in order to write.

what is mapping a file into memory? enter image description here For more refer here

In this memory, what type of data is present after mapping a file. It contains copy of the file or the address of the file which resides in hard disk.

A memory-mapped file is a segment of virtual memory which has been assigned a direct byte-for-byte correlation with some portion of a file or file-like resource.Refer this

It is possible to mmap a file to a region of memory. When this is done, the file can be accessed just like an array in the program.This is more efficient than read or write, as only the regions of the file that a program actually accesses are loaded. Accesses to not-yet-loaded parts of the mmapped region are handled in the same way as swapped out pages.

Improve this answer

5 Comments

Usually, the libc buffers the data in userland when using fwrite/fprintf/etc, not in the kernel. Furthermore, data usually is not copied into the kernel but directly written from its userspace location through DMA
Updating the answer :-)
within memory mapped region of file what type of data is present. If it is address, where it is points to? It directly points the address of files which is stored in hard disk?
@Ctx Furthermore, data usually is not copied into the kernel but directly written from its userspace location through DMA That would be direct IO. Most IO operations on files are done via kernel-level caching - the page cache.
@ctx you are absolutely right. I am happy if You can edit my answer and make it more useful to someone.
0

After this open where the file resides for processing. It is in memory(copy of the file which is in hard disk) or not?

On the disk. It may also be partly or completely in memory if the operating system does a read-ahead, but that isn't detectable by you. You still have to issue reads to get data from the file.

If it is not in memory where the data is fetch by consequent read or write system call

From the disk.

or It fetchs data from the hard disk for each time of calling read or write.

In effect, but you also have to consider the effect of any caching.

Otherwise the copy of the file is stored in memory and the file is accessed by process for furthur manipulation and once the process is completed the file is copied to hard disk.

No. The file behaves as though it is all on the disk.

And here, they defined the memory is placed in between stack and heap.

Not in what you quoted.

In this memory, what type of data is present after mapping a file.

The data in the file. The question 'what type of data' doesn't make sense. Data is data.

It contains copy of the file or the address of the file which resides in hard disk.

It effectively contains a copy of the file.

And how the above scenario becomes true.

Via virtual memory. Too broad to cover here.

Improve this answer

Comments

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.