3

I have a large binary file (more than 500mb) that I want to read a part of and extract data from. I'm sure I shouldn't load the whole file into memory all at once, so how could I load only a part of it, using hex offsets?

I've never worked with this kind of thing before, so I have no idea where to start. All I want to read and write to the file is text.

Improve this question
17
  • 4
    If you know the offset (the human representation of this offset, be it octal, decimal, or hex is totally irrelevant) then just seek to that part of the file and read the correct number of bytes. If this doesn't make sense to you, it's time for a new book. Commented Jan 14, 2013 at 21:06
  • 1
    @LightnessRacesinOrbit: He is working with text, so seeking may not be an option... Commented Jan 14, 2013 at 21:08
  • 1
    It says binary file in the question... Commented Jan 14, 2013 at 21:09
  • 1
    Hex offsets? Are they more efficient than decimal offsets? Commented Jan 14, 2013 at 21:13
  • 2
    @Kerrek: They're prettier! And can take you so much further... Commented Jan 14, 2013 at 21:15

1 Answer 1

Reset to default
6

Use a Memory Mapped File. This will allow you to process the file as if it were in memory, but it only ever gets loaded into memory a page at a time.

Improve this answer
Sign up to request clarification or add additional context in comments.

6 Comments

might also want to link mmap for posix OSs, but the idea is just right :)
It does get loaded into memory, just one page at a time.
Great! That sounds perfect. Do you know of a guide or any other resource for using these in C++?
@Alex, correct - but it never gets totally loaded to memory. Thanks
@OtávioDécio That's for C#/VB. What about for C++?
|

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.