How to read file into a pointer / raw vec?

Question

I'm using a copy of the stdlib's raw vec to build my own data structure. I'd like to reaad a chunk of a file directly into my data structure (no extra copies). RawVec has a *const u8 as it's underlying storage and I'd like to read the file directly into that.

// Goal: // Takes a file, a pointer to read bytes into, a number of free bytes @ that pointer // and returns the number of bytes read fn read_into_ptr(file: &mut File, ptr: *mut u8, free_space: usize) -> usize { // read up to free_space bytes into ptr todo!() }

// What I have now requires an extra copy. First I read into my read buffer // Then move copy into my destination where I actually want to data. // How can I remove this extra copy? fn read_into_ptr(file: &mut File, ptr: *mut u8, read_buf: &mut[u8; 4096]) -> usize { let num_bytes = file.read(read_buf).unwrap(); unsafe { ptr::copy_nonoverlapping(...) } num_bytes } ``

If your intention is to avoid memory copies as much as possible, you may be interested in using a wrapper around the mmap system call instead of a Vec. — prog-fh
– prog-fh, Commented Dec 19, 2022 at 9:49
@prog-fh mmap ended up being slower than streaming reads into a ring buffer — nlta
– nlta, Commented Dec 20, 2022 at 1:07

Chayim Friedman · Accepted Answer · 2022-12-19 09:31:16Z

2

Create a slice from the pointer, and read into it:

let slice = unsafe { std::slice::from_raw_parts_mut(ptr, free_space) }; file.read(slice).unwrap()

answered Dec 19, 2022 at 9:31

Chayim Friedman

76.1k5 gold badges96 silver badges140 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Overlook Motel Jun 19 at 14:14

That would be undefined behavior unless all the bytes in the memory range ptr to ptr.add(free_space) are already initialized. RawVec makes no such guarantee (and generally they won't be).

Collectives™ on Stack Overflow

How to read file into a pointer / raw vec?

1 Answer 1

1 Comment

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Related