33

I have a struct with a BufStream<T> where T: Read+Write. The BufStream can be a TcpStream and I'd like to read n bytes from it. Not a fixed amount of bytes in a predefined buffer, but I have a string/stream which indicates the number of bytes to read next.

Is there a nice way to do that?

2 Answers 2

38

Since Rust 1.6, Read::read_exact can be used to do this. If bytes_to_read is the number of bytes you need to read, possibly determined at runtime, and reader is the stream to read from:

let mut buf = vec![0u8; bytes_to_read]; reader.read_exact(&mut buf)?; 

The part that wasn't clear to me from the read_exact documentation was that the target buffer can be a dynamically-allocated Vec.

Thanks to the Rust Gitter community for pointing me to this solution.

Sign up to request clarification or add additional context in comments.

7 Comments

It's also not clear what happens if EOF is returned before bytes_to_read.
That's specified in the documentation: "If this function encounters an "end of file" before completely filling the buffer, it returns an error of the kind ErrorKind::UnexpectedEof."
True, so I guess you can't actually do this. In the sense that it's not defined what happens to the buffer contents/impossible to recover.
@Nanos Hopefully you've already figured this out elsewhere, but, the answer is you need to fill the Vec with some value first, (or unsafely set the length I guess). The vec![0u8; bytes_to_read] used above fills the Vec with bytes_to_read zeroes.
Same problem. Vec::with_capacity won't allocate until you append to it, so if you try to read_exact into it, it'll read 0 bytes. You can either call vec.resize(bytes_to_read, 0); or use the vec! macro - the macro is faster.
|
26

It sounds like you want Read::take and Read::read_to_end.

This will allow you to read data into a &mut Vec<u8>, which is useful when you want to reuse an existing buffer or don't have an appropriately sized slice already. This allows you to avoid initializing the data with dummy values before overwriting them with the newly-read information:

use std::{ io::{prelude::*, BufReader}, str, }; fn read_n<R>(reader: R, bytes_to_read: u64) -> Vec<u8> where R: Read, { let mut buf = vec![]; let mut chunk = reader.take(bytes_to_read); // Do appropriate error handling for your situation // Maybe it's OK if you didn't read enough bytes? let n = chunk.read_to_end(&mut buf).expect("Didn't read enough"); assert_eq!(bytes_to_read as usize, n); buf } fn main() { let input_data = b"hello world"; let mut reader = BufReader::new(&input_data[..]); let first = read_n(&mut reader, 5); let _ = read_n(&mut reader, 1); let second = read_n(&mut reader, 5); println!( "{:?}, {:?}", str::from_utf8(&first), str::from_utf8(&second) ); } 

If you are worried that Read::take consumes the reader by reference, note that take comes from Read and Read is implemented for any mutable reference to a type that implements Read. You can also use Read::by_ref to create this mutable reference.

See also:

9 Comments

Thanks.. doesn't take() consume the reader? My usecase is that i have a tcpstream (part of a struct) from which i read lines. Sometimes in between i don't need to read a line but a specific amount of bytes. Afterwards i'll need to read lines again...
It does, but "the reader" can be a &mut R i.e a temporary mutable reference to another reader. Take uses a by-value API so that all use cases are supported. This model appears in some different corners of rust.
@buster as bluss points out, &mut R also implements the Read trait. My example uses that to good effect - that's why I pass in &mut reader in the main method of the example.
@buster @Shepmaster how did you guys figure out that Read::take also accepts a &mut reader? Wondering how I would be able to figure that out myself next time. :)
|

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.