76

I usually use stringstream to write into in-memory string. Is there a way to write to a char buffer in binary mode? Consider the following code:

stringstream s; s << 1 << 2 << 3; const char* ch = s.str().c_str();

The memory at ch will look like this: 0x313233 - the ASCII codes of the characters 1, 2 and 3. I'm looking for a way to write the binary values themselves. That is, I want 0x010203 in the memory. The problem is that I want to be able to write a function

void f(ostream& os) { os << 1 << 2 << 3; }

And decide outside what kind of stream to use. Something like this:

mycharstream c; c << 1 << 2 << 3; // c.data == 0x313233; mybinstream b; b << 1 << 2 << 3; // b.data == 0x010203;

Any ideas?

Improve this question
4
  • 1
    That's hex, not binary. Why can't you write 0x01, 0x02, etc., though... those are actual ASCII characters, after all. Commented Oct 13, 2009 at 10:03
  • 5
    He wants the contents of memory (the actual bytes) to be 0x010203 (66051 decimal), not the string "0x010203". Commented Oct 13, 2009 at 11:18
  • 1
    I've modified the question. Hope it's more clear now. Commented Oct 13, 2009 at 14:20
  • 3
    Excellent question. Too bad it is impossible to give a good answer, because this is a design bug in the standard libraries. Commented Jan 12, 2013 at 3:50

7 Answers 7

Reset to default
49

To read and write binary data to streams, including stringstreams, use the read() and write() member functions. So

unsigned char a(1), b(2), c(3), d(4); std::stringstream s; s.write(reinterpret_cast<const char*>(&a), sizeof(unsigned char)); s.write(reinterpret_cast<const char*>(&b), sizeof(unsigned char)); s.write(reinterpret_cast<const char*>(&c), sizeof(unsigned char)); s.write(reinterpret_cast<const char*>(&d), sizeof(unsigned char)); s.read(reinterpret_cast<char*>(&v), sizeof(unsigned int)); std::cout << std::hex << v << "\n";

This gives 0x4030201 on my system.

Edit: To make this work transparently with the insertion and extraction operators (<< and >>), your best bet it to create a derived streambuf that does the right thing, and pass that to whatever streams you want to use.

Improve this answer
Sign up to request clarification or add additional context in comments.

4 Comments

It definitely answers the first part of the question, but is there a way to make the insertion look always the same (i.e. s << a) but the inner data representation differ depending on the type of the stream?
Your own streambuf can't do this; formatting is done in the (non-virtual) istream and ostream methods and the result of that is what the streambuf sees.
The question actually shows the in-memory result 0x010203 whereas this will likely produce 0x00000001 0x00000002 0x00000003 (assuming sizeof(int)==4).
@MSalters You're right, apparently 6 year younger me was an idiot.
14

You can do this sort of thing with templates. E.g:

//struct to hold the value: template<typename T> struct bits_t { T t; }; //no constructor necessary //functions to infer type, construct bits_t with a member initialization list //use a reference to avoid copying. The non-const version lets us extract too template<typename T> bits_t<T&> bits(T &t) { return bits_t<T&>{t}; } template<typename T> bits_t<const T&> bits(const T& t) { return bits_t<const T&>{t}; } //insertion operator to call ::write() on whatever type of stream template<typename S, typename T> S& operator<<(S &s, bits_t<T> b) { return s.write((char*)&b.t, sizeof(T)); } //extraction operator to call ::read(), require a non-const reference here template<typename S, typename T> S& operator>>(S& s, bits_t<T&> b) { return s.read((char*)&b.t, sizeof(T)); }

It could use some cleanup, but it's functional. E.g:

//writing std::ofstream f = /*open a file*/; int a = 5, b = -1, c = 123456; f << bits(a) << bits(b) << bits(c); //reading std::ifstream f2 = /*open a file*/; int a, b, c; f >> bits(a) >> bits(b) >> bits(c);
Improve this answer

2 Comments

I prefer this answer because it's not confusing and it can also wrap other things like vector<float>
Hey, @SamuelPowell I like this approach so much, I took it a bit further, and wrote more serializers on top of this approach. I like this as it has such low complexity compared to other C++ serializers. If interested look at github.com/goblinhack/simple-c-plus-plus-serializer - would be interested in your comments. I found I had to remove the stream type in the templates due to (I think) operator overloading issues. Anyway, it works well for many types.
6

Well, just use characters, not integers.

s << char(1) << char(2) << char(3);
Improve this answer

Comments

4

overloading some unusual operators works rather well. Here below I choosed to overload <= because it has the same left-to-right associativity as << and has somehow a close look-and-feel ...

#include <iostream> #include <stdint.h> #include <arpa/inet.h> using namespace std; ostream & operator<= (ostream& cout, string const& s) { return cout.write (s.c_str(), s.size()); } ostream & operator<= (ostream& cout, const char *s) { return cout << s; } ostream & operator<= (ostream&, int16_t const& i) { return cout.write ((const char *)&i, 2); } ostream & operator<= (ostream&, int32_t const& i) { return cout.write ((const char *)&i, 4); } ostream & operator<= (ostream&, uint16_t const& i) { return cout.write ((const char *)&i, 2); } ostream & operator<= (ostream&, uint32_t const& i) { return cout.write ((const char *)&i, 4); } int main() { string s("some binary data follow : "); cout <= s <= " (machine ordered) : " <= (uint32_t)0x31323334 <= "\n" <= s <= " (network ordered) : " <= htonl(0x31323334) ; cout << endl; return 0; }

There are several drawbacks :

  • the new meaning of <= may confuse readers or lead to unexpected results :

    cout <= 31 <= 32;

    won't give the same result as

    cout <= (31 <= 32);

  • the endianess isn't clearly mentionned at reading the code, as illustrated in the above example.

  • it cannot mix simply with << because it doesn't belong to the same group of precedence. I usually use parenthesis to clarify such as :

    ( cout <= htonl(a) <= htonl(b) ) << endl;
Improve this answer

1 Comment

That's a cool proof of concept, but note that C++'s overloaded operators are considered evil because they allow for this. The non-obvious overload of << is justified only because it's a standard overload. No new hacky overloads should be invented and the overloading itself should be used with a great care.
2

For this use case I implemented myself a "raw shift operator":

template <typename T, class... StreamArgs> inline std::basic_ostream<StreamArgs...> & operator <= (std::basic_ostream<StreamArgs...> & out, T const & data) { out.write(reinterpret_cast<char const *>(&data), sizeof(T)); return out; }

Put it somewhere convenient and use it like this:

std::cout <= 1337 <= 1337ULL <= 1337. <= 1337.f;

Advantages:

  • chainable
  • automatic sizeof()
  • takes arrays and struct/class instances, too

Disadvantages:

  • unsafe for non-POD objects: leaks pointers and padding
  • output is platform specific: padding, endianess, integer types
Improve this answer

Comments

1

I really like Han Luo's approach and have verified it to work great! If changing the oss member variable to use std::stringstream (vs. ostringstream) this class could also be used for extraction with overloaded stream extraction operators like this:

 template <typename T, typename std::enable_if<std::is_fundamental<T>::value, bool>::type = true> bostringstream& operator>> (T& v) { char buffer[sizeof(T)]; oss.read(buffer, sizeof(T)); v = *(T*)buffer; return *this; }

The example templates support integral types and if adding a new template for std::is_compound it's possible to also support types like std::map. For things like std::vector, that are "is_fundemental" I'd suggest pushing the size to the stream first, so on the extraction side it can be pulled to know how many elements to pull afterwards. This approach can work with the common std::vector and std::map types quite nicely.

Improve this answer

Comments

1 
#include <sstream> class bostringstream { public: bostringstream() : oss() {} template <typename T, typename std::enable_if<std::is_fundamental<T>::value, bool>::type = true> bostringstream& operator<<(const T& v) { oss.write((char*)&v, sizeof(T)); return *this; } template <typename T, typename std::enable_if< std::is_fundamental<typename T::value_type>::value, bool>::type = true> bostringstream& operator<<(const T& v) { oss.write((char*)v.data(), v.size() * sizeof(typename T::value_type)); return *this; } template <typename _InputIterator> bostringstream& write(_InputIterator first, _InputIterator last) { char* data = (char*)&(*first); auto n = std::distance(first, last); oss.write(data, n * sizeof(*first)); return *this; } template <typename T, typename std::enable_if<std::is_fundamental<T>::value, bool>::type = true> bostringstream& write(const T* v, std::streamsize count) { oss.write((char*)v, sizeof(T) * count); return *this; } auto rdbuf() const { return oss.rdbuf(); } auto str() const { return oss.str(); } std::size_t size() { return oss.tellp(); } protected: std::ostringstream oss; };

Example:

#include <array> #include <string> #include <vector> #include <iostream> #include <fstream> #include "bsstream.hpp" int main(int argc, char **argv) { int i = 1; float j = 1.1; double k = 1.2; std::vector<int> ii{1,2}; std::vector<double> jj{1.2,2.2}; std::string kk = "abcd"; std::array<int, 2> ll{3,4}; int l[] = {1,2}; bostringstream of; of << i << j <<k; of <<ii << jj << kk << ll; of.write(l, 2); std::ofstream oof("foo.bin", std::ios::binary); oof << of.str(); oof.close(); }

Not an elegant solution but works and is flexible

Edit:

  • Nov 12th, 2023: I have used similar code in my project for a while. Instead of using a std::ostringstream as a class member, I pass a pointer to type std::ostream to the class and do the writing part. It in general works fine.
Improve this answer

Comments

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.