binary write of std::string and std::vector

[Leslaw Bieniasz]

Hi,

I need to make binary write/read of std::string and std::vector
to a file (by using ofstream/ifstream). Is there any quick way of
doing this, or one has to write/read the contents element by element
by using ofstream::write(buffer,size)?

L.B.


!!! PLEASE NOTE MY NEW ADDRESS SINCE January 1st, 2008, INDICATED BELOW !!!
*-------------------------------------------------------------------*
| Dr. Leslaw Bieniasz, |
| Institute of Physical Chemistry of the Polish Academy of Sciences,|
| Department of Complex Systems and Chemical |
| Processing of Information |
| ul. Niezapominajek 8, 30-239 Cracow, Poland. |
| tel. (room) +48 (12) 6395212 |
| tel./fax. (secretariat) +48 (12) 4251923 |
| E-mail: (e-mail address removed) |
*-------------------------------------------------------------------*
| Interested in Computational Electrochemistry? |
| Visit my web site: http://www.cyf-kr.edu.pl/~nbbienia |
*-------------------------------------------------------------------*

 

[gilia.benczik]

Hi,

I need to make binary write/read of std::string and std::vector
to a file (by using ofstream/ifstream). Is there any quick way of
doing this, or one has to write/read the contents element by element
by using ofstream::write(buffer,size)?

If you have:
ofstream OUT;
vector<int> vec;
string str;

then try:
copy(vec.begin(), vec.end(), ostreambuf_iterator<int>(OUT));
copy(str.begin(), str.end(), ostreambuf_iterator<char>(OUT));

You might need to include the standard header <algorithm>.

S.

 

[Sanyi]

copy(vec.begin(), vec.end(), ostreambuf_iterator<int>(OUT));

I am sorry, a correction:
ostreambuf_iterator<T>
works for T being char and wchar_t only, so you can't use it here.
What is the type of your vector's elements?

S.

 

[Fred Zwarts]

Hi,

I need to make binary write/read of std::string and std::vector
to a file (by using ofstream/ifstream). Is there any quick way of
doing this, or one has to write/read the contents element by element
by using ofstream::write(buffer,size)?

Suppose you have:
ofstream Out;
vector<int> Vec;
string Str;

What about something like

Out.write (&*Vec.begin (), Vec.size () * sizeof (int));
Out.write (Str.c_str (), Str.size ());

or

Out.write (&Vec[0], Vec.size () * sizeof (int));
Out.write (&Str[0], Str.size ());

 

[Erik Wikström]

Hi,

I need to make binary write/read of std::string and std::vector
to a file (by using ofstream/ifstream). Is there any quick way of
doing this, or one has to write/read the contents element by element
by using ofstream::write(buffer,size)?

See the write() and read() functions of ofstream/ifstream respectively.
I'm not 100% sure that string guarantees that the chars are stored
contiguously but on most implementations they will be. I also assume
that you know about the dangers of binary reading and writing objects of
non-POD types (and have thought of portability data portability).

 

[Jeff Schwab]

See the write() and read() functions of ofstream/ifstream respectively.
I'm not 100% sure that string guarantees that the chars are stored
contiguously

Nope. That's why we have std::string::c_str().

but on most implementations they will be.

So c_str() will be cheap.

 

[James Kanze]

messagenews[email protected]...

Suppose you have:
ofstream Out;
vector<int> Vec;
string Str;

What about something like

Out.write (&*Vec.begin (), Vec.size () * sizeof (int));
Out.write (Str.c_str (), Str.size ());

or

Out.write (&Vec[0], Vec.size () * sizeof (int));
Out.write (&Str[0], Str.size ());

These will work if you don't count on ever having to read the
data. Otherwise, I wouldn't recommend them. (Formally, what
you're doing with string is undefined behavior according to the
current standard. But all actual implementations do use
contiguous memory, and the next version of the standard will
require it, so that aspect is probably OK.)

 

[James Kanze]

Nope. That's why we have std::string::c_str().

The next version of the standard will guarantee it. And provide
a non-const data() member function. In the meantime,
&someString[] can be used if you need a non-const char*.

Not that any of this is relevant to the problem at hand. The
original poster didn't say what format the data had to be in, so
we can't say how it should be written, but one thing is sure:
using write() and read() aren't going to do the trick.

More generally, the abstraction of [io]stream is text I/O. The
write() and read() functions are there for the cases when you
have already formatted text---note that they (correctly) take a
char*, and not a void* (which would be the appropriate type if
they were meant to work with any type of data). If you want
binary I/O, you probably have to define your own equivalents of
[io]stream, something like [io]xdrstream, [io]berstream, or
whatever.

There's no danger of binary reading and writing, if you do it
correctly; I've used BER format extensively in the past.

Of course, there's a great danger in doing anything if you don't
know what you're doing. (One of the other responders suggested
"write( s.c_str(), s.size() )", for example. Which can never be
made to work.)

 

[Fred Zwarts]

Nope. That's why we have std::string::c_str().

The next version of the standard will guarantee it. And provide
a non-const data() member function. In the meantime,
&someString[] can be used if you need a non-const char*.

Not that any of this is relevant to the problem at hand. The
original poster didn't say what format the data had to be in, so
we can't say how it should be written, but one thing is sure:
using write() and read() aren't going to do the trick.

Can you elaborate on this? I use read and write often to handle
binary data I/O so I don't understand which trick is not done.

More generally, the abstraction of [io]stream is text I/O.

Can you point to places in the standard from which you conclude this?
The iostream open function explicitly had the option to open files for
unformatted I/O. For me unformatted is equivalent to binary, so I
assumed that binary I/O was OK.

The write() and read() functions are there for the cases when you
have already formatted text

Can you point to places in the standard to support this assumtion?

---note that they (correctly) take a
char*, and not a void* (which would be the appropriate type if
they were meant to work with any type of data).

Since the read and write function need a size parameter,
I thought a char* type is more appropriate, because it makes clear
that the size is calculated in units of char.
The void type has no size association.

If you want
binary I/O, you probably have to define your own equivalents of
[io]stream, something like [io]xdrstream, [io]berstream, or
whatever.

I have not yet experience the need for it. Although I frequently use
binary I/O.

 

[James Kanze]

"James Kanze" <[email protected]> wrote in messagenews:[email protected]...

[...]

Can you elaborate on this? I use read and write often to
handle binary data I/O so I don't understand which trick is
not done.

In what format? istream::read and ostream::write don't do any
formatting---just a dump of the bits "as they lay". Since the
actual bit representations are not defined (except for unsigned
char), and in fact vary enormously, that's obviously only useful
if you've already formatted the data to some specification
before hand.

More generally, the abstraction of [io]stream is text I/O.

Can you point to places in the standard from which you
conclude this?

The fact that that's what they do, basically. They format and
parse text. They have additional functions to support reading
an writing of pre-formatted text, but no functions which support
output of e.g. int or double in a binary format. They do locale
dependent *text* code translation.

The iostream open function explicitly had the option to open
files for unformatted I/O.

No it doesn't. It has an option to allow supporting two
different file formats, but that has nothing to do with the data
format.

For me unformatted is equivalent to binary, so I assumed that
binary I/O was OK.

There is no such thing as "unformatted". All data has some
format. The question is simply whether you know the format, or
whether you're outputting basically random data.

Can you point to places in the standard to support this
assumtion?

The specifications of the functions themselves. They don't do
*any* formatting, what so ever. (Also, the fact that they take
a char*, rather than a void*, is significant.)

Since the read and write function need a size parameter, I
thought a char* type is more appropriate, because it makes
clear that the size is calculated in units of char. The void
type has no size association.

And? The void type is for raw data, of unspecified type. The
char type is for char's.

If you want binary I/O, you probably have to define your own
equivalents of [io]stream, something like [io]xdrstream,
[io]berstream, or whatever.

I have not yet experience the need for it. Although I
frequently use binary I/O.

If you reread the data using the same executable, it will
generally work. Recompile with a different version of the
compiler, different compiler options, etc., and it may or may
not work. Try reading the data on another machine, and it
almost certainly won't work.

Unless you write your data with a defined format, you're
basically creating a time bomb, guaranteeing that at some time
in the future, it will become unreadable. Obviously, this is
acceptable in some cases: temporary files reread by the same
executable, and deleted at the end of the run, for example. Or
even saved game files, of such---I don't think most games have
to worry about rereading game files after an upgrade. But it's
certainly not acceptable for any business data---depending on
the data and the jurisdiction, it may even be a violation of the
law. (I know that we're required by law to maintain much of our
data for 50 years or more. "Maintain" meaning not just that
some bits which represent the data are there, but that we can
actually re-read them.)