binary write of std::string and std::vector

Discussion in 'C++' started by Leslaw Bieniasz, Feb 21, 2008.

  1. Hi,

    I need to make binary write/read of std::string and std::vector
    to a file (by using ofstream/ifstream). Is there any quick way of
    doing this, or one has to write/read the contents element by element
    by using ofstream::write(buffer,size)?

    L.B.


    !!! PLEASE NOTE MY NEW ADDRESS SINCE January 1st, 2008, INDICATED BELOW !!!
    *-------------------------------------------------------------------*
    | Dr. Leslaw Bieniasz, |
    | Institute of Physical Chemistry of the Polish Academy of Sciences,|
    | Department of Complex Systems and Chemical |
    | Processing of Information |
    | ul. Niezapominajek 8, 30-239 Cracow, Poland. |
    | tel. (room) +48 (12) 6395212 |
    | tel./fax. (secretariat) +48 (12) 4251923 |
    | E-mail: |
    *-------------------------------------------------------------------*
    | Interested in Computational Electrochemistry? |
    | Visit my web site: http://www.cyf-kr.edu.pl/~nbbienia |
    *-------------------------------------------------------------------*
     
    Leslaw Bieniasz, Feb 21, 2008
    #1
    1. Advertising

  2. Leslaw Bieniasz

    Guest

    On Feb 21, 1:49 pm, Leslaw Bieniasz <> wrote:
    > Hi,
    >
    > I need to make binary write/read of std::string and std::vector
    > to a file (by using ofstream/ifstream). Is there any quick way of
    > doing this, or one has to write/read the contents element by element
    > by using ofstream::write(buffer,size)?


    If you have:
    ofstream OUT;
    vector<int> vec;
    string str;

    then try:
    copy(vec.begin(), vec.end(), ostreambuf_iterator<int>(OUT));
    copy(str.begin(), str.end(), ostreambuf_iterator<char>(OUT));

    You might need to include the standard header <algorithm>.

    S.
     
    , Feb 21, 2008
    #2
    1. Advertising

  3. Leslaw Bieniasz

    Sanyi Guest

    On Feb 21, 4:01 pm, wrote:

    > copy(vec.begin(), vec.end(), ostreambuf_iterator<int>(OUT));


    I am sorry, a correction:
    ostreambuf_iterator<T>
    works for T being char and wchar_t only, so you can't use it here.
    What is the type of your vector's elements?

    S.
     
    Sanyi, Feb 21, 2008
    #3
  4. Leslaw Bieniasz

    Fred Zwarts Guest

    "Leslaw Bieniasz" <> wrote in message news:p-kr.edu.pl...
    >
    >
    > Hi,
    >
    > I need to make binary write/read of std::string and std::vector
    > to a file (by using ofstream/ifstream). Is there any quick way of
    > doing this, or one has to write/read the contents element by element
    > by using ofstream::write(buffer,size)?


    Suppose you have:
    ofstream Out;
    vector<int> Vec;
    string Str;

    What about something like

    Out.write (&*Vec.begin (), Vec.size () * sizeof (int));
    Out.write (Str.c_str (), Str.size ());

    or

    Out.write (&Vec[0], Vec.size () * sizeof (int));
    Out.write (&Str[0], Str.size ());
     
    Fred Zwarts, Feb 21, 2008
    #4
  5. On 2008-02-21 12:49, Leslaw Bieniasz wrote:
    >
    > Hi,
    >
    > I need to make binary write/read of std::string and std::vector
    > to a file (by using ofstream/ifstream). Is there any quick way of
    > doing this, or one has to write/read the contents element by element
    > by using ofstream::write(buffer,size)?


    See the write() and read() functions of ofstream/ifstream respectively.
    I'm not 100% sure that string guarantees that the chars are stored
    contiguously but on most implementations they will be. I also assume
    that you know about the dangers of binary reading and writing objects of
    non-POD types (and have thought of portability data portability).

    --
    Erik Wikström
     
    Erik Wikström, Feb 21, 2008
    #5
  6. Leslaw Bieniasz

    Jeff Schwab Guest

    Erik Wikström wrote:
    > On 2008-02-21 12:49, Leslaw Bieniasz wrote:
    >> Hi,
    >>
    >> I need to make binary write/read of std::string and std::vector
    >> to a file (by using ofstream/ifstream). Is there any quick way of
    >> doing this, or one has to write/read the contents element by element
    >> by using ofstream::write(buffer,size)?

    >
    > See the write() and read() functions of ofstream/ifstream respectively.
    > I'm not 100% sure that string guarantees that the chars are stored
    > contiguously


    Nope. That's why we have std::string::c_str().

    > but on most implementations they will be.


    So c_str() will be cheap. :)

    > I also assume
    > that you know about the dangers of binary reading and writing objects of
    > non-POD types (and have thought of portability data portability).
     
    Jeff Schwab, Feb 21, 2008
    #6
  7. Leslaw Bieniasz

    James Kanze Guest

    On Feb 21, 4:49 pm, "Fred Zwarts" <> wrote:
    > "Leslaw Bieniasz" <> wrote in
    > messagenews:p-kr.edu.pl...


    > > I need to make binary write/read of std::string and std::vector
    > > to a file (by using ofstream/ifstream). Is there any quick way of
    > > doing this, or one has to write/read the contents element by element
    > > by using ofstream::write(buffer,size)?


    > Suppose you have:
    > ofstream Out;
    > vector<int> Vec;
    > string Str;


    > What about something like


    > Out.write (&*Vec.begin (), Vec.size () * sizeof (int));
    > Out.write (Str.c_str (), Str.size ());


    > or


    > Out.write (&Vec[0], Vec.size () * sizeof (int));
    > Out.write (&Str[0], Str.size ());


    These will work if you don't count on ever having to read the
    data. Otherwise, I wouldn't recommend them. (Formally, what
    you're doing with string is undefined behavior according to the
    current standard. But all actual implementations do use
    contiguous memory, and the next version of the standard will
    require it, so that aspect is probably OK.)

    --
    James Kanze (GABI Software) email:
    Conseils en informatique orientée objet/
    Beratung in objektorientierter Datenverarbeitung
    9 place Sémard, 78210 St.-Cyr-l'École, France, +33 (0)1 30 23 00 34
     
    James Kanze, Feb 22, 2008
    #7
  8. Leslaw Bieniasz

    James Kanze Guest

    On Feb 21, 8:59 pm, Jeff Schwab <> wrote:
    > Erik Wikström wrote:
    > > On 2008-02-21 12:49, Leslaw Bieniasz wrote:


    > >> I need to make binary write/read of std::string and std::vector
    > >> to a file (by using ofstream/ifstream). Is there any quick way of
    > >> doing this, or one has to write/read the contents element by element
    > >> by using ofstream::write(buffer,size)?


    > > See the write() and read() functions of ofstream/ifstream respectively.
    > > I'm not 100% sure that string guarantees that the chars are stored
    > > contiguously


    > Nope. That's why we have std::string::c_str().


    The next version of the standard will guarantee it. And provide
    a non-const data() member function. In the meantime,
    &someString[] can be used if you need a non-const char*.

    Not that any of this is relevant to the problem at hand. The
    original poster didn't say what format the data had to be in, so
    we can't say how it should be written, but one thing is sure:
    using write() and read() aren't going to do the trick.

    More generally, the abstraction of [io]stream is text I/O. The
    write() and read() functions are there for the cases when you
    have already formatted text---note that they (correctly) take a
    char*, and not a void* (which would be the appropriate type if
    they were meant to work with any type of data). If you want
    binary I/O, you probably have to define your own equivalents of
    [io]stream, something like [io]xdrstream, [io]berstream, or
    whatever.

    > > I also assume that you know about the dangers of binary
    > > reading and writing objects of non-POD types (and have
    > > thought of portability data portability).


    There's no danger of binary reading and writing, if you do it
    correctly; I've used BER format extensively in the past.

    Of course, there's a great danger in doing anything if you don't
    know what you're doing. (One of the other responders suggested
    "write( s.c_str(), s.size() )", for example. Which can never be
    made to work.)

    --
    James Kanze (GABI Software) email:
    Conseils en informatique orientée objet/
    Beratung in objektorientierter Datenverarbeitung
    9 place Sémard, 78210 St.-Cyr-l'École, France, +33 (0)1 30 23 00 34
     
    James Kanze, Feb 22, 2008
    #8
  9. Leslaw Bieniasz

    Fred Zwarts Guest

    "James Kanze" <> wrote in message news:...
    >On Feb 21, 8:59 pm, Jeff Schwab <> wrote:
    >> Erik Wikström wrote:
    >> > On 2008-02-21 12:49, Leslaw Bieniasz wrote:

    >
    >> >> I need to make binary write/read of std::string and std::vector
    >> >> to a file (by using ofstream/ifstream). Is there any quick way of
    >> >> doing this, or one has to write/read the contents element by element
    >> >> by using ofstream::write(buffer,size)?

    >
    >> > See the write() and read() functions of ofstream/ifstream respectively.
    >> > I'm not 100% sure that string guarantees that the chars are stored
    >> > contiguously

    >
    >> Nope. That's why we have std::string::c_str().

    >
    >The next version of the standard will guarantee it. And provide
    >a non-const data() member function. In the meantime,
    >&someString[] can be used if you need a non-const char*.
    >
    >Not that any of this is relevant to the problem at hand. The
    >original poster didn't say what format the data had to be in, so
    >we can't say how it should be written, but one thing is sure:
    >using write() and read() aren't going to do the trick.


    Can you elaborate on this? I use read and write often to handle
    binary data I/O so I don't understand which trick is not done.

    >More generally, the abstraction of [io]stream is text I/O.


    Can you point to places in the standard from which you conclude this?
    The iostream open function explicitly had the option to open files for
    unformatted I/O. For me unformatted is equivalent to binary, so I
    assumed that binary I/O was OK.

    > The write() and read() functions are there for the cases when you
    >have already formatted text


    Can you point to places in the standard to support this assumtion?

    >---note that they (correctly) take a
    >char*, and not a void* (which would be the appropriate type if
    >they were meant to work with any type of data).


    Since the read and write function need a size parameter,
    I thought a char* type is more appropriate, because it makes clear
    that the size is calculated in units of char.
    The void type has no size association.

    >If you want
    >binary I/O, you probably have to define your own equivalents of
    >[io]stream, something like [io]xdrstream, [io]berstream, or
    >whatever.


    I have not yet experience the need for it. Although I frequently use
    binary I/O.
     
    Fred Zwarts, Feb 25, 2008
    #9
  10. Leslaw Bieniasz

    James Kanze Guest

    On Feb 25, 10:11 am, "Fred Zwarts" <> wrote:
    > "James Kanze" <> wrote in messagenews:...


    [...]
    > >Not that any of this is relevant to the problem at hand. The
    > >original poster didn't say what format the data had to be in, so
    > >we can't say how it should be written, but one thing is sure:
    > >using write() and read() aren't going to do the trick.


    > Can you elaborate on this? I use read and write often to
    > handle binary data I/O so I don't understand which trick is
    > not done.


    In what format? istream::read and ostream::write don't do any
    formatting---just a dump of the bits "as they lay". Since the
    actual bit representations are not defined (except for unsigned
    char), and in fact vary enormously, that's obviously only useful
    if you've already formatted the data to some specification
    before hand.

    > >More generally, the abstraction of [io]stream is text I/O.


    > Can you point to places in the standard from which you
    > conclude this?


    The fact that that's what they do, basically. They format and
    parse text. They have additional functions to support reading
    an writing of pre-formatted text, but no functions which support
    output of e.g. int or double in a binary format. They do locale
    dependent *text* code translation.

    > The iostream open function explicitly had the option to open
    > files for unformatted I/O.


    No it doesn't. It has an option to allow supporting two
    different file formats, but that has nothing to do with the data
    format.

    > For me unformatted is equivalent to binary, so I assumed that
    > binary I/O was OK.


    There is no such thing as "unformatted". All data has some
    format. The question is simply whether you know the format, or
    whether you're outputting basically random data.

    > > The write() and read() functions are there for the cases
    > > when you
    > >have already formatted text


    > Can you point to places in the standard to support this
    > assumtion?


    The specifications of the functions themselves. They don't do
    *any* formatting, what so ever. (Also, the fact that they take
    a char*, rather than a void*, is significant.)

    > >---note that they (correctly) take a char*, and not a void*
    > >(which would be the appropriate type if they were meant to
    > >work with any type of data).


    > Since the read and write function need a size parameter, I
    > thought a char* type is more appropriate, because it makes
    > clear that the size is calculated in units of char. The void
    > type has no size association.


    And? The void type is for raw data, of unspecified type. The
    char type is for char's.

    > >If you want binary I/O, you probably have to define your own
    > >equivalents of [io]stream, something like [io]xdrstream,
    > >[io]berstream, or whatever.


    > I have not yet experience the need for it. Although I
    > frequently use binary I/O.


    If you reread the data using the same executable, it will
    generally work. Recompile with a different version of the
    compiler, different compiler options, etc., and it may or may
    not work. Try reading the data on another machine, and it
    almost certainly won't work.

    Unless you write your data with a defined format, you're
    basically creating a time bomb, guaranteeing that at some time
    in the future, it will become unreadable. Obviously, this is
    acceptable in some cases: temporary files reread by the same
    executable, and deleted at the end of the run, for example. Or
    even saved game files, of such---I don't think most games have
    to worry about rereading game files after an upgrade. But it's
    certainly not acceptable for any business data---depending on
    the data and the jurisdiction, it may even be a violation of the
    law. (I know that we're required by law to maintain much of our
    data for 50 years or more. "Maintain" meaning not just that
    some bits which represent the data are there, but that we can
    actually re-read them.)

    --
    James Kanze (GABI Software) email:
    Conseils en informatique orientée objet/
    Beratung in objektorientierter Datenverarbeitung
    9 place Sémard, 78210 St.-Cyr-l'École, France, +33 (0)1 30 23 00 34
     
    James Kanze, Feb 26, 2008
    #10
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Anonymous
    Replies:
    20
    Views:
    4,427
    Pete Becker
    Mar 30, 2005
  2. Jason Heyes
    Replies:
    8
    Views:
    767
    Andrew Koenig
    Jan 15, 2006
  3. Replies:
    8
    Views:
    1,998
    Csaba
    Feb 18, 2006
  4. Rune Allnor
    Replies:
    4
    Views:
    992
    Rune Allnor
    Dec 11, 2008
  5. John H.
    Replies:
    0
    Views:
    667
    John H.
    Mar 1, 2010
Loading...

Share This Page