ostream implementations (str, f, cout ) work slightly differently

Discussion in 'C++' started by Ingo Nolden, Apr 26, 2004.

  1. Ingo Nolden

    Ingo Nolden Guest

    Hi there,
    I am writing c++ for some months now. I think I know the language now, but not yet all the tricky things of stl.

    For a kernel which is not using mfc I am writing a serialization. For some reasons I want to implement it on my
    own.
    One goal is to make it robust against version changes and corrupted files. That means, when an object cannot
    be deserialized, it should be skipped. The validity of the remaining document can be verified later.
    For this purpose I begin every object with a code that identifies the class. A factory can use it to construct
    the class, giving it the node of the serialized stream.
    The next is the size of the object. This way the object can be skipped if something is wrong. An earlier version
    might not contain all the variables for a class, a later one may contain too much. The serializer or deserializer
    methods of the class will have apropriate default values at hand.
    The problem is that the size is of course unknown before th object is already serialized. For performance
    matters I dont want to have a kind of size inquiry before the serialization and I dont want something to serialize
    to another place and then copy to the real stream. Even though my files might not be going to be soo big, I
    don't know what comes in the future.
    So my way is to save the current stream position before the serialization, and then jump back and forth to put
    the size in the right place.
    My first question is: Does anyone now how costly that is? I guess not too much, but I don't really knwo about
    the implementation of the streams.
    Then I see that it sometimes doesn't behave as expected.
    Ok, the cout can get the position it returns only zero. No problem since serializing to cout is not really useful
    to me.
    one might ask why I want to use this general ostream to serialize to. Well I dont know what it is going to do
    later: saving docs, transaction ( undo/redo), network communication or just with another process, dragn drop...
    therefore I think a general apporoach is ok.
    Now the file stream sometime puts the very first int size not into the first place but shifted about one byte. THis
    is weird, ugly and it corrupts the actual data. But it depends on the size of the actual data. It happens if it is
    small, and depends on wether the size is odd or even !?! I tried to overcome this with an initializing "initialize"
    string wich will be known and therefore needs no size info. But in this case the problem appears after this
    initilaizing string. So it is something with my jumping around with the seekp() method.
    The strstream is a little similar but the first data value is not corrupted. But the whole data is shifted by one bit
    :-(.
    So if one is really familiar with the streams: I just want to know what do I have to consider to make my code
    work securely. It should not only work no, but in any case on any machine and on compilers different than my
    VS.net2003 compiler
    thanks for your help in advance. I tried to post source, but the server rejected. Probably too big. Ask if
    necessary and I will post it
    cheers
    Ingo
     
    Ingo Nolden, Apr 26, 2004
    #1
    1. Advertising

  2. "Ingo Nolden" <> wrote in message
    news:c6jmoj$84v$-online.net...
    >
    > Hi there,
    > I am writing c++ for some months now. I think I know the language now, but

    not yet all the tricky things of stl.
    >
    > For a kernel which is not using mfc I am writing a serialization. For some

    reasons I want to implement it on my
    > own.
    > One goal is to make it robust against version changes and corrupted files.

    That means, when an object cannot
    > be deserialized, it should be skipped. The validity of the remaining

    document can be verified later.
    > For this purpose I begin every object with a code that identifies the

    class. A factory can use it to construct
    > the class, giving it the node of the serialized stream.
    > The next is the size of the object. This way the object can be skipped if

    something is wrong. An earlier version
    > might not contain all the variables for a class, a later one may contain

    too much. The serializer or deserializer
    > methods of the class will have apropriate default values at hand.
    > The problem is that the size is of course unknown before th object is

    already serialized. For performance
    > matters I dont want to have a kind of size inquiry before the

    serialization and I dont want something to serialize
    > to another place and then copy to the real stream. Even though my files

    might not be going to be soo big, I
    > don't know what comes in the future.
    > So my way is to save the current stream position before the serialization,

    and then jump back and forth to put
    > the size in the right place.
    > My first question is: Does anyone now how costly that is? I guess not too

    much, but I don't really knwo about
    > the implementation of the streams.
    > Then I see that it sometimes doesn't behave as expected.
    > Ok, the cout can get the position it returns only zero. No problem since

    serializing to cout is not really useful
    > to me.
    > one might ask why I want to use this general ostream to serialize to. Well

    I dont know what it is going to do
    > later: saving docs, transaction ( undo/redo), network communication or

    just with another process, dragn drop...
    > therefore I think a general apporoach is ok.
    > Now the file stream sometime puts the very first int size not into the

    first place but shifted about one byte. THis
    > is weird, ugly and it corrupts the actual data. But it depends on the size

    of the actual data. It happens if it is
    > small, and depends on wether the size is odd or even !?!


    Probably you have not opened the file in binary mode. Absolute positioning
    of file streams is not possible unless you do this

    E.g.

    ostream file("somefile.txt", ios_base::eek:ut|ios_base::binary);

    john
     
    John Harrison, Apr 26, 2004
    #2
    1. Advertising

  3. >
    > E.g.
    >
    > ostream file("somefile.txt", ios_base::eek:ut|ios_base::binary);
    >


    ofstream of course.

    john
     
    John Harrison, Apr 26, 2004
    #3
  4. Ingo Nolden

    Ingo Nolden Guest

    thank you

    some times things are simple.
    the file thing is working now as expected.

    Is there such sthing as a binary mode for the strstream, too? Or is there some template related to the
    strstream that I can use for binary data. It works nearly perfect just occasionally it shifts by a byte. Of coure I
    could find a way to synchronize it somehow, even though I dont think this is how it should be done.

    cheers



    "John Harrison"
    wrote in message: <c6js94$cqfhn$-berlin.de>

    >>
    >> E.g.
    >>
    >> ostream file("somefile.txt", ios_base::eek:ut|ios_base::binary);
    >>

    >
    >ofstream of course.
    >
    >john
    >
     
    Ingo Nolden, Apr 26, 2004
    #4
  5. "Ingo Nolden" <> wrote in message
    news:c6jvrt$b5h$-online.net...
    > thank you
    >
    > some times things are simple.
    > the file thing is working now as expected.
    >
    > Is there such sthing as a binary mode for the strstream, too? Or is there

    some template related to the
    > strstream that I can use for binary data. It works nearly perfect just

    occasionally it shifts by a byte. Of coure I
    > could find a way to synchronize it somehow, even though I dont think this

    is how it should be done.
    >
    > cheers
    >


    I don't believe there is a binary mode for strstream. Text mode/binary mode
    is C++'s way of dealing with different line ending conventions in different
    OS. Obviously that only applies to files. I'd say that what you are seeing
    in strstream is probably just a bug in your code. Why not post some code
    that reproduces the problem?

    BTW stringstream is usually preferred to strstream, but that has nothing to
    do with binary data.

    john
     
    John Harrison, Apr 27, 2004
    #5
  6. Ingo Nolden

    Ingo Nolden Guest

    I can hardly believe it is the code, but I try to put it in here. Last time the server rejected. I think my
    newsreader is doing something strange with the includes thinking they were an xml token...

    >in strstream is probably just a bug in your code. Why not post some code
    >that reproduces the problem?
    >



    >BTW stringstream is usually preferred to strstream, but that has nothing to
    >do with binary data.


    I thried this also. But derivation of the stringstream seams a little different to strstream as the function that
    takes the ostream will also take the strstream and the ofstream but not the stringstream nor the
    ostringstream. I have to admit that I don't really look through the inheritance of the stream classes

    So here my code:
    I replace the include brackets with " for my proggie

    #include "strstream"
    #include "iostream"
    #include "fstream"

    using namespace std;

    void f( int i );

    void f( int i)
    {

    }
    class CleverStream
    {
    public:
    CleverStream( ostream& str )
    {
    s = &str;
    // check if position can be set:

    int i = s->tellp( );

    if( i == -1 )
    throw exception( "Invalid stream" );

    }


    void SaveData( char* sData )
    {
    if( s == &cout )
    s->seekp( 0 );
    unsigned int i = 0;
    unsigned int dwBefore = s->tellp( );
    s->write( (char*)&i, 4 );// << (char[8])&i; // placeholder int
    *s << sData;
    unsigned int dwAfter = s->tellp( );
    s->seekp( dwBefore );
    i = dwAfter - dwBefore;
    s->write( (char*)&i, 4 );
    // s->flush( );
    s->seekp( dwAfter );
    }
    void Initialize( char* sData )
    {
    *s << sData;
    }

    ostream* s;

    };
    void PutThings( CleverStream& s )
    {
    // s.Initialize( "Initializer" );
    s.SaveData( "Khadij" );
    s.SaveData( "Khadija und Ingo machen einen ganz langen Satzg" );
    s.SaveData( "S" );
    s.SaveData( "Fi" );
    s.SaveData( "Dep" );
    s.SaveData( "Subj" );
    s.SaveData( "Final" );
    }

    int main( )
    {

    try
    {
    // ofstream x( "iotest.txt" );
    ofstream x( "iotest.txt", ios_base::eek:ut|ios_base::binary );
    CleverStream s_file( x );
    PutThings( s_file );
    }
    catch ( exception e )
    {
    cout << e.what( );
    }

    try
    {
    CleverStream s_cout( cout );

    PutThings( s_cout );
    }
    catch ( exception e )
    {
    cout << e.what( );
    }

    try
    {
    cout << endl << endl << endl;
    char buf[100];
    ostringstream x_str( );
    ostringstream y_str( );
    //strstream x_str( buf, 100 );
    //strstream y_str( buf, 100 );
    //strstream x_str( buf, 100 , ios_base::eek:ut|ios_base::binary);
    //strstream y_str( buf, 100, ios_base::eek:ut|ios_base::binary );

    CleverStream s_str( x_str );

    PutThings( s_str );

    ofstream y( "strtest.txt" );

    y.write( x_str.str( ), x_str.pcount( ) );
    }
    catch ( exception e )
    {
    cout << e.what( );
    }

    return 0;
    }
     
    Ingo Nolden, Apr 27, 2004
    #6
  7. "Ingo Nolden" <> wrote in message
    news:c6mjhm$2ro$-online.net...
    > I can hardly believe it is the code, but I try to put it in here. Last

    time the server rejected. I think my
    > newsreader is doing something strange with the includes thinking they were

    an xml token...
    >
    > >in strstream is probably just a bug in your code. Why not post some code
    > >that reproduces the problem?
    > >

    >
    >
    > >BTW stringstream is usually preferred to strstream, but that has nothing

    to
    > >do with binary data.

    >
    > I thried this also. But derivation of the stringstream seams a little

    different to strstream as the function that
    > takes the ostream will also take the strstream and the ofstream but not

    the stringstream nor the
    > ostringstream. I have to admit that I don't really look through the

    inheritance of the stream classes
    >


    That's easily fixed, see comments below.

    > So here my code:
    > I replace the include brackets with " for my proggie
    >
    > #include "strstream"
    > #include "iostream"
    > #include "fstream"
    >
    > using namespace std;
    >
    > void f( int i );
    >
    > void f( int i)
    > {
    >
    > }
    > class CleverStream
    > {
    > public:
    > CleverStream( ostream& str )
    > {
    > s = &str;
    > // check if position can be set:
    >
    > int i = s->tellp( );
    >
    > if( i == -1 )
    > throw exception( "Invalid stream" );
    >
    > }
    >
    >
    > void SaveData( char* sData )
    > {
    > if( s == &cout )
    > s->seekp( 0 );
    > unsigned int i = 0;
    > unsigned int dwBefore = s->tellp( );
    > s->write( (char*)&i, 4 );// << (char[8])&i; // placeholder int
    > *s << sData;
    > unsigned int dwAfter = s->tellp( );
    > s->seekp( dwBefore );
    > i = dwAfter - dwBefore;
    > s->write( (char*)&i, 4 );
    > // s->flush( );
    > s->seekp( dwAfter );
    > }
    > void Initialize( char* sData )
    > {
    > *s << sData;
    > }
    >
    > ostream* s;
    >
    > };
    > void PutThings( CleverStream& s )
    > {
    > // s.Initialize( "Initializer" );
    > s.SaveData( "Khadij" );
    > s.SaveData( "Khadija und Ingo machen einen ganz langen Satzg" );
    > s.SaveData( "S" );
    > s.SaveData( "Fi" );
    > s.SaveData( "Dep" );
    > s.SaveData( "Subj" );
    > s.SaveData( "Final" );
    > }
    >
    > int main( )
    > {
    >
    > try
    > {
    > // ofstream x( "iotest.txt" );
    > ofstream x( "iotest.txt", ios_base::eek:ut|ios_base::binary );
    > CleverStream s_file( x );
    > PutThings( s_file );
    > }
    > catch ( exception e )
    > {
    > cout << e.what( );
    > }
    >
    > try
    > {
    > CleverStream s_cout( cout );
    >
    > PutThings( s_cout );
    > }
    > catch ( exception e )
    > {
    > cout << e.what( );
    > }
    >
    > try
    > {
    > cout << endl << endl << endl;
    > char buf[100];
    > ostringstream x_str( );
    > ostringstream y_str( );


    I think you mean

    ostringstream x_str;
    ostringstream y_str;

    What you had is a function prototype, not a variable declaration.

    You also need to change the header from "strstream" to "sstream".

    > //strstream x_str( buf, 100 );
    > //strstream y_str( buf, 100 );
    > //strstream x_str( buf, 100 , ios_base::eek:ut|ios_base::binary);
    > //strstream y_str( buf, 100, ios_base::eek:ut|ios_base::binary );
    >
    > CleverStream s_str( x_str );
    >
    > PutThings( s_str );
    >
    > ofstream y( "strtest.txt" );
    >
    > y.write( x_str.str( ), x_str.pcount( ) );


    With strstream this is a memory leak, its one of the main arguments against
    strstream that after calling str() you have to free the returned pointer.

    For stringstream this becomes

    y.write( x_str.str( ).c_str(), x_str.str( ),size() );

    and no memory leak.

    However even with these fixes I couldn't get ostringstream to work. The
    problem is your test to see if a stream is positionable. On my system seekp
    returns -1 before any output has happened even though ostringstream is
    positionable (the reason being that the output buffer isn't actually
    allocated until some output has occurred). I'm not sure if this is correct
    behaviour or not. If you remove the test, or if you do some output before
    making the test then everything works.

    john
     
    John Harrison, Apr 28, 2004
    #7
  8. Ingo Nolden

    Ingo Nolden Guest

    thank you John for your efforts. Indeed it works nearly in any case. Unless the first "counted" output has
    exaclty 6 bytes. Any other number of bytes is OK. But with 6 bytes an additional 0x0D ( 14 ) will be inserted
    between my dummy intitilaizing string and the number.
    The weird thing is that it doesn't happen where I expected. I walked through every single step that the
    stringstream does on its buffer and I see that everything works fine. So it must be the writing to the output file.
    But I think it is just a matter of copying some bytes. The output file can not know that there were words and
    integers, does it? I also see that the output file is not actually written unless my program returns control to
    windows. Now it seems it is a problem not directly related to what kind of stream I use but how I put the
    content into a file.
    Another question is. Since the stringstream has no constructer to tell the initial buffer size or that takes a
    buffer and it has no reserve method, how can I force a initial buffer size to avoid too much copying. For the
    case lets say I know the approximate amount of data and just want to make the buffer a little greater.

    this is my current listing:


    using namespace std;

    void f( int i );

    void f( int i)
    {

    }
    class CleverStream
    {
    public:
    CleverStream( ostream& str )
    {
    s = &str;

    }


    void SaveData( char* sData )
    {
    // check if position can be set:

    //int ip = s->tellp( );

    //if( ip == -1 )
    // throw exception( "Invalid stream" );

    if( s == &cout )
    s->seekp( 0 );
    unsigned int i = 0;
    unsigned int dwBefore = s->tellp( );
    s->write( (char*)&i, 4 );// << (char[8])&i; // placeholder int
    *s << sData;
    unsigned int dwAfter = s->tellp( );
    s->seekp( dwBefore );
    i = dwAfter - dwBefore;
    s->write( (char*)&i, 4 );
    // s->flush( );
    s->seekp( dwAfter );
    }
    void Initialize( char* sData )
    {
    *s << sData;
    }

    ostream* s;

    };
    void PutThings( CleverStream& s )
    {
    s.Initialize( "Initialize" );
    s.SaveData( "Khadij" );// 6,
    s.SaveData( "Khadija und Ingo machen einen ganz langen Satzg" );
    s.SaveData( "S" );
    s.SaveData( "Fi" );
    s.SaveData( "Dep" );
    s.SaveData( "Subj" );
    s.SaveData( "Final" );
    }

    int main( )
    {

    try
    {
    // ofstream x( "iotest.txt" );
    ofstream x( "iotest.txt", ios_base::eek:ut|ios_base::binary );
    CleverStream s_file( x );
    PutThings( s_file );
    }
    catch ( exception e )
    {
    cout << e.what( );
    }

    try
    {
    CleverStream s_cout( cout );

    PutThings( s_cout );
    }
    catch ( exception e )
    {
    cout << e.what( );
    }

    try
    {
    cout << endl << endl << endl;
    char buf[100];
    ostringstream x_str;
    ostringstream y_str;
    //strstream x_str( buf, 100 );
    //strstream y_str( buf, 100 );
    //strstream x_str( buf, 100 , ios_base::eek:ut|ios_base::binary);
    //strstream y_str( buf, 100, ios_base::eek:ut|ios_base::binary );

    CleverStream s_str( x_str );

    PutThings( s_str );

    ofstream y( "strtest.txt" );

    //y.write( x_str.str( ), x_str.tellp( ) );
    y.write( x_str.str( ).c_str(), x_str.str( ).size() );

    }
    catch ( exception e )
    {
    cout << e.what( );
    }

    return 0;
    }
     
    Ingo Nolden, Apr 30, 2004
    #8
  9. Ingo Nolden

    Buster Guest

    Ingo Nolden wrote:
    > thank you John for your efforts. Indeed it works nearly in any case. Unless the first "counted" output has
    > exaclty 6 bytes. Any other number of bytes is OK. But with 6 bytes an additional 0x0D ( 14 ) will be inserted
    > between my dummy intitilaizing string and the number.


    0x0D = 13, carriage return. Open the file in binary, not text, mode.

    --
    Regards,
    Buster.
     
    Buster, Apr 30, 2004
    #9
  10. Ingo Nolden

    Ingo Nolden Guest

    Ok, thank you

    that was it. Its my stupid fault, because I had the same thing when I wrote directly to the filestream.

    But, I wonder why it happens in a very sppecial case only and when does the stream apply carriage return?
    I mean it seemed to be in a operation of copying the whole stream.
    Anyway, I am glad to have this solved. Thank you all.

    Ingo
     
    Ingo Nolden, Apr 30, 2004
    #10
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. abi

    man cout or info cout

    abi, Jun 27, 2003, in forum: C++
    Replies:
    2
    Views:
    1,674
  2. Trevor
    Replies:
    2
    Views:
    537
  3. Pmb

    std::cout vs cout

    Pmb, Jun 2, 2004, in forum: C++
    Replies:
    2
    Views:
    4,449
    Leor Zolman
    Jun 2, 2004
  4. Replies:
    4
    Views:
    344
    Howard
    Sep 15, 2005
  5. Replies:
    2
    Views:
    1,975
Loading...

Share This Page