The performance of strstream vs. stringstrem

[forrest]

I wrote a function like this:

#define LEN 8
void foo(ostream& decry_stream )
{
ifstream ifile( filepath.c_str(), ios_base::binary );
while ( !ifile.eof() )
{
ifile.read((char *)(&aucCipherText), LEN);
read_count = ifile.gcount();

//do some work of decryption...
Decrypt(aucCipherText, aucDecryText, LEN);

for (i = 0; i < read_count; i++)
decry_stream << aucDecryText;
}

return;
}

The input file is about 1M bytes. The time cosumed when i used a
ofstream, strstream and stringstream as the parameter respectively is:
---------------------
ofstream ofile;
foo(ofile);
---------------------
ostrstream osst;
foo(osst);

 

[forrest]

oh, the size of input file for testing is about 270K bytes, not 1M
bytes. osrry.

 

[Mike Wahler]

I wrote a function like this:

#define LEN 8
void foo(ostream& decry_stream )
{
ifstream ifile( filepath.c_str(), ios_base::binary );
while ( !ifile.eof() )
{
ifile.read((char *)(&aucCipherText), LEN);
read_count = ifile.gcount();

//do some work of decryption...
Decrypt(aucCipherText, aucDecryText, LEN);

for (i = 0; i < read_count; i++)
decry_stream << aucDecryText;
}

return;
}

The input file is about 1M bytes. The time cosumed when i used a
ofstream, strstream and stringstream as the parameter respectively is:
---------------------
ofstream ofile;
foo(ofile);
---------------------
ostrstream osst;
foo(osst);

It's impossible for us to tell, since you don't tell us the
states of the output stream objects that get passed to your
function. But I suspect the difference you're seeing is due
to differences in allocation strategies between your implementation's
'ostringstream' and 'ostrstream' types. E.g. I ran a test using
(VC++) default behavior will results similar to yours, but then
preallocated the storage (a static array) for 'ostringstream', and
used the default behavior of 'ostrstream'. The 'ostringstream' ran
approximately 2.5 times as fast as 'ostrstream'. I.e.:

char data[1024 * 1024];

int main()
{
std:strstream ost;
std:stringstream oss(std::string(data, data + sizeof data));
foo(ost);
foo(oss);
return 0;
}


ostrstream: 11329 ms
ostringstream: 4375 ms


-Mike

 

[Mike Wahler]

oh, the size of input file for testing is about 270K bytes, not 1M
bytes. osrry.

The test file I used was approx 1.3 MB
(Pentium 2.8 Ghz, 512 MB RAM)

-Mike

 

[Stephen Howe]

..., but then

preallocated the storage (a static array) for 'ostringstream'

That does no such thing. There is no "pre-allocation" in ostringstream.
It just sets the internal buffer to what is passed as std::string. That is
all.
And it is not clear from the standard as to where the put seek position is
after such an operation (I cannot remember whether a DR resolved this or
not).
Will any << overwrite the internal buffer?
Will and << append to the internal buffer?

Stephen Howe

 

[Stephen Howe]

You can get better performance for the fstreams if read/writing sequentially
by using a bigger buffer:

ifile.rdbuf()->pubsetbuf(NULL, 16384);

after a successful open and before first read/write.

Stephen Howe

 

[Mike Wahler]

That does no such thing. There is no "pre-allocation" in ostringstream.
It just sets the internal buffer to what is passed as std::string. That is
all.

Perhaps it's not technically called 'pre-allocation', but
it does serve the purpose, in that there's ready-to-use
storage when the stringstream is created.

And it is not clear from the standard as to where the put seek position is
after such an operation (I cannot remember whether a DR resolved this or
not).
Will any << overwrite the internal buffer?
Will and << append to the internal buffer?

It's certainly not definitive, but the result I got
indicates that my implementation overwrites (I presume
only until the buffer is full, whereupon I'd expect
allocation to occur).

-Mike

 

[Stephen Howe]

Perhaps it's not technically called 'pre-allocation', but

it does serve the purpose, in that there's ready-to-use
storage when the stringstream is created.

No it is not the same. It is entirely analogous to

std::vector <int> v;

v.reserve(4000); // 1
v.resize(4000); // 2

// 1 reserves memory but v.size() is still 0. There are no elements in the
vector unlike //2. In contrast

char data[1024 * 1024];
std:stringstream oss(std::string(data, data + sizeof data));
std::string s = oss.str();

The fact that s.size() is not 0, means that there _IS_ something occupying
the ostringstream (every character all 0). It is not a reserve of memory.

I wish stringstream's interface was a little better. If I get a chance of
raising it here in Oxford, UK at ACCU with some of the ISO C++ committee
members here, I will do so.

Stephen Howe