why is my iostream program so much slower than equivalent cstdio program?

M

Mark

I am using gnu g++ version 3.3.2,
trying a simple test to read in and then
write out a large (100,000 line) text file

##########################################
CSTDIO VERSION TO READ/WRITE TEXT FILE:

#include <cstdlib>
#include <cstdio>
using namespace std;

main(int argc, void** argv) {

FILE* in;
if ((in = fopen("zinput","r")) == NULL) { exit(1); }

FILE* out;
if ((out = fopen("zoutput","w")) == NULL) { exit(1); }

char string[200];

while (fgets(string,199,in) != NULL) {
fprintf(out, "%s", string);
}

}

##########################################
EQUIVALENT IOSTREAM VERSION TO READ/WRITE TEXT FILE:

#include <cstdlib>
#include <iostream>
#include <fstream>
using namespace std;

main(int argc, void** argv) {

ifstream input("zinput");;
if ( !input.good() ) { exit(-1); }

ofstream output("zoutput");
if ( !output.good() ) { exit(-1); }

char string[200];

while ( !input.getline(string, 199).eof() ) {
output << string << endl;
}

}

##########################################

The problem is, the CSTDIO version runs
10 TIMES FASTER thant the IOSTREAM version!
Here are results from running the program with
the time command:

CSTDIO VERSION:
real 0.2
user 0.1
sys 0.0

IOSTREAM VERSION:
real 2.6
user 1.0
sys 1.4

Does anyone know why the IOSTREAM version of this
simple program is so much slower than the CSTDIO version?
 
D

David White

Mark said:
I am using gnu g++ version 3.3.2,
trying a simple test to read in and then
write out a large (100,000 line) text file

##########################################
CSTDIO VERSION TO READ/WRITE TEXT FILE:

#include <cstdlib>
#include <cstdio>
using namespace std;

main(int argc, void** argv) {

FILE* in;
if ((in = fopen("zinput","r")) == NULL) { exit(1); }

FILE* out;
if ((out = fopen("zoutput","w")) == NULL) { exit(1); }

char string[200];

while (fgets(string,199,in) != NULL) {
fprintf(out, "%s", string);
}

}

##########################################
EQUIVALENT IOSTREAM VERSION TO READ/WRITE TEXT FILE:

#include <cstdlib>
#include <iostream>
#include <fstream>
using namespace std;

main(int argc, void** argv) {

ifstream input("zinput");;
if ( !input.good() ) { exit(-1); }

ofstream output("zoutput");
if ( !output.good() ) { exit(-1); }

char string[200];

while ( !input.getline(string, 199).eof() ) {
output << string << endl;
}

}

##########################################

The problem is, the CSTDIO version runs
10 TIMES FASTER thant the IOSTREAM version!
Here are results from running the program with
the time command:

CSTDIO VERSION:
real 0.2
user 0.1
sys 0.0

IOSTREAM VERSION:
real 2.6
user 1.0
sys 1.4

Does anyone know why the IOSTREAM version of this
simple program is so much slower than the CSTDIO version?
Click to expand...

These are the results for a test of my own. This was years ago and was for
VC++ 5 or 6.
// Time to read a file with 26000 points total:
// Using ifstream: 11 sec.
// Using FILE: 0.9 sec.

I think I posted this once before and PJ Plauger, who wrote at least that
part of the MS standard library (maybe all of it), said the cause was some
bug or problem in the library. Whether GNU has a similar problem I don't
know. I'd be interested to know if there is a library out there whose
streams are not so inefficient that you are forced to do your file I/O in C.

DW
 
D

Dave O'Hearn

Mark said:
output << string << endl;
Click to expand...

std::endl does an implicit flush. I would do this instead,

output << string << '\n';

As long as the file is openned in text mode, the '\n' will convert to
the proper line terminator for the platform.

On my platform, changing the std::endl to '\n' changed 225% slowdown to
61% slowdown. If you wanted to do better, you could try dealing with
the streambufs directly instead of iostreams. It might help.
 
E

E. Robert Tisdale

Mark said:
I am using gnu g++ version 3.3.2,
trying a simple test to read in and then
write out a large (100,000 line) text file
Click to expand...
cat main.C
Click to expand...
#include <cstdlib>
#include <cstdio>

int main(int argc, char* argv[]) {
int value = EXIT_SUCCESS;
if (1 < argc) {
FILE* in = fopen(argv[1], "r");
if (NULL != in) {
FILE* out = fopen("zoutput", "w");
if (NULL != out) {
char string[200];
while (NULL != fgets(string, 199, in)) {
fprintf(out, "%s", string);
}
}
else {
fprintf(stderr, "Failed to open output file: "
"zoutput\n");
value = EXIT_FAILURE;
}
}
else {
fprintf(stderr, "Failed to open input file: %s\n",
argv[1]);
value = EXIT_FAILURE;
}
}
else {
fprintf(stderr, "usage: %s <input file name>\n", argv[0]);
}
return value;
}
g++ -Wall -ansi -pedantic -O3 -o main main.C
time ./main zinput
Click to expand...
0.070u 0.050s 0:00.11 109.0% 0+0k 0+0io 183pf+0w
rm zoutput
time ./main zinput
Click to expand...
0.080u 0.040s 0:00.11 109.0% 0+0k 0+0io 183pf+0w
rm zoutput
time ./main zinput
Click to expand...
0.070u 0.050s 0:00.11 109.0% 0+0k 0+0io 183pf+0w
rm zoutput
cat main.cc
Click to expand...
#include <cstdlib>
#include <iostream>
#include <fstream>

int main(int argc, char* argv[]) {
int value = EXIT_SUCCESS;
if (1 < argc) {
std::ifstream input(argv[1]);
if (input.good()) {
std::eek:fstream output("zoutput");
if (output.good()) {
char string[200];
while (!input.getline(string, 199).eof()) {
output << string << '\n';
}
}
else {
std::cerr << "Failed to open output file: zoutput"
<< std::endl;
fprintf(stderr, "Failed to open output file: "
"zoutput\n");
value = EXIT_FAILURE;
}
}
else {
std::cerr << "Failed to open input file: " << argv[1]
<< std::endl;
value = EXIT_FAILURE;
}
}
else {
std::cerr << "usage: " << argv[0] << "<input file name>"
<< std::endl;
}
return value;
}
g++ -Wall -ansi -pedantic -O3 -o main main.cc
time ./main zinput
Click to expand...
0.100u 0.060s 0:00.16 100.0% 0+0k 0+0io 217pf+0w
rm zoutput
time ./main zinput
Click to expand...
0.120u 0.040s 0:00.16 100.0% 0+0k 0+0io 217pf+0w
rm zoutput
time ./main zinput
Click to expand...
0.120u 0.050s 0:00.16 106.2% 0+0k 0+0io 217pf+0w
ls -s zoutput 6416 zoutput
g++ --version g++ (GCC) 3.4.0
cat /etc/redhat-release
Click to expand...
Red Hat Linux release 8.0 (Psyche)
 
D

Dave Rahardja

Mark said:
The problem is, the CSTDIO version runs
10 TIMES FASTER thant the IOSTREAM version!
Here are results from running the program with
the time command:

CSTDIO VERSION:
real 0.2
user 0.1
sys 0.0

IOSTREAM VERSION:
real 2.6
user 1.0
sys 1.4

Does anyone know why the IOSTREAM version of this
simple program is so much slower than the CSTDIO version?
Click to expand...

Did you turn on optimizations?

-dr
 
G

GianGuz

Mark said:
I am using gnu g++ version 3.3.2,
trying a simple test to read in and then
write out a large (100,000 line) text file

##########################################
CSTDIO VERSION TO READ/WRITE TEXT FILE:

#include <cstdlib>
#include <cstdio>
using namespace std;

main(int argc, void** argv) {

FILE* in;
if ((in = fopen("zinput","r")) == NULL) { exit(1); }

FILE* out;
if ((out = fopen("zoutput","w")) == NULL) { exit(1); }

char string[200];

while (fgets(string,199,in) != NULL) {
fprintf(out, "%s", string);
}

}

##########################################
EQUIVALENT IOSTREAM VERSION TO READ/WRITE TEXT FILE:

#include <cstdlib>
#include <iostream>
#include <fstream>
using namespace std;

main(int argc, void** argv) {

ifstream input("zinput");;
if ( !input.good() ) { exit(-1); }

ofstream output("zoutput");
if ( !output.good() ) { exit(-1); }

char string[200];

while ( !input.getline(string, 199).eof() ) {
output << string << endl;
}

}

##########################################

The problem is, the CSTDIO version runs
10 TIMES FASTER thant the IOSTREAM version!
Here are results from running the program with
the time command:

CSTDIO VERSION:
real 0.2
user 0.1
sys 0.0

IOSTREAM VERSION:
real 2.6
user 1.0
sys 1.4

Does anyone know why the IOSTREAM version of this
simple program is so much slower than the CSTDIO version?
Click to expand...


Try also this funny option in your code:

output.sync_with_stdio(true);

while ( !input.getline(string, 199).eof() ) {
output << string << endl;

}

And flush() everything!;)
 
D

Dave O'Hearn

Mark said:
Thanks for the advice. Now the IOSTREAM version is only twice
as slow.
Click to expand...

At that point, it's probably a quality of implementation issue. E.
Robert Tisdale posted results where the iostreams code was faster. I
used the same compiler he did, but on a much older Red Hat, and mine
was 60% slower.

These benchmarks also may not be relevant to real performance, as the
file ends up paged in, and there are no page faults while reading it.
In most real situations, the page faults would dwarf any 60% or even
100% overhead.
 
M

Mark

output.sync_with_stdio(false) has no effect on runtime

However, using
<< '\n'
instead of
<< endl
did dramtically improve performance, from IOSTREAM
being 10 times slower to being only twice as slow

Mark
 
G

GianGuz

This is an interesting evaluation! ;O
I also have a program in which a lot of debugging informations were
flushed into a
synchronized ostream while the program, with its different threads,
runs.
I will try to substitute the endl with '\n' everywhere. I'm curious to
see if also my performance
will be improved!
 
A

Alex Vinokur

Mark said:
output.sync_with_stdio(false) has no effect on runtime

However, using
<< '\n'
instead of
<< endl
did dramtically improve performance, from IOSTREAM
being 10 times slower to being only twice as slow
Click to expand...
[snip]

Yes, look at C/C++ Performance Tests:
endl vs. "\n/" and '\n'
for
stdout, stderr, cout, cerr, clog, ostringstream, cout-to-file, cerr-to-file, clog-to-file
http://groups-beta.google.com/group/comp.lang.c++/msg/f0a3f2c6e789dde7
http://groups-beta.google.com/group/misc.test/msg/ebdc73df3fd7df08
 
R

Ron Natalie

Dave said:
Did you turn on optimizations?
Click to expand...
This is very important. The C++ library files tend to make lots
of small inline functions (where as the STDIO uses alot of macros
and large monolithic functions). If you've got inlining disabled
it will be a LOT slower.
 
M

Martin Stettner

GianGuz said:
This is an interesting evaluation! ;O
I also have a program in which a lot of debugging informations were
flushed into a
synchronized ostream while the program, with its different threads,
runs.
I will try to substitute the endl with '\n' everywhere. I'm curious to
see if also my performance
will be improved!
Click to expand...
Hi,

be careful with replacing 'endl' by '\n' when you write debugging
information. If your program crashes, there may be unflushed data in the
stream which won't be written to the hard disk, so valuable debugging
information (which you may need to find the reason for the crash) gets lost!

greetings
Martin
 
G

GianGuz

Ok i'll follow your suggestions. By the way a macro that substitutes
endl with '\n' when the
program isn't in debugging mode seems to me a good compromise!
Gianguglielmo
 
A

Adam Peterson

GianGuz said:
By the way a macro that substitutes
endl with '\n' when the
program isn't in debugging mode seems to me a good compromise!
Click to expand...

I think this is a bad idea. There are other times when buffering and
flushing can cause behavior differences (esp. as regards deadlock in
some programming types) and it would be bad if the problem disappears
when you compile with debugging. I think it's better to just use what
you mean while you program.

I also tend to send debugging information to std::cerr instead of
std::cout. It makes it easier to debug filters, and (correct me if I'm
wrong) std::cerr isn't buffered. (OTOH, I generally use std::endl with
std::cerr anyway, so it wouldn't bite me if it were.)
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Similar Threads

Why is java consumer/producer so much faster than C++ 39
Adding adressing of IPv6 to program 1
Insertion Sort : C++ implementation 100 times slower than C implementation 7
Why the undefined references in this simple program? 5
is MS newer <iostream> is slower than older <iostream.h>? 9
Help optimize nbody bench program (c++ sse2 intrinsics) 3
Linux: using "clone3" and "waitid" 0
compilation error for not including iostream 2

Members online

No members online now.
Total: 25 (members: 0, guests: 25)
Robots: 322

Forum statistics

Threads
473,755
Messages
2,569,536
Members
45,007
Latest member
obedient dusk

Latest Threads

Top