why is my iostream program so much slower than equivalent cstdio program?

Discussion in 'C++' started by Mark, Dec 23, 2004.

  1. Mark

    Mark Guest

    I am using gnu g++ version 3.3.2,
    trying a simple test to read in and then
    write out a large (100,000 line) text file

    ##########################################
    CSTDIO VERSION TO READ/WRITE TEXT FILE:

    #include <cstdlib>
    #include <cstdio>
    using namespace std;

    main(int argc, void** argv) {

    FILE* in;
    if ((in = fopen("zinput","r")) == NULL) { exit(1); }

    FILE* out;
    if ((out = fopen("zoutput","w")) == NULL) { exit(1); }

    char string[200];

    while (fgets(string,199,in) != NULL) {
    fprintf(out, "%s", string);
    }

    }

    ##########################################
    EQUIVALENT IOSTREAM VERSION TO READ/WRITE TEXT FILE:

    #include <cstdlib>
    #include <iostream>
    #include <fstream>
    using namespace std;

    main(int argc, void** argv) {

    ifstream input("zinput");;
    if ( !input.good() ) { exit(-1); }

    ofstream output("zoutput");
    if ( !output.good() ) { exit(-1); }

    char string[200];

    while ( !input.getline(string, 199).eof() ) {
    output << string << endl;
    }

    }

    ##########################################

    The problem is, the CSTDIO version runs
    10 TIMES FASTER thant the IOSTREAM version!
    Here are results from running the program with
    the time command:

    CSTDIO VERSION:
    real 0.2
    user 0.1
    sys 0.0

    IOSTREAM VERSION:
    real 2.6
    user 1.0
    sys 1.4

    Does anyone know why the IOSTREAM version of this
    simple program is so much slower than the CSTDIO version?
     
    Mark, Dec 23, 2004
    #1
    1. Advertising

  2. Mark

    David White Guest

    "Mark" <> wrote in message
    news:...
    > I am using gnu g++ version 3.3.2,
    > trying a simple test to read in and then
    > write out a large (100,000 line) text file
    >
    > ##########################################
    > CSTDIO VERSION TO READ/WRITE TEXT FILE:
    >
    > #include <cstdlib>
    > #include <cstdio>
    > using namespace std;
    >
    > main(int argc, void** argv) {
    >
    > FILE* in;
    > if ((in = fopen("zinput","r")) == NULL) { exit(1); }
    >
    > FILE* out;
    > if ((out = fopen("zoutput","w")) == NULL) { exit(1); }
    >
    > char string[200];
    >
    > while (fgets(string,199,in) != NULL) {
    > fprintf(out, "%s", string);
    > }
    >
    > }
    >
    > ##########################################
    > EQUIVALENT IOSTREAM VERSION TO READ/WRITE TEXT FILE:
    >
    > #include <cstdlib>
    > #include <iostream>
    > #include <fstream>
    > using namespace std;
    >
    > main(int argc, void** argv) {
    >
    > ifstream input("zinput");;
    > if ( !input.good() ) { exit(-1); }
    >
    > ofstream output("zoutput");
    > if ( !output.good() ) { exit(-1); }
    >
    > char string[200];
    >
    > while ( !input.getline(string, 199).eof() ) {
    > output << string << endl;
    > }
    >
    > }
    >
    > ##########################################
    >
    > The problem is, the CSTDIO version runs
    > 10 TIMES FASTER thant the IOSTREAM version!
    > Here are results from running the program with
    > the time command:
    >
    > CSTDIO VERSION:
    > real 0.2
    > user 0.1
    > sys 0.0
    >
    > IOSTREAM VERSION:
    > real 2.6
    > user 1.0
    > sys 1.4
    >
    > Does anyone know why the IOSTREAM version of this
    > simple program is so much slower than the CSTDIO version?


    These are the results for a test of my own. This was years ago and was for
    VC++ 5 or 6.
    // Time to read a file with 26000 points total:
    // Using ifstream: 11 sec.
    // Using FILE: 0.9 sec.

    I think I posted this once before and PJ Plauger, who wrote at least that
    part of the MS standard library (maybe all of it), said the cause was some
    bug or problem in the library. Whether GNU has a similar problem I don't
    know. I'd be interested to know if there is a library out there whose
    streams are not so inefficient that you are forced to do your file I/O in C.

    DW
     
    David White, Dec 23, 2004
    #2
    1. Advertising

  3. Mark

    Dave O'Hearn Guest

    Mark wrote:
    > output << string << endl;


    std::endl does an implicit flush. I would do this instead,

    output << string << '\n';

    As long as the file is openned in text mode, the '\n' will convert to
    the proper line terminator for the platform.

    On my platform, changing the std::endl to '\n' changed 225% slowdown to
    61% slowdown. If you wanted to do better, you could try dealing with
    the streambufs directly instead of iostreams. It might help.
    --
    Dave O'Hearn
     
    Dave O'Hearn, Dec 23, 2004
    #3
  4. Mark

    Mark Guest

    Thanks for the advice. Now the IOSTREAM version is only twice as slow.
    Mark
     
    Mark, Dec 23, 2004
    #4
  5. Re: why is my iostream program so much slower than equivalent cstdioprogram?

    Mark wrote:

    > I am using gnu g++ version 3.3.2,
    > trying a simple test to read in and then
    > write out a large (100,000 line) text file


    > cat main.C

    #include <cstdlib>
    #include <cstdio>

    int main(int argc, char* argv[]) {
    int value = EXIT_SUCCESS;
    if (1 < argc) {
    FILE* in = fopen(argv[1], "r");
    if (NULL != in) {
    FILE* out = fopen("zoutput", "w");
    if (NULL != out) {
    char string[200];
    while (NULL != fgets(string, 199, in)) {
    fprintf(out, "%s", string);
    }
    }
    else {
    fprintf(stderr, "Failed to open output file: "
    "zoutput\n");
    value = EXIT_FAILURE;
    }
    }
    else {
    fprintf(stderr, "Failed to open input file: %s\n",
    argv[1]);
    value = EXIT_FAILURE;
    }
    }
    else {
    fprintf(stderr, "usage: %s <input file name>\n", argv[0]);
    }
    return value;
    }

    > g++ -Wall -ansi -pedantic -O3 -o main main.C
    > time ./main zinput

    0.070u 0.050s 0:00.11 109.0% 0+0k 0+0io 183pf+0w
    > rm zoutput
    > time ./main zinput

    0.080u 0.040s 0:00.11 109.0% 0+0k 0+0io 183pf+0w
    > rm zoutput
    > time ./main zinput

    0.070u 0.050s 0:00.11 109.0% 0+0k 0+0io 183pf+0w
    > rm zoutput
    > cat main.cc

    #include <cstdlib>
    #include <iostream>
    #include <fstream>

    int main(int argc, char* argv[]) {
    int value = EXIT_SUCCESS;
    if (1 < argc) {
    std::ifstream input(argv[1]);
    if (input.good()) {
    std::eek:fstream output("zoutput");
    if (output.good()) {
    char string[200];
    while (!input.getline(string, 199).eof()) {
    output << string << '\n';
    }
    }
    else {
    std::cerr << "Failed to open output file: zoutput"
    << std::endl;
    fprintf(stderr, "Failed to open output file: "
    "zoutput\n");
    value = EXIT_FAILURE;
    }
    }
    else {
    std::cerr << "Failed to open input file: " << argv[1]
    << std::endl;
    value = EXIT_FAILURE;
    }
    }
    else {
    std::cerr << "usage: " << argv[0] << "<input file name>"
    << std::endl;
    }
    return value;
    }

    > g++ -Wall -ansi -pedantic -O3 -o main main.cc
    > time ./main zinput

    0.100u 0.060s 0:00.16 100.0% 0+0k 0+0io 217pf+0w
    > rm zoutput
    > time ./main zinput

    0.120u 0.040s 0:00.16 100.0% 0+0k 0+0io 217pf+0w
    > rm zoutput
    > time ./main zinput

    0.120u 0.050s 0:00.16 106.2% 0+0k 0+0io 217pf+0w
    > ls -s zoutput

    6416 zoutput
    > g++ --version

    g++ (GCC) 3.4.0
    > cat /etc/redhat-release

    Red Hat Linux release 8.0 (Psyche)
     
    E. Robert Tisdale, Dec 23, 2004
    #5
  6. Re: why is my iostream program so much slower than equivalent cstdioprogram?

    Mark wrote:

    > The problem is, the CSTDIO version runs
    > 10 TIMES FASTER thant the IOSTREAM version!
    > Here are results from running the program with
    > the time command:
    >
    > CSTDIO VERSION:
    > real 0.2
    > user 0.1
    > sys 0.0
    >
    > IOSTREAM VERSION:
    > real 2.6
    > user 1.0
    > sys 1.4
    >
    > Does anyone know why the IOSTREAM version of this
    > simple program is so much slower than the CSTDIO version?
    >


    Did you turn on optimizations?

    -dr
     
    Dave Rahardja, Dec 23, 2004
    #6
  7. Mark

    Alex Vinokur Guest

    "Mark" <> wrote in message news:...
    [snip]
    > Does anyone know why the IOSTREAM version of this
    > simple program is so much slower than the CSTDIO version?
    >


    Results of comparative performance measurement "cstdio vs. iostream" can be seen at
    http://article.gmane.org/gmane.comp.lang.c .perfometer/45
    http://article.gmane.org/gmane.comp.lang.c .perfometer/38
    http://groups-beta.google.com/group/log-files/msg/c352787bac28942e

    --
    Alex Vinokur
    email: alex DOT vinokur AT gmail DOT com
    http://mathforum.org/library/view/10978.html
    http://sourceforge.net/users/alexvn
     
    Alex Vinokur, Dec 23, 2004
    #7
  8. Mark

    Alex Vinokur Guest

    "Alex Vinokur" <> wrote in message news:...
    >
    > "Mark" <> wrote in message news:...
    > [snip]
    > > Does anyone know why the IOSTREAM version of this
    > > simple program is so much slower than the CSTDIO version?
    > >

    >
    > Results of comparative performance measurement "cstdio vs. iostream" can be seen at
    > http://article.gmane.org/gmane.comp.lang.c .perfometer/45
    > http://article.gmane.org/gmane.comp.lang.c .perfometer/38
    > http://groups-beta.google.com/group/log-files/msg/c352787bac28942e
    >


    Also
    http://article.gmane.org/gmane.comp.lang.c .perfometer/71
    http://groups-beta.google.com/group/log-files/msg/e4a0e2e589527647


    --
    Alex Vinokur
    email: alex DOT vinokur AT gmail DOT com
    http://mathforum.org/library/view/10978.html
    http://sourceforge.net/users/alexvn
     
    Alex Vinokur, Dec 23, 2004
    #8
  9. Mark

    GianGuz Guest

    Mark wrote:
    > I am using gnu g++ version 3.3.2,
    > trying a simple test to read in and then
    > write out a large (100,000 line) text file
    >
    > ##########################################
    > CSTDIO VERSION TO READ/WRITE TEXT FILE:
    >
    > #include <cstdlib>
    > #include <cstdio>
    > using namespace std;
    >
    > main(int argc, void** argv) {
    >
    > FILE* in;
    > if ((in = fopen("zinput","r")) == NULL) { exit(1); }
    >
    > FILE* out;
    > if ((out = fopen("zoutput","w")) == NULL) { exit(1); }
    >
    > char string[200];
    >
    > while (fgets(string,199,in) != NULL) {
    > fprintf(out, "%s", string);
    > }
    >
    > }
    >
    > ##########################################
    > EQUIVALENT IOSTREAM VERSION TO READ/WRITE TEXT FILE:
    >
    > #include <cstdlib>
    > #include <iostream>
    > #include <fstream>
    > using namespace std;
    >
    > main(int argc, void** argv) {
    >
    > ifstream input("zinput");;
    > if ( !input.good() ) { exit(-1); }
    >
    > ofstream output("zoutput");
    > if ( !output.good() ) { exit(-1); }
    >
    > char string[200];
    >
    > while ( !input.getline(string, 199).eof() ) {
    > output << string << endl;
    > }
    >
    > }
    >
    > ##########################################
    >
    > The problem is, the CSTDIO version runs
    > 10 TIMES FASTER thant the IOSTREAM version!
    > Here are results from running the program with
    > the time command:
    >
    > CSTDIO VERSION:
    > real 0.2
    > user 0.1
    > sys 0.0
    >
    > IOSTREAM VERSION:
    > real 2.6
    > user 1.0
    > sys 1.4
    >
    > Does anyone know why the IOSTREAM version of this
    > simple program is so much slower than the CSTDIO version?



    Try also this funny option in your code:

    output.sync_with_stdio(true);

    while ( !input.getline(string, 199).eof() ) {
    output << string << endl;

    }

    And flush() everything!;)
     
    GianGuz, Dec 23, 2004
    #9
  10. Mark

    Siemel Naran Guest

    "GianGuz" <> wrote in message

    > Try also this funny option in your code:
    >
    > output.sync_with_stdio(true);


    Won't this make it slower?
     
    Siemel Naran, Dec 23, 2004
    #10
  11. Mark

    void Guest

    Re: why is my iostream program so much slower than equivalent cstdioprogram?

    GianGuz wrote:

    > Try also this funny option in your code:
    >
    > output.sync_with_stdio(true);
    >


    Rather:
    output.sync_with_stdio(false);

    Best
    Darek
     
    void, Dec 23, 2004
    #11
  12. Mark

    GianGuz Guest

    Right! Sorry for the misleading...I would have write that, void!
     
    GianGuz, Dec 23, 2004
    #12
  13. Mark

    Dave O'Hearn Guest

    Mark wrote:
    > Thanks for the advice. Now the IOSTREAM version is only twice
    > as slow.


    At that point, it's probably a quality of implementation issue. E.
    Robert Tisdale posted results where the iostreams code was faster. I
    used the same compiler he did, but on a much older Red Hat, and mine
    was 60% slower.

    These benchmarks also may not be relevant to real performance, as the
    file ends up paged in, and there are no page faults while reading it.
    In most real situations, the page faults would dwarf any 60% or even
    100% overhead.

    --
    Dave O'Hearn
     
    Dave O'Hearn, Dec 23, 2004
    #13
  14. Mark

    Mark Guest

    output.sync_with_stdio(false) has no effect on runtime

    However, using
    << '\n'
    instead of
    << endl
    did dramtically improve performance, from IOSTREAM
    being 10 times slower to being only twice as slow

    Mark
     
    Mark, Dec 23, 2004
    #14
  15. Mark

    GianGuz Guest

    This is an interesting evaluation! ;O
    I also have a program in which a lot of debugging informations were
    flushed into a
    synchronized ostream while the program, with its different threads,
    runs.
    I will try to substitute the endl with '\n' everywhere. I'm curious to
    see if also my performance
    will be improved!
     
    GianGuz, Dec 24, 2004
    #15
  16. Mark

    Alex Vinokur Guest

    "Mark" <> wrote in message news:...
    > output.sync_with_stdio(false) has no effect on runtime
    >
    > However, using
    > << '\n'
    > instead of
    > << endl
    > did dramtically improve performance, from IOSTREAM
    > being 10 times slower to being only twice as slow

    [snip]

    Yes, look at C/C++ Performance Tests:
    endl vs. "\n/" and '\n'
    for
    stdout, stderr, cout, cerr, clog, ostringstream, cout-to-file, cerr-to-file, clog-to-file
    http://groups-beta.google.com/group/comp.lang.c /msg/f0a3f2c6e789dde7
    http://groups-beta.google.com/group/misc.test/msg/ebdc73df3fd7df08


    --
    Alex Vinokur
    email: alex DOT vinokur AT gmail DOT com
    http://mathforum.org/library/view/10978.html
    http://sourceforge.net/users/alexvn
     
    Alex Vinokur, Dec 24, 2004
    #16
  17. Mark

    Ron Natalie Guest

    Re: why is my iostream program so much slower than equivalent cstdioprogram?

    Dave Rahardja wrote:

    >
    > Did you turn on optimizations?
    >

    This is very important. The C++ library files tend to make lots
    of small inline functions (where as the STDIO uses alot of macros
    and large monolithic functions). If you've got inlining disabled
    it will be a LOT slower.
     
    Ron Natalie, Dec 24, 2004
    #17
  18. Re: why is my iostream program so much slower than equivalent cstdioprogram?

    GianGuz wrote:
    > This is an interesting evaluation! ;O
    > I also have a program in which a lot of debugging informations were
    > flushed into a
    > synchronized ostream while the program, with its different threads,
    > runs.
    > I will try to substitute the endl with '\n' everywhere. I'm curious to
    > see if also my performance
    > will be improved!
    >

    Hi,

    be careful with replacing 'endl' by '\n' when you write debugging
    information. If your program crashes, there may be unflushed data in the
    stream which won't be written to the hard disk, so valuable debugging
    information (which you may need to find the reason for the crash) gets lost!

    greetings
    Martin
     
    Martin Stettner, Dec 26, 2004
    #18
  19. Mark

    GianGuz Guest

    Ok i'll follow your suggestions. By the way a macro that substitutes
    endl with '\n' when the
    program isn't in debugging mode seems to me a good compromise!
    Gianguglielmo
     
    GianGuz, Dec 27, 2004
    #19
  20. Re: why is my iostream program so much slower than equivalent cstdioprogram?

    GianGuz wrote:
    > By the way a macro that substitutes
    > endl with '\n' when the
    > program isn't in debugging mode seems to me a good compromise!


    I think this is a bad idea. There are other times when buffering and
    flushing can cause behavior differences (esp. as regards deadlock in
    some programming types) and it would be bad if the problem disappears
    when you compile with debugging. I think it's better to just use what
    you mean while you program.

    I also tend to send debugging information to std::cerr instead of
    std::cout. It makes it easier to debug filters, and (correct me if I'm
    wrong) std::cerr isn't buffered. (OTOH, I generally use std::endl with
    std::cerr anyway, so it wouldn't bite me if it were.)
     
    Adam Peterson, Dec 27, 2004
    #20
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. ai@work
    Replies:
    9
    Views:
    549
    Ron Natalie
    Dec 16, 2004
  2. Mr. SweatyFinger
    Replies:
    2
    Views:
    2,033
    Smokey Grindel
    Dec 2, 2006
  3. Replies:
    27
    Views:
    540
    Gabriel Genellina
    Jun 14, 2007
  4. dustbort
    Replies:
    3
    Views:
    3,946
    JosipJaic
    Jun 18, 2010
  5. Tim Meagher
    Replies:
    3
    Views:
    106
    Ray Costanzo [MVP]
    Oct 4, 2005
Loading...

Share This Page