Abubakar said:recently some C programmer told me that using fwrite/fopen functions
are not efficient because the output that they do to the file is
actually buffered and gets late in writing.
Is that true?
Abubakar said:Hi,
recently some C programmer told me that using fwrite/fopen functions
are not efficient because the output that they do to the file is
actually buffered and gets late in writing. Is that true?
Hi,
recently some C programmer told me that using fwrite/fopen functions
are not efficient because the output that they do to the file is
actually buffered and gets late in writing. Is that true?
Abubakar said:Hi,
recently some C programmer told me that using fwrite/fopen functions
are not efficient because the output that they do to the file is
actually buffered and gets late in writing. Is that true?
Abubakar said:well he is saying that using the *native* read and write to do the
same task using file descriptors is much faster than the fwrite etc.
He says using the FILE * that is used in case of the fopen/fwrite
buffers data and is not as fast and predictable as read write are.
James Kuyper said:fwrite() buffers data only if you don't tell it not to, by calling
setvbuf(). When talking about buffered I/O, he's right about the
predictability, if he's talking about predicting when data gets written
to the file.
Abubakar said:well he is saying that using the *native* read and write to do the
same task using file descriptors is much faster than the fwrite etc.
He says using the FILE * that is used in case of the fopen/fwrite
buffers data and is not as fast and predictable as read write are.
I have given him the link to this discussion and he is going to be
posting his replies soon hopefully.
Nowadays it is very seldom worth the extra effort and loss of portability
in not calling fwrite(). However that wasn't always true. General-purpose
PCs used to be slower by a factor of 1000 than modern machines, and then
you had to squeeze every last drop of performance out of the machine to
make your games run fast enough.
But disk IO is less likely to be the bottleneck.
My programs take up to
two weeks to run on as many processors as I can lay my hands on, however
they only read and write a few kilobytes of data.
Malcolm McLean said:But disk IO is less likely to be the bottleneck. My programs take up
to two weeks to run on as many processors as I can lay my hands on,
however they only read and write a few kilobytes of data.
However data has got to mean something. 1000 times more processing power
doesn't necessarily mean 1000 times more data. For instance a database
with the address of every taxpayer in the country would comfortably sit
on my PC hard drive. However the cost of collecting and checking that
data would be several milion pounds. The limit is the data itself, not
the machine power needed to process it.
Malcolm McLean said:However data has got to mean something. 1000 times more processing
power doesn't necessarily mean 1000 times more data. For instance a
database with the address of every taxpayer in the country would
comfortably sit on my PC hard drive. However the cost of collecting
and checking that data would be several milion pounds. The limit is
the data itself, not the machine power needed to process it.
He's right.
No, he's not. *Sometimes* it is faster to use write() instead
of fwrite(). Other times it's not. Here's the result of a
quick test:
$ time ./a.out 500000 > /dev/null # Use putc
real 0m0.015s
user 0m0.012s
sys 0m0.002s
$ time ./a.out 500000 USE_WRITE > /dev/null
real 0m0.891s
user 0m0.329s
sys 0m0.558s
(The code used to generate this is below.)
The interpretation of this is simple: using putc
improves performance precisely because of the buffering.
There is no doubt that using write() will often improve
the performance, but NOT if you are making lots of
small writes. When in doubt (and if there is an
actual, observed performance problem) you must
profile the process to determine the bottleneck.
If the problem is IO performance due to fwrite(),
it might be worthwhile to use write() instead.
Might be. Repeated again for emphasis. *Might* be.
If you invest the time re-writing the code to
use write(), you must verify that the performance
gain (or loss) is what you want. You may find
that you have improved throughput by .00001%. Or
maybe you have reduced it by 20%. There are many
managers who will take the 20% performance hit and
call it an improvement and give you a bonus. Take
the money, but find a more competent manager.
Here's the code used to generate the above timings:
/* Unix specific code */
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
int
use_write( int c )
{
int status = c;
char C = c;
if( write( STDOUT_FILENO, &C, 1 ) != 1 )
status = EOF;
return status;
}
int
use_putc( int c )
{
return putchar( c );
}
int
main( int argc, char **argv )
{
unsigned count;
int (*f)(int);
f = ( argc > 2 ) ? use_write : use_putc;
count = ( argc > 1 ) ? strtoul( argv[ 1 ], NULL, 10 ) : BUFSIZ;
for( ; count; count-- )
if( f( 'y' ) != 'y' )
break;;
return count ? EXIT_FAILURE : EXIT_SUCCESS;
}
Want to reply to this thread or ask your own question?
You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.