Data redundancy, performance and formatted read/write operations.

[Jason Heyes]

I am starting to worry about the performance of formatted read/write
operations on data-redundant objects in my program.What can I do to improve
this performance should it become an issue?

Thanks.

 

[Alf P. Steinbach]

* Jason Heyes:

I am starting to worry about the performance of formatted read/write
operations on data-redundant objects in my program.What can I do to improve
this performance should it become an issue?

Measure.

 

[Jason Heyes]

* Jason Heyes:

Measure.

Whoops. I didn't expect a reply this short!

 

[Thomas Matthews]

I am starting to worry about the performance of formatted read/write
operations on data-redundant objects in my program.What can I do to improve
this performance should it become an issue?

Thanks.

Here are some time-proven methods for speeding up I/O:
1. Don't perform I/O.

2. Reduce to the absolute minimum necessary.
Eliminate debug writes. Don't output the same data
twice or more. Etc.

3. Format all data then output it.
Format all of your data into a buffer, then output
that buffer. This eliminates the need to switch
back and forth between formatting and actual output.

4. Format only data that is necessary.
Minimize the text in formatted data requests. If
the text doesn't change, consider placing the text
into a buffer, then only format data into specific
locations. Then use block output on the whole buffer.

5. Use block input and output.
Generally, one call to output a lot of data is faster
than many calls to output small chunks of data.

6. Keep the stream continuous.
Some devices on the other end of the stream have
start-up or slow down overhead, but incur very
few penalties for processing data.


--
Thomas Matthews

C++ newsgroup welcome message:
http://www.slack.net/~shiva/welcome.txt
C++ Faq: http://www.parashift.com/c++-faq-lite
C Faq: http://www.eskimo.com/~scs/c-faq/top.html
alt.comp.lang.learn.c-c++ faq:
http://www.comeaucomputing.com/learn/faq/
Other sites:
http://www.josuttis.com -- C++ STL Library book
http://www.sgi.com/tech/stl -- Standard Template Library

 

[Jason Heyes]

Here are some time-proven methods for speeding up I/O:
1. Don't perform I/O.

2. Reduce to the absolute minimum necessary.
Eliminate debug writes. Don't output the same data
twice or more. Etc.

3. Format all data then output it.
Format all of your data into a buffer, then output
that buffer. This eliminates the need to switch
back and forth between formatting and actual output.

4. Format only data that is necessary.
Minimize the text in formatted data requests. If
the text doesn't change, consider placing the text
into a buffer, then only format data into specific
locations. Then use block output on the whole buffer.

5. Use block input and output.
Generally, one call to output a lot of data is faster
than many calls to output small chunks of data.

6. Keep the stream continuous.
Some devices on the other end of the stream have
start-up or slow down overhead, but incur very
few penalties for processing data.

As a start, I am going to focus on (2). My task is to eliminate redundant
reads and writes in my program. It just so happens that all these operations
occur during the read or write operation of a single object. So I might use
the following functions to optimise my program:

std::istream &read_redundant(std::istream &is, Foo &foo)
{
if (!(is >> foo))
return is;
foo.add_duplicates();
return is;
}

std:stream &write_redundant(std:stream &os, Foo foo)
{
foo.remove_duplicates();
return os << foo;
}

The member functions Foo::add_duplicates and Foo::remove_duplicates must be
"inverses" of each other to make the optimisation process invisible to the
user.

Anyway. This is all up in the air. Thank you very much for your reply.

 

[Stephen Howe]

I am starting to worry about the performance of formatted read/write

operations on data-redundant objects in my program.What can I do to improve
this performance should it become an issue?

Buffering
Binary Mode

SH

 

[Mark Stijnman]

Here are some time-proven methods for speeding up I/O:
1. Don't perform I/O.

2. Reduce to the absolute minimum necessary.
Eliminate debug writes. Don't output the same data
twice or more. Etc.

3. Format all data then output it.
Format all of your data into a buffer, then output
that buffer. This eliminates the need to switch
back and forth between formatting and actual output.

4. Format only data that is necessary.
Minimize the text in formatted data requests. If
the text doesn't change, consider placing the text
into a buffer, then only format data into specific
locations. Then use block output on the whole buffer.

5. Use block input and output.
Generally, one call to output a lot of data is faster
than many calls to output small chunks of data.

6. Keep the stream continuous.
Some devices on the other end of the stream have
start-up or slow down overhead, but incur very
few penalties for processing data.


--
Thomas Matthews

C++ newsgroup welcome message:
http://www.slack.net/~shiva/welcome.txt
C++ Faq: http://www.parashift.com/c++-faq-lite
C Faq: http://www.eskimo.com/~scs/c-faq/top.html
alt.comp.lang.learn.c-c++ faq:
http://www.comeaucomputing.com/learn/faq/
Other sites:
http://www.josuttis.com -- C++ STL Library book
http://www.sgi.com/tech/stl -- Standard Template Library

And don't use endl at the end of lines, use "\n" instead. Most ostream
objects support buffered output, but endl will force the buffer to be
flushed. Using endl on every line can completely destroy all the
benefits of buffered output.