"endl and "\n"

[dover]

For the code, outputfile << *p << endl;

Someone suggests: Don't use endl here; it is flushing output stream every
time. Use plain '\n'.

What's the difference of using "endl" and "\n" above? What's the meaning of
flushing output stream? Isn't that what we want?

Thanks!

 

[Sharad Kala]

For the code, outputfile << *p << endl;

Someone suggests: Don't use endl here; it is flushing output stream every
time. Use plain '\n'.

What's the difference of using "endl" and "\n" above? What's the meaning of
flushing output stream? Isn't that what we want?

Somebody had already given you the answer ;-), it is flushing output stream
every time. Buffering here means that not everything that you write into the
stream goes to the external device, instead IOStreams is buffering (storing)
it and at some point they flush it to the external device. This boosts the
efficiency. With every endl you flush the stream which may be an overkill.

-Sharad

 

[Karl Heinz Buchegger]

For the code, outputfile << *p << endl;

Someone suggests: Don't use endl here; it is flushing output stream every
time. Use plain '\n'.

What's the difference of using "endl" and "\n" above? What's the meaning of
flushing output stream? Isn't that what we want?

The point is:
Most operating systems or most C++ libraries do a sort of buffering.
If you do an output, the output is not immediatly sent to the
output device, but instead is buffered until a specific amount of
buffered output is collected, which is then sent in one chunk to the
device. The reason why this is done is simply efficiency. To hand some
data through to the receiving device, the data has to pass various stages
in either the C++ library and/or the operating system. All of them take
time and it would be a waste of time to do that for eg. each single
character.

But sometimes you want exactly that: no buffering, each character is sent
immediatly to the device as soon as it is available. This is the situation
where flushing comes into play. It is the request to empty all those buffers
and sent the data to the device, regardless of buffering strategy.
Example: If you do debugging the old way by inserting output statements into
your source code and watching which one you saw last before the crash. Here
you don't want some buffering to occour and thus you flush the output stream.

 

[Minti]

The point is:
Most operating systems or most C++ libraries do a sort of buffering.
If you do an output, the output is not immediatly sent to the
output device, but instead is buffered until a specific amount of
buffered output is collected, which is then sent in one chunk to the
device. The reason why this is done is simply efficiency. To hand some
data through to the receiving device, the data has to pass various stages
in either the C++ library and/or the operating system. All of them take
time and it would be a waste of time to do that for eg. each single
character.

But sometimes you want exactly that: no buffering, each character is sent
immediatly to the device as soon as it is available. This is the situation
where flushing comes into play. It is the request to empty all those buffers
and sent the data to the device, regardless of buffering strategy.
Example: If you do debugging the old way by inserting output statements into
your source code and watching which one you saw last before the crash. Here
you don't want some buffering to occour and thus you flush the output stream.

Yes, we all know this but what surprises me is that most of textbook
authors pretty much NEVER use "\n", they just keep on using std::endl,
without highlighting the potential loss in efficiency.

 

[Howard]

Karl Heinz Buchegger <[email protected]> wrote in message stream.

Yes, we all know this but what surprises me is that most of textbook
authors pretty much NEVER use "\n", they just keep on using std::endl,
without highlighting the potential loss in efficiency.

Well, I'm assuming, since it was asked in the first place, that we *don't*
all know this! (Also, I saw a post not too long ago that made the
blanket statement that "std::endl is evil". No follow-ups explaining *why*
ever emerged, though.)

-Howard

 

[Alf P. Steinbach]

* Minti:

* Karl Heinz Buchegger:


Yes, we all know this but what surprises me is that most of textbook
authors pretty much NEVER use "\n", they just keep on using std::endl,
without highlighting the potential loss in efficiency.

In situations where you want performance you'd probably not use the
standard streams anyway...

But I agree that it would be nice if textbooks made an attempt at
clarifying this issue.

Doing what would be "right" in another context, just to sort of hammer
the message home, could however be a disservice to the reader. Instead
the authors should explain that since they're using standard iostreams
efficiency is not a concern, but readability is. And then use std::endl.

 

[Dietmar Kuehl]

In situations where you want performance you'd probably not use the
standard streams anyway...

What alternatives to I/O you are using? ... and where do you suspect
inherent performance limitations for the standard streams?

 

[Alf P. Steinbach]

* Dietmar Kuehl:

What alternatives to I/O you are using? ... and where do you suspect
inherent performance limitations for the standard streams?

Regarding the first question, there are no alternatives to I/O when I/O
is what one needs; however, the standard streams cover more than I/O.

Regarding the second question, the word "suspect" is a bit personal, and
the word "inherent" makes it a pure rhetorical question for which no
non-rhetorical answer is possible.

In other words, I'm not sure these are honest technical questions.

But that perception may be due to a recent unfortunate experience with
another poster here, so, responding to the drift of the questions, if not
their literal meanings: although there's probably no inherent limitation
in the standard stream's non-design, in practice it turns out these
streams are extremely inefficient, extremely complicated (to wit, whole
volumes have been written about how to use them for very simple tasks),
and, not the least, they are extremely unsafe, like novice code.

In one discussion about this topic in [no.it.programmering.c++] we
measured performance that was, as I recall, 20 (yes, 20!) or more times
worse than alternatives (for std::stringstream conversion to text).

 

[Dietmar Kuehl]

* Dietmar Kuehl:

Regarding the first question, there are no alternatives to I/O when I/O
is what one needs; however, the standard streams cover more than I/O.

Sorry if my question was ambiguous. What I meant to ask was what you are
using instead of using IOStreams. ... and even I/O was not precise as I
intended to include formatting and the corresponding parsing.

Regarding the second question, the word "suspect" is a bit personal, and
the word "inherent" makes it a pure rhetorical question for which no
non-rhetorical answer is possible.

In other words, I'm not sure these are honest technical questions.

I don't think "inherent" makes it a pure rhetorical question: your
statement is that IOStreams have bad performance. This, at least to me,
implies that there is something inherently wrong with IOStreams which
makes it impossible to implement them such that they are at least as
efficient as an alternative approach. I happen to disagree with such a
statement and since pure disagreement does not provide a good techical
argument, I have actually implemented the IOStreams (and locales)
library of the standard C++ library. Although I have not optimized all
areas of this implementation, I think it for most uses roughly equivalent
to typical stdio implementations.

But that perception may be due to a recent unfortunate experience with
another poster here, so, responding to the drift of the questions, if not
their literal meanings: although there's probably no inherent limitation
in the standard stream's non-design, in practice it turns out these
streams are extremely inefficient, extremely complicated (to wit, whole
volumes have been written about how to use them for very simple tasks),
and, not the least, they are extremely unsafe, like novice code.

Just to address things:
- I disagree about there not being a design, actually I consider it to
be quite good since independent issues are handled by different
entities: i18n (i.e. location dependent variation in formatting) is
handled by locales, access to "external" destinations by stream buffers,
and formatting by the respective classes. What is wrong with this? The
fact that people refuse to at least look at the design and not to try to
derive from ostream is hardly the fault of the design...
A few rarely used things are not optimal (e.g. the callback mechanism
for resource management of associated data; today this would probably
use a shared pointer or something like this). However, as said, they
are rarely used and rarely necessary.
- I don't argue about typical IOStream *implementations* being indeed
slow. Actually, this is something I complain about to the various
library implementors and I try to help them in improving their
implementation. However, I don't think there is any inherent performance
problem. At least, I'm not aware of any.
- Complicated? Well, yes, complicated things are indeed complicated with
IOStreams like seeking within a file requiring a code conversion.
However, this only applies to the implementation of the underlying
mechanism and not its use. I don't see where IOStream use is
complicated. ... and I don't see where extending the system, e.g. by
implementing new stream buffers for typical needs is complicated. Yes,
a stream buffer involving a code conversion is complicated but it is
readily available as file streams. The fact that there is a book (I
assume you are referring to Angelika's and Klaus' book) doesn't mean
they are complicated themselves, just that you can do lots of things
with them.
- ... and I don't know at all what you are referring to as being "unsafe".

In one discussion about this topic in [no.it.programmering.c++] we
measured performance that was, as I recall, 20 (yes, 20!) or more times
worse than alternatives (for std::stringstream conversion to text).

Unfortunately, I can't read Norwegian and all I could locate was a test
where single letters were combined (the thread is from early this year).
With those test, I could not reproduce a 20 times performance hit, "only"
one of 2. However, the alternatives were the way to go anyway since
concatenation of strings does not really require any form of stream.

Other than this, I'm aware of really bad stringstream implementations
which grow by a constant number of bytes (e.g. the one on the system we
are using grows by 128 bytes) resulting in a quadratic complexity where
it should be linear. Contemporary implementations fix this problem.

 

[Alf P. Steinbach]

* Dietmar Kuehl:

Sorry if my question was ambiguous. What I meant to ask was what you are
using instead of using IOStreams. ... and even I/O was not precise as I
intended to include formatting and the corresponding parsing.

I'm not. But previously C++ wrappers around the C standard library have
served me well, and for i/o also wrappers around native OS functionality.
Generally C++ lib = unacceptable in all respects, C func = so so but
portable, OS API = fast but non-portable, and these are the main options.

I don't think "inherent" makes it a pure rhetorical question: your
statement is that IOStreams have bad performance. This, at least to me,
implies that there is something inherently wrong with IOStreams which
makes it impossible to implement them such that they are at least as
efficient as an alternative approach.

Well no, on two counts. "inherent" is something you can point at, but
you cannot point at an emergent property from the, uh, design. And
"impossible" is too strong: I believe it is theoretically possible
to make anything fast, for example, if there are no inherent limits such
as O(2^n) behavior; the question is how hard it is to do that in practice.

I happen to disagree with such a statement

So do I... ;-)

and since pure disagreement does not provide a good techical
argument, I have actually implemented the IOStreams (and locales)
library of the standard C++ library. Although I have not optimized all
areas of this implementation, I think it for most uses roughly equivalent
to typical stdio implementations.

Splicing in comments out of order:

In one discussion about this topic in [no.it.programmering.c++] we
measured performance that was, as I recall, 20 (yes, 20!) or more times
worse than alternatives (for std::stringstream conversion to text).

Unfortunately, I can't read Norwegian and all I could locate was a test
where single letters were combined (the thread is from early this year).
With those test, I could not reproduce a 20 times performance hit, "only"
one of 2. However, the alternatives were the way to go anyway since
concatenation of strings does not really require any form of stream.

The reason I did not provide a link was that it was "hard" to find. I
have still not found the real meat of the discussion, but I think it
followed this posting, <url: http://tinyurl.com/2z8ap>, where I reported
a factor of 17 times slower for boost::lexical_cast compared to a function
written by Thore Karlsen, using Visual C++. In direct follow-up Tarjei
Knapstad then reported a factor of 15 using gcc, and 33 using Intel.

17, 15 and 33, with typical compilers.

The difference being that boost::lexical_cast uses std::stringstream
while Thore's function used, as I recall, itoa or sprintf or direct
conversion (his website seems to be down so difficult to check).

Just to address things:
- I disagree about there not being a design, actually I consider it to
be quite good since independent issues are handled by different
entities: i18n (i.e. location dependent variation in formatting) is
handled by locales, access to "external" destinations by stream buffers,
and formatting by the respective classes. What is wrong with this? The
fact that people refuse to at least look at the design and not to try to
derive from ostream is hardly the fault of the design...
A few rarely used things are not optimal (e.g. the callback mechanism
for resource management of associated data; today this would probably
use a shared pointer or something like this). However, as said, they
are rarely used and rarely necessary.

I think the single most illuminating comment is the one in the standard,
from memory, "if the object is destroyed before it is initialized by
calling the init function the effect is undefined"...

Some of it could perhaps be fixed by choosing better names, e.g. as it is
now 'clear( flag )' _sets_ the specified flag(s) (and clears all others).

But that would just be fiddling with cosmetic issues. For example, how to
make streams work when exception throwing is enabled? Then encountering
end of file when reading from a file will throw an exception...

- I don't argue about typical IOStream *implementations* being indeed
slow. Actually, this is something I complain about to the various
library implementors and I try to help them in improving their
implementation. However, I don't think there is any inherent performance
problem. At least, I'm not aware of any.

That is exactly my position, with one little difference: I think the
in-practice is very important.

- Complicated?
Yes.


- ... and I don't know at all what you are referring to as being "unsafe".

Random sampling of issues: raw pointers (buffer overflow issues); not
supporting exceptions (see above); encoding failure of single operation as
failure _state_; ignoring (skipping) input; accepting arguments which will
not be treated according to type (e.g. on a wide stream); etc. The "etc."
is mostly ignorance and laziness on my part -- I remember encountering the
issues and concluding, but I won't delve into the dirty details again just
to find examples: when a car has square wheels it doesn't really matter
that there's also much wrong with engine, etc. I avoid the standard
streams except when writing small test programs or recommending ways to do
things for novices (there is a small subset of functionality that is
suitable for novices who need some kind of type-safety in "Hello, there").