couple of questions on streams

[Mantorok Redgormor]

7.19.3p7

"At program startup, three text streams are predefined and need not
be opened explicitly — standard input (for reading conventional
input), standard output (for writing conventional output), and
standard error
(for writing diagnostic output). As initially opened, the standard
error stream is not fully buffered; the standard input and standard
output streams are fully buffered if and only if the stream can be
determined not to refer to an interactive device."

What is fully buffered? I was looking at some archived
posts and some experts were using fully buffered as
though it was different from line based buffering.

Isn't line based buffering synonymous with fully buffered?


An interactive device is implementation defined but it states
that standard input and output streams are fully buffered
if it can be determined if they do not refer to an interactive
device. Can't we consider entering input by way of a terminal
to be an interactive device?

Lastly, because of the previous thing mentioned, this means that
something like a standard hello world program is not strictly
conforming unless an explicit call to setvbuf is made with
_IOFBF. Is my conclusion correct?

 

[Richard Bos]

7.19.3p7

As initially opened, the standard error stream is not fully buffered;
the standard input and standard output streams are fully buffered if
and only if the stream can be determined not to refer to an interactive
device."

What is fully buffered?

Four paragraphs up, 7.19.3#3.

# When a stream is fully buffered, characters are intended to be
# transmitted to or from the host environment as a block when a buffer
# is filled.

Isn't line based buffering synonymous with fully buffered?
No.

Lastly, because of the previous thing mentioned, this means that
something like a standard hello world program is not strictly
conforming unless an explicit call to setvbuf is made with
_IOFBF.

I don't see how that follows from the text, or even from your assumption
that fully buffered equals line-buffered. If a stream _is_ opened as
fully buffered from the start of the program, I think you may assume
that a buffer is also provided at the same time.

Richard

 

[Flash Gordon]

On 26 Oct 2004 23:24:11 -0700

7.19.3p7

"At program startup, three text streams are predefined and need not
be opened explicitly — standard input (for reading conventional
input), standard output (for writing conventional output), and
standard error
(for writing diagnostic output). As initially opened, the standard
error stream is not fully buffered; the standard input and standard
output streams are fully buffered if and only if the stream can be
determined not to refer to an interactive device."

What is fully buffered? I was looking at some archived
posts and some experts were using fully buffered as
though it was different from line based buffering.

Isn't line based buffering synonymous with fully buffered?

Fully buffered typically means buffering at least at the block level
(i.e. 512 octets of data, for example).

An interactive device is implementation defined but it states
that standard input and output streams are fully buffered
if it can be determined if they do not refer to an interactive
device. Can't we consider entering input by way of a terminal
to be an interactive device?

A terminal is an interactive device. However, on many systems stdin,
stdout & stderr can all be redirected to files. For example
# prog < in.txt > out.txt 2> err.txt
Alternatively the streams could be connected to pipes, something which
is frequently done in Unix.

Lastly, because of the previous thing mentioned, this means that
something like a standard hello world program is not strictly
conforming unless an explicit call to setvbuf is made with
_IOFBF. Is my conclusion correct?

I can't answer that. However, I would also point out that the output
routines can fail for any reason, so you are never guaranteed to get
*any* output.--
Flash Gordon
Sometimes I think shooting would be far too good for some people.
Although my email address says spam, it is real and I read it.

 

[Malcolm]

What is fully buffered? I was looking at some archived
posts and some experts were using fully buffered as
though it was different from line based buffering.

Isn't line based buffering synonymous with fully buffered?

You don't need to worry about this. In the olden days there were low-level
(unbuffered) and high-level (buffered|) IO functions. You could tweak the
low-level functions to squeeze out an extra bit of performance. In these
modern times it is no longer easy to dot this, and you may assume that the
stdio routines manage data as efficiently as humanly possible.
There is probably a difference between "line based buffering" and "full
buffering", but I have no motivation to find out what it is.

Lastly, because of the previous thing mentioned, this means that
something like a standard hello world program is not strictly
conforming unless an explicit call to setvbuf is made with
_IOFBF. Is my conclusion correct?

No. IO can fail so strictly you should check the return from printf() to see
if the call succeeded. No-one ever does this. IO can still fail if you set
the buffer with setvbuf(). No-one ever does this unless doing very
specialised and time critical IO.

 

[Eric Sosman]

You don't need to worry about this. In the olden days there were low-level
(unbuffered) and high-level (buffered|) IO functions. You could tweak the
low-level functions to squeeze out an extra bit of performance. In these
modern times it is no longer easy to dot this, and you may assume that the
stdio routines manage data as efficiently as humanly possible.
There is probably a difference between "line based buffering" and "full
buffering", but I have no motivation to find out what it is.

Since you don't know what the difference is, it
seems a little rash to tell someone else they needn't
worry about it ... "Here's my advice on something I
don't comprehend" is not a preamble that inspires a
lot of confidence.

For the record, there is a difference and it is
sometimes important. The C Standard describes three
kinds of buffering (in 7.19.3, paragraph 3). You can
read the section for yourself (strongly advised, if
you're going to dish out advice about it), or you can
accept this paraphrase:

An *unbuffered* stream deals in single characters.
Output characters are delivered to the destination
as soon as possible, and characters generated at
an input device are made available to an input
stream as soon as possible.

A *fully-buffered* stream deals in "large" blocks
of characters. Output characters are accumulated
in a buffer until it fills, and are then delivered
to the destination all at once. Input characters
are similarly accumulated until "enough" are ready,
and are then made available to an input stream all
at once.

A *line-buffered* stream deals in newline-terminated
lines. Output characters are accumulated in a buffer
until a newline is written, when they're all sent to
the destination at once. Input characters are gathered
until an end-of-line occurs, at which point all the
line's characters are made available to the input
stream.

Since the Standard doesn't try to get into the finicky
details of the host environment's I/O capabilities, the
actual language is filled with phrases like "intended to"
and "implementation-defined" and "as soon as possible."
Still, the intent is clear and an implementor will probably
expend some effort to do something that makes sense for the
platform in light of the Standard's stated intentions.

No. IO can fail so strictly you should check the return from printf() to see
if the call succeeded. No-one ever does this. IO can still fail if you set
the buffer with setvbuf(). No-one ever does this unless doing very
specialised and time critical IO.

Now that you understand C's three buffering modes, you
may be able to see why they might be useful in other than
"specialised" and "time critical" situations. Notice that
the Standard specifies (in paragraph 7 of the same section)
the buffering modes of the three standard streams; I think
we can assume that if the Standard takes the trouble to
specify the modes, the modes are far from "specialised."

 

[Michael Wojcik]

In the olden days there were low-level
(unbuffered) and high-level (buffered|) IO functions. You could tweak the
low-level functions to squeeze out an extra bit of performance. In these
modern times it is no longer easy to dot this, and you may assume that the
stdio routines manage data as efficiently as humanly possible.

Eric has kindly demolished most of this already, but I thought it
worth pointing out, for any readers who might be laboring under the
same misapprehensions as Malcolm, that many C implementations today
still run in environments with both "low-level" and "high-level" I/O,
and with "buffered" and "unbuffered" I/O, and where it is still as
easy as it ever was to "tweak" the "low-level" flavors of I/O.

Further, while anyone certainly may assume that a given C stdio
implementation is "as efficient[] as humanly possible", that
assumption is frequently unjustified. For one thing, efficiency is
measured differently for different requirements; since not all
programs have the same needs, they do not all require the same things
of an I/O implementation.

--
Michael Wojcik (e-mail address removed)

_
| 1
| _______ d(cabin) = log cabin + c = houseboat
| (cabin)
_| -- Thomas Pynchon

 

[CBFalconer]

.... snip ...

Eric has kindly demolished most of this already, but I thought it
worth pointing out, for any readers who might be laboring under the
same misapprehensions as Malcolm, that many C implementations today
still run in environments with both "low-level" and "high-level" I/O,
and with "buffered" and "unbuffered" I/O, and where it is still as
easy as it ever was to "tweak" the "low-level" flavors of I/O.

Barring the presence of independant i/o processors, the presence of
an unbuffered output mechanism at some level is guaranteed. From
there on other things are done to make the system more docile and
generally usable, which often include buffers, and often generate
problems in error reporting.

In general you don't worry about this, you just use the facilities
guaranteed by the language standard. In the rare case where you
have to worry about it, you are going to be writing system
dependant code anyhow, so isolate it.

 

[Malcolm]

I think we need to tackle this one. In the sense that there are more
varieties of compiler that run on little embedded systems than hosted
environments, this is correct. In the sense that someone learning C is
likely to use such a system, it is not. You can assume virtually unlimited
memory and a processor speed of many MegaHertz

In general you don't worry about this, you just use the facilities
guaranteed by the language standard. In the rare case where you
have to worry about it, you are going to be writing system
dependant code anyhow, so isolate it.

Thank you. If you don't use stdio you are not writing standard C, and stdio
is generally buffered. The "demolition" was just silly sniping.

 

[Michael Wojcik]

I think we need to tackle this one. In the sense that there are more
varieties of compiler that run on little embedded systems than hosted
environments, this is correct. In the sense that someone learning C is
likely to use such a system, it is not.

All POSIX systems offer "low-level" and "unbuffered" I/O facilities
to their hosted C implementations; it's required by the standard
(currently the "Austin Group" standard, which reconciles POSIX, SUS,
and some others).

I think it's entirely likely that someone will learn to use C on such
a system. Care to provide any evidence otherwise?

You can assume virtually unlimited
memory and a processor speed of many MegaHertz

You can make that assumption, but it would be a stupid one. On POSIX
systems, for example, processes with ordinary privileges are generally
run with resource limits in force, so their available storage is most
definitely constrained.

Thank you. If you don't use stdio you are not writing standard C, and stdio
is generally buffered.

Who claimed otherwise? Of course, this claim isn't particularly
useful. If you perform I/O by using facilities outside those
provided by the standard library, you no longer have a strictly
conforming program, but nearly all useful C programs aren't strictly
conforming.

The "demolition" was just silly sniping.

Justified sniping at silliness was what it was. Your original post
was wrong and misleading.

 

[Malcolm]

All POSIX systems offer "low-level" and "unbuffered" I/O facilities
to their hosted C implementations; it's required by the standard
(currently the "Austin Group" standard, which reconciles POSIX, SUS,
and some others).

I think it's entirely likely that someone will learn to use C on such
a system. Care to provide any evidence otherwise?

Any decent C system will ofer some access to low-level routines for doing
wonderful things with the OS. Generally these can be ignored. This didn't
use to be true, for instance in the 286 days you had to access the screen
buffer directly to get decent performance in a full-screen app (the conio
routines were too slow). Nowadays it is usually a waste of time trying to
speed up a program by replacing stdio calls with your own routines that call
low-level ops directly. Note I said usually, if you are implementing a
database engine or something similar then maybe it would be worthwhile.

Who claimed otherwise? Of course, this claim isn't particularly
useful. If you perform I/O by using facilities outside those
provided by the standard library, you no longer have a strictly
conforming program, but nearly all useful C programs aren't strictly
conforming.

Most real program are not strictly conforming, but most modules within those
programs either are strictly conforming or could be made strictly conforming
with a minimum of effort, and this would greatly improve the design.

Justified sniping at silliness was what it was. Your original post
was wrong and misleading.

There are two ways of being misleading. One is to post incorrect
information, the other is to be technically correct but to have no awareness
of the intended audience

" Now that you understand C's three buffering modes, you
may be able to see why they might be useful in other than
"specialised" and "time critical" situations. Notice that
the Standard specifies (in paragraph 7 of the same section)
the buffering modes of the three standard streams; I think
we can assume that if the Standard takes the trouble to
specify the modes, the modes are far from "specialised." "

There is nothing factually wrong here, but is it useful to someone who
thinks that "Hello world" might not be strictly conforming unless he messes
about with the buffering modes? I think not.