Problem with streambuf

[Matt Chaplain]

Hi there.

I'm writing a program that uses the Telnet protocol over TCP/IP
sockets. Of course, that has no bearing here, so I'll rephrase that
in Standard C++

In essense, I'm trying to manipulate a data stream by using a custom
streambuf, so that I can access this as "just another iostream".
Alas, this data stream has a couple of "interesting" properties:

1) It is non-blocking. There is a chance that, when reading from this
stream, there will be a flag of "no data". This doesn't mean "there
will not ever be any data", just "nothing right now, check back
later".

2) 0 is a meaningful value.

Combined, I am unsure of how to proceed. Sending negative numbers
seems not to work: Returning -1 in the case of no data sets fail bits
and looks identical to EOF (indeed, I believe it is the value returned
in the case of an EOF, at least on my platform). Returning -2 is
detectable when reading from the stream as strm.getc(), but tends
towards garbage when using the >> operator.

In short, does anybody know how to communicate "no data" from
streambuf to stream, or would I have to write a custom stream too?

Thanks,

Matt Chaplain

 

[John Harrison]

Hi there.

I'm writing a program that uses the Telnet protocol over TCP/IP
sockets. Of course, that has no bearing here, so I'll rephrase that
in Standard C++

In essense, I'm trying to manipulate a data stream by using a custom
streambuf, so that I can access this as "just another iostream".
Alas, this data stream has a couple of "interesting" properties:

1) It is non-blocking. There is a chance that, when reading from this
stream, there will be a flag of "no data". This doesn't mean "there
will not ever be any data", just "nothing right now, check back
later".

2) 0 is a meaningful value.

Combined, I am unsure of how to proceed. Sending negative numbers
seems not to work: Returning -1 in the case of no data sets fail bits
and looks identical to EOF (indeed, I believe it is the value returned
in the case of an EOF, at least on my platform). Returning -2 is
detectable when reading from the stream as strm.getc(), but tends
towards garbage when using the >> operator.

In short, does anybody know how to communicate "no data" from
streambuf to stream, or would I have to write a custom stream too?

Thanks,

Matt Chaplain

The mechanism to use nonblocking input from a stream is to use the readsome
method.

To implement the readsome method correctly you must implement showmanyc on
your streambuf.

None of this means that a user who does uses one of the blocking input
methods (e.g. operator>>, get or read) will get non-blocking input. Even
peek is a blocking input method. To get non-blocking input you must use
readsome.

john

 

[tom_usenet]

Hi there.

I'm writing a program that uses the Telnet protocol over TCP/IP
sockets. Of course, that has no bearing here, so I'll rephrase that
in Standard C++

In essense, I'm trying to manipulate a data stream by using a custom
streambuf, so that I can access this as "just another iostream".
Alas, this data stream has a couple of "interesting" properties:

1) It is non-blocking. There is a chance that, when reading from this
stream, there will be a flag of "no data". This doesn't mean "there
will not ever be any data", just "nothing right now, check back
later".

This is still EOF - it just means that you can clear EOF and try again
later, if you so choose.

2) 0 is a meaningful value.

This is true of all standard streams.

Combined, I am unsure of how to proceed. Sending negative numbers
seems not to work: Returning -1 in the case of no data sets fail bits
and looks identical to EOF (indeed, I believe it is the value returned
in the case of an EOF, at least on my platform). Returning -2 is
detectable when reading from the stream as strm.getc(), but tends
towards garbage when using the >> operator.

Returning traits_type::eof() is the correct thing to do if you can't
get any more data without blocking, and you don't want to block.
However, to make sure you don't lose bytes, clients will have to stick
to the istream::read function - operator>> functions may end up
throwing away data if EOF is reached in the middle of the operation.

It would be nice if operator>> functions at least attempted to return
a stream to its former state if the operation failed (using putback),
but implementations don't seem to bother since the interfaces aren't
set up to do it easily.

You could make sure that your streambuf can handle a reasonable number
of putback calls, and that your own operator>> functions put back any
characters they had read if they failed, and then the lost character
problem wouldn't occur.

The final solution is to ensure that the stream will return enough
characters to complete any input operation (e.g. only whole things
will be written, and can thus be read back without blocking), in which
case you'll only lose characters if something has gone "properly"
wrong.

In short, does anybody know how to communicate "no data" from
streambuf to stream, or would I have to write a custom stream too?

Just return EOF, and remind users that they have to clear the stream
if they want more data.

Tom

 

[Dietmar Kuehl]

2) 0 is a meaningful value.

You mean the character '0' is a meaningful value? Sure, this is always the
case. This is the reason why there is a distinction between "char_type"
which is used to represent all meaningful values and "int_type" which is
used to represent all meaningful value plus one value indicating that there
is no such value which is called "EOF".

Note that for some character types, the char_type and the int_type are
actually identical: for example, for 'wchar_t' on 32 bit systems where
'wchar_t' has 32 bits (eg. when using gcc), the corresponding int_type is
also 'wchar_t': 32 bits are sufficient to encode all characters (eg. all
Unicode charactesr) and still have a distinct value for EOF.

Combined, I am unsure of how to proceed. Sending negative numbers
seems not to work: Returning -1 in the case of no data sets fail bits
and looks identical to EOF (indeed, I believe it is the value returned
in the case of an EOF, at least on my platform).

-1 is the typical choice for EOF but it is not a requirement. You should
use a symbolic name anyway.

Returning -2 is
detectable when reading from the stream as strm.getc(), but tends
towards garbage when using the >> operator.

Well, the -2 should be used as a normal character when returned from
functions like 'std::streambuf::sgetc()': the IOStream classes form two
groups of values returned from 'sgetc()' and family:

- normal characters which are identified by being distinct from EOF
- EOF or, more precisely, "traits_type::eof()"

Thus, to indicate an expected problem (like "no more data [yet]"), you would
simply return EOF: this is what these functions use to detect failures.
.... and it results in a failure to attempt reading a character when there is
none: there is different processing depending on whether there were
characters or not. Since the IOStream functions set error flags when EOF is
returned, you should simply 'clear()' these error flags.

In short, does anybody know how to communicate "no data" from
streambuf to stream, or would I have to write a custom stream too?

I would use EOF to signal "no data" or, as I would formulated it, "[current]
end of file". On streams which might receive new data in the future, I would
then just clear the error flags: since I need to do different processing
in the two cases (there was a character or there was none) there is no
problem anyway. The only interesting issue is how to signal "real" end of
file as is indicated eg. on POSIX systems by a -1 return from 'read()' rather
than a '0' return in case there is currently no data. A possible approach is
to throw an exception in this case from 'underflow()' or whatever functions
you have overridden and explicitly check for 'badbit' in the streams.

Personally, I would also override 'showmanyc()' to check whether there is a
character in the buffer or if there is a character available via 'read()'.
This way, normal processing could look something like this:

if (stream.in_avail())
...

rather than 'clear()'ing the stream after each failed attempted: a read
attempt is only made when it is known that there is at least a character.
However, if the a priori check is not made, reading a character will, of
course, still fail. That is, a client knowing that the stream is non-blocking
and may receive characters later even if currently none are available will
take advantage of this. However, any other approach also has to an impact on
the client side be it by 'clear()'ing the state flags, by testing for special
value, etc.

 

[Matt Chaplain]

You mean the character '0' is a meaningful value? Sure, this is always the
case. This is the reason why there is a distinction between "char_type"
which is used to represent all meaningful values and "int_type" which is
used to represent all meaningful value plus one value indicating that there
is no such value which is called "EOF".

Indeed, but I just threw this out to pre-empt the otherwise possible
response of returning '0' in case of "no data" which would, in
essence, create an empty string.

Note that for some character types, the char_type and the int_type are
actually identical: for example, for 'wchar_t' on 32 bit systems where
'wchar_t' has 32 bits (eg. when using gcc), the corresponding int_type is
also 'wchar_t': 32 bits are sufficient to encode all characters (eg. all
Unicode charactesr) and still have a distinct value for EOF.

I'll admit I hadn't given the whole portability issue much thought,
what with using common, but ultimately platform-dependent code behind
this streambuf, and I don't think it's even possible to use wchar_t
for the purpose I'm putting it to, though it would be an interesting
ability. Feel free to correct me.

Returning -2 is
detectable when reading from the stream as strm.getc(), but tends
towards garbage when using the >> operator.

Well, the -2 should be used as a normal character when returned from
functions like 'std::streambuf::sgetc()': the IOStream classes form two
groups of values returned from 'sgetc()' and family:

- normal characters which are identified by being distinct from EOF
- EOF or, more precisely, "traits_type::eof()"

Thus, to indicate an expected problem (like "no more data [yet]"), you would
simply return EOF: this is what these functions use to detect failures.

So the nitty-gritty of it is: sgetc returns one value from one of two
sets: {all characters} and {EOF}. There is no way of signalling
anything else.

I would use EOF to signal "no data" or, as I would formulated it, "[current]
end of file". On streams which might receive new data in the future, I would
then just clear the error flags: since I need to do different processing
in the two cases (there was a character or there was none) there is no
problem anyway. The only interesting issue is how to signal "real" end of
file

Unfortunately, I missed off a couple of requirements: that "no current
data" (Fake EOF) must be distinguishable from "no data ever" (Real
EOF), and that the user should not need very much (if any) extraneous
code to handle the stream. In the ideal case, no-data should be
returned as an empty string, but this seems not to be possible.

A possible approach is to throw an exception in this case from 'underflow()'
or whatever functions you have overridden and explicitly check for 'badbit'
in the streams.

It is unlikely that having no data is an exceptional condition. In
fact, it's more likely the expected norm for the purpose I'm going to
put it to.

Personally, I would also override 'showmanyc()' to check whether there is a
character in the buffer or if there is a character available via 'read()'.
This way, normal processing could look something like this:

if (stream.in_avail())
...

I'd never heard of showmanyc before posting my question, since I tend
to look in Josuttis first before Stroustrup, where it's completely
overlooked.

Anyway, this seems like the best solution: to allow the cases that
require the non-blocking to explicitly check for it and guard against
the blocking calls, with a minimum of effort.

Thanks everyone who replied.

Matt Chaplain