block on reading a half-filled buffer for ifstream

[inkapyrite]

Hi all. I'm using ifstream to read from a named pipe but
i've encountered an annoying problem. For some reason, the
program blocks on reading an ifstream's internal buffer that's
only half-filled. Only when the buffer becomes full does
it resume execution.

Here's my test code for reading from a pipe:
//(compiled with g++ -std=c++98)
//---------------------------------------------
#include <iostream>
#include <fstream>
using namespace std;

const int BUFF_SIZE = 20;

int main()
{
char buffer[BUFF_SIZE+1];
std::ifstream fin("npipe");

std::streambuf * pbuf;
pbuf = fin.rdbuf();
pbuf->pubsetbuf(buffer,BUFF_SIZE);

char c[200];
while(1){

if(pbuf->in_avail() > 0) {
cout << "stuff in buffer: " << pbuf->in_avail() << endl;

fin >> c; //program blocks HERE if buffer not full

cout << "read in stuff : " << c << endl;
cout << "---------------------------" << endl;
}
else
usleep(500000);
}
fin.close();
}
//---------------------------------------------



And for writing to the pipe, i just use:

cat > npipe

on the terminal.



So when i type something down the pipe and hit the return key,
the program does immediately recognise that there's data
available (cos pbuf->in_avail() immediately becomes > 0).

But when i try to read in the data with >> operator, it blocks if
the data being sent does not completely fill up the buffer. So
in the above case, the program blocks until BUFF_SIZE chars have
been sent down the pipe.

incidentally, i tried other read functions in place of >>
but it yield the same result.

//fin.read(c,pbuf->in_avail()); //none of these work either
//fin.readsome(c, 1);
//char test= pbuf->sbumpc();


It is unlikely that it's a pipe problem because i've written
a separate program that uses the C's FILE pointer for reading
where I don't encounter this problem at all.

My question is: doesn't in_avail() specify the number of
characters that are collected and can be read from ifstream's buffer
without blocking? Is there any way i can read a
half-filled streambuf without it blocking on me? Or have i
misunderstood the concept of in_avail() + the readability of
a buffer here?

As a temporary make-do solution, i've set pubsetbuf(buffer,1);
so that whenever a char is received the buffer becomes full, which
kinda resembles an unbuffered stream. (and yes, i've tried
pubsetbuf(0,0) & it doesnt' work. see earlier post on this by
someone else.
http://groups.google.co.nz/groups?h...coff=1&selm=3EBA9C5F.6EC7CC19%40gmx.de&rnum=1
)

It would be nice if there's a way where i can set streambuf's
buffer size to something > 1 & still be able to read from
the half filled buffer without blocking.

Any suggestions?

Thanks heaps in advance,
Des

 

[Dietmar Kuehl]

std::ifstream fin("npipe");
std::streambuf * pbuf;
pbuf = fin.rdbuf();

while(1){
if(pbuf->in_avail() > 0) {

Whether this condition ever yields true is actually not
specified. Assuming that the file buffer implemetnation
merely does what the standard requires it returns the number
of characters in the input buffer, i.e. the number if knows
are present before blocking: this is what 'in_avail()' is
supposed to return. Since no character was read, no attempt
is made to read a character by 'in_avail()' since this could
block. If an implementation ever yields something different
from 'in_avail()' than '0' prior to the first read, you are
already relying on behavior which is not required by the
standard!

fin >> c; //program blocks HERE if buffer not full

If this block even though 'in_avail()' returned a value bigger
than zero, it has a very simple reason: the input of the
string only terminates when it detects a space character or
when it detect end of file. Whether there is a space in
currently available buffer or not is not told by 'in_avail()'.
.... and the end of the internal buffer does not count as EOF.
Thus, I'm actually a little puzzled why a buffer size of one
character should solve the problem. I would rather guess that
this assures that the newline on which "cat" sents its internal
buffer reaches the file buffer's buffer and allows the input
of string to terminate.

BTW, if you use the formatted input inserter for character
arrays you should setup a width for the stream to have a point
where input terminates prior to a buffer overrun: if you
receive a sequence of non-spaces which longer than the size of
your buffer, you get a buffer overrun. Probably, you should
better use input to a 'std::string' anyway: this does not have
this error prone feature. If you insist in using a character
array, use it like this:

std::size_t const size = 20;
char buf[size];
fin >> std::setw(size) >> buf;

It is safe to use the same size as the size of the buffer with
'std::setw()' because the stream will only read 'width() - 1'
characters and use the remaining character for a terminating
zero.

My question is: doesn't in_avail() specify the number of
characters that are collected and can be read from ifstream's buffer
without blocking?

No. It does something related: it specifies the number of
characters known to be available. It may return a smaller
number. The default implementation is to return the number of
characters in the stream buffer's buffer. If no character is
read, it is likely to return 0.

Is there any way i can read a half-filled streambuf without
it blocking on me?

Yes but it is not covered by the C++ standard: the standard has
no concept of non-blocking input. You would need to set your
stream to non-blocking using an environment specific approach,
e.g. fcntl(2), to allow non-blocking input. This might mean
that you need to create a simple file stream (it's easy: it is
just 10 lines of code) because you cannot assume that you can
fiddle with a file buffers file descriptor.

Or have i misunderstood the concept of in_avail() + the
readability of a buffer here?

I think you did. You also misunderstood how formatted input is
working.