InputStreamReader reads data only partially

[Markus Barchfeld]

When I create an InputStreamReader on an InputStream obtained from a
UrlConnection, I do not get all the data until I enter a sleep-call.
This seems to be a bug. Has anybody an explanation or a more
sophisticated workaround?
Thanks

Here is the code:

HttpURLConnection urlCon = (HttpURLConnection) new
URL("").openConnection();
urlCon.setRequestMethod("POST");
urlCon.setDoOutput(true);
// .. code for writing request data
urlCon.connect();
InputStream in = urlCon.getInputStream();
InputStreamReader inReader = new InputStreamReader(in);
StringBuffer content = new StringBuffer();
char[] chars = new char[1<<16];
while (inReader.ready()) {
int read = inReader.read(chars);
content.append(chars,0,read);
// the following sleep is necessary to obtain all the data
// from the InputStream, otherwise this loop exits before all
// the data has been read from the InputStream
try {
Thread.sleep(500) ;
} catch (InterruptedException e) {}
}
inReader.close() ;

 

[Gordon Beaton]

When I create an InputStreamReader on an InputStream obtained from a
UrlConnection, I do not get all the data until I enter a sleep-call.
This seems to be a bug. Has anybody an explanation or a more
sophisticated workaround?

TCP sends a stream of data that may be more or less continuous.
Depending on how fast you read, you may "run out" of data for a short
moment until more arrives. This is expected behaviour and normally not
a problem.

InputStreamReader.ready() tells you that there is data available to be
read. But because of the above, it will be false several times before
you reach the end of the stream. Your problem is that you use it to
determine when to quit.

My suggestion is that you lose the call to ready() altogether. It
gains you nothing at all, while at the same time making it difficult
for you to detect EOF.

This is an efficient and simple way to read all of the data:

BufferedReader br = new BufferedReader(inReader);
String line;

while ((line = br.readLine()) != null) {
content.append(line);
}

br.close();

Neither sleep() nor ready() are necessary.

/gordon

 

[Dave Monroe]

When I create an InputStreamReader on an InputStream obtained from a
UrlConnection, I do not get all the data until I enter a sleep-call.
This seems to be a bug. Has anybody an explanation or a more
sophisticated workaround?
Thanks

Here is the code:

HttpURLConnection urlCon = (HttpURLConnection) new
URL("").openConnection();
urlCon.setRequestMethod("POST");
urlCon.setDoOutput(true);
// .. code for writing request data
urlCon.connect();
InputStream in = urlCon.getInputStream();
InputStreamReader inReader = new InputStreamReader(in);
StringBuffer content = new StringBuffer();
char[] chars = new char[1<<16];
while (inReader.ready()) {
int read = inReader.read(chars);
content.append(chars,0,read);
// the following sleep is necessary to obtain all the data
// from the InputStream, otherwise this loop exits before all
// the data has been read from the InputStream
try {
Thread.sleep(500) ;
} catch (InterruptedException e) {}
}
inReader.close() ;

I'm guessing you're looking at network latency.

Basically, you can't read it if it ain't there.

I'll suggest wrapping a BufferedReader around the InputStreamReader
and use the BufferedReader's readLine() method. That seems to handle
latency consistently for me.


Hope this helps.

Dave Monroe

 

[Igor Buzatovic]

here is the code I wrote for my application, this is method which reads all
bytes from the inputstreamof any kind.
It shows you how to read all bytes from stream no matter how slow is the
stream:

public static byte[] getAllBytes(InputStream in) throws Exception
{
int chunkSize=4096;
byte[] b=new byte[chunkSize];
int borb=-1;
ByteArrayOutputStream fos=new ByteArrayOutputStream();
while((borb=in.read(b))!=-1)
{
if(borb==chunkSize)
fos.write(b);
else
fos.write(b,0,borb);
}
return fos.toByteArray();
}

 

[Markus Barchfeld]

Thanks for your answers. I am now using the solution from igor, because
it might be more efficient than the readline solution (although: who knows?).
I had short glance at the implementation of InputStreamReader, but could
not find out, why the ready() method does not handle the EOF properly while
the read() method does (both use the available() method of the underlying stream).

Now the code looks like that and works perfectly:

StringBuffer content = new StringBuffer();
char[] chars = new char[1<<16];
int read ;
while ((read = inReader.read(chars)) != -1) {
content.append(chars,0,read);
}

 

[Gordon Beaton]

I had short glance at the implementation of InputStreamReader, but
could not find out, why the ready() method does not handle the EOF
properly while the read() method does (both use the available()
method of the underlying stream).

ready() (or available()) can be false (or 0) for two different
reasons:

- at EOF
- when the reader has gotten ahead of the writer, so that there is
currently no data available to be read

The only way to tell the difference between these cases is to actually
call read(). If you only call read() when there's data available, you
won't call it at EOF and therefore won't detect that EOF has occurred.

Also, unless you're doing some other task in the same thread, there's
no reason *not* to simply block in read() (or one of its variants),
which will return when data has been read or you're at EOF.

/gordon