BufferedReader.readLine() blocks unexpected

[roele]

I have a method where i will handle an Input/Output Stream from a
ServerSocket. The Client sends several requests and these should be
handled in one connection.
The first request is handled as expected but the seconds request is
blocked by the requestReader.readLine() method... According to JavaDoc
i cant find anything about readLine() blocking.

Has anybody a clue what could be wrong here?


--
public void handleRequest() throws IOException {

BufferedReader requestReader = new BufferedReader(new
InputStreamReader(socket.getInputStream));
OutputStream responseStream = new
BufferedOutputStream(socket.getOutputStream());

do {

Request request = new Request();
Response response = new Response();

parseRequest(requestReader, request);

processRequest(request, response);

writeResponse(responseStream, response);

}while(keepAlive && !hasError);

request.close();
response.close();
socket.close();

}

private void parseRequest(HttpRequest request) throws IOException {

String reqString = requestReader.readLine();

//.. do something with String here

//Read on
while(requestReader.ready()) {
String reqHeaders = requestReader.readLine();
//Reached end?
if(reqHeaders == null || reqHeaders.length() == 0) {
break;
}
//.. do something with String here
}

}
--

 

[rossum]

I have a method where i will handle an Input/Output Stream from a
ServerSocket. The Client sends several requests and these should be
handled in one connection.
The first request is handled as expected but the seconds request is
blocked by the requestReader.readLine() method... According to JavaDoc
i cant find anything about readLine() blocking.

Has anybody a clue what could be wrong here?


--
public void handleRequest() throws IOException {

BufferedReader requestReader = new BufferedReader(new
InputStreamReader(socket.getInputStream));
OutputStream responseStream = new
BufferedOutputStream(socket.getOutputStream());

do {

Request request = new Request();
Response response = new Response();

parseRequest(requestReader, request);

processRequest(request, response);

writeResponse(responseStream, response);

}while(keepAlive && !hasError);

request.close();
response.close();
socket.close();

}

private void parseRequest(HttpRequest request) throws IOException {

String reqString = requestReader.readLine();

//.. do something with String here

//Read on
while(requestReader.ready()) {
String reqHeaders = requestReader.readLine();
//Reached end?
if(reqHeaders == null || reqHeaders.length() == 0) {
break;
}
//.. do something with String here
}

}

The documentation says: "Reads a line of text. A line is considered to
be terminated by any one of a line feed ('\n'), a carriage return
('\r'), or a carriage return followed immediately by a linefeed."

I have had problems in similar circumstances when the incoming stream
was not terminated with and end-of-line character. This might be what
is happening here if readLine is waiting for the current line to
complete before returning and in the absence of an end-of-line it just
keeps waiting and does not return.

rossum

 

[Gordon Beaton]

I have a method where i will handle an Input/Output Stream from a
ServerSocket. The Client sends several requests and these should be
handled in one connection.

The first request is handled as expected but the seconds request is
blocked by the requestReader.readLine() method... According to
JavaDoc i cant find anything about readLine() blocking.

Has anybody a clue what could be wrong here?

BufferedReader.readLine() *will* block until the sender does one of
the following:

- sends a complete line, or
- closes the socket, causing readLine() to return null, indicating EOF.

So if the remote still has an open connection, then make sure it ends
*every* message with a newline.

I see too that you unwisely use ready() to check before calling
readLine(). Solutions involving Reader.ready() or
InputStream.available() are often fragile; they can stop reading
prematurely (i.e. mid-message) and can miss EOF entirely. It's better
to simply block while reading, or to use a Selector if you really need
non-blocking behaviour (which this example doesn't).

/gordon

--

 

[roele]

The documentation says: "Reads a line of text. A line is considered to
be terminated by any one of a line feed ('\n'), a carriage return
('\r'), or a carriage return followed immediately by a linefeed."

I have had problems in similar circumstances when the incoming stream
was not terminated with and end-of-line character. This might be what
is happening here if readLine is waiting for the current line to
complete before returning and in the absence of an end-of-line it just
keeps waiting and does not return.

rossum

But shouldn't it at least throw an exception or return null then?

 

[Gordon Beaton]

But shouldn't it at least throw an exception or return null then?

At what point should readLine() decide that a newline is not on the
way, in order to throw that exception or return null?

EOF is clearly defined, it will occur when the sender closes his end.
readLine() *will* return null at that point.

Failing that, a newline is required, and the BufferedReader will wait
patiently for it to arrive.

/gordon

--

 

[roele]

At what point should readLine() decide that a newline is not on the
way, in order to throw that exception or return null?

EOF is clearly defined, it will occur when the sender closes his end.
readLine() *will* return null at that point.

Failing that, a newline is required, and the BufferedReader will wait
patiently for it to arrive.

/gordon

--

Yes right. In my case i tried to read HTTP GET requests from the input
stream which should end with CRLF. I guess there might be any
unconsumed bytes left on the stream or something that causes the
blocking...

 

[Rogan Dawes]

I have a method where i will handle an Input/Output Stream from a
ServerSocket. The Client sends several requests and these should be
handled in one connection.
The first request is handled as expected but the seconds request is
blocked by the requestReader.readLine() method... According to JavaDoc
i cant find anything about readLine() blocking.

Has anybody a clue what could be wrong here?


--
public void handleRequest() throws IOException {

BufferedReader requestReader = new BufferedReader(new
InputStreamReader(socket.getInputStream));
OutputStream responseStream = new
BufferedOutputStream(socket.getOutputStream());

do {

Request request = new Request();
Response response = new Response();

parseRequest(requestReader, request);

processRequest(request, response);

writeResponse(responseStream, response);

}while(keepAlive && !hasError);

request.close();
response.close();
socket.close();

}

private void parseRequest(HttpRequest request) throws IOException {

String reqString = requestReader.readLine();

//.. do something with String here

//Read on
while(requestReader.ready()) {
String reqHeaders = requestReader.readLine();
//Reached end?
if(reqHeaders == null || reqHeaders.length() == 0) {
break;
}
//.. do something with String here
}

}

As mentioned by many people, BufferedReader will block until the CR or
LF is seen. From another email, it sounds like you are trying to write
an HTTP server. I suggest you read RFC2616 carefully to understand
exactly what is involved.

For example, a GET request is defined as a request line
(<METHOD><SP><URL>[<SP><VERSION>]<CRLF>) where <SP><VERSION> is optional
and if missing indicates HTTP/0.9. This is followed by zero or more
header lines, defined as <HEADERNAME>:<SP>*<HEADERVALUE><CRLF>. Finally,
the request is terminated by a blank line (<CRLF>).

This is relatively simple to parse.

However, a POST request is a different beast completely. The basic
structure is similar, starting with a request line. However, a POST
request should also have a header that indicates the Content-Length
(i.e. how many bytes to read after the blank line), or alternatively,
the Chunked Transfer-Encoding mechanism may be used (Transfer-Encoding:
chunked), which tells the server to read the size of the next chunk,
followed by a <CRLF>, then that size bytes, repeated until a chunk of
zero bytes is read.

This was just a snippet from the RFC (from the top of my head, so don't
shoot me if I got some details wrong) - the point is that it is a
COMPLICATED standard, so trying to implement it without reading the
standard is just silly.

Hope this helps.

Rogan

 

[Rogan Dawes]

I have a method where i will handle an Input/Output Stream from a
ServerSocket. The Client sends several requests and these should be
handled in one connection.
The first request is handled as expected but the seconds request is
blocked by the requestReader.readLine() method... According to JavaDoc
i cant find anything about readLine() blocking.

Has anybody a clue what could be wrong here?


--
public void handleRequest() throws IOException {

BufferedReader requestReader = new BufferedReader(new
InputStreamReader(socket.getInputStream));
OutputStream responseStream = new
BufferedOutputStream(socket.getOutputStream());

do {

Request request = new Request();
Response response = new Response();

parseRequest(requestReader, request);

processRequest(request, response);

writeResponse(responseStream, response);

}while(keepAlive && !hasError);

request.close();
response.close();
socket.close();

}

private void parseRequest(HttpRequest request) throws IOException {

String reqString = requestReader.readLine();

//.. do something with String here

//Read on
while(requestReader.ready()) {
String reqHeaders = requestReader.readLine();
//Reached end?
if(reqHeaders == null || reqHeaders.length() == 0) {
break;
}
//.. do something with String here
}

}

As mentioned by many people, BufferedReader will block until the CR or
LF is seen. From another email, it sounds like you are trying to write
an HTTP server. I suggest you read RFC2616 carefully to understand
exactly what is involved.

For example, a GET request is defined as a request line
(<METHOD><SP><URL>[<SP><VERSION>]<CRLF>) where <SP><VERSION> is optional
and if missing indicates HTTP/0.9. This is followed by zero or more
header lines, defined as <HEADERNAME>:<SP>*<HEADERVALUE><CRLF>. Finally,
the request is terminated by a blank line (<CRLF>).

This is relatively simple to parse.

However, a POST request is a different beast completely. The basic
structure is similar, starting with a request line. However, a POST
request should also have a header that indicates the Content-Length
(i.e. how many bytes to read after the blank line), or alternatively,
the Chunked Transfer-Encoding mechanism may be used (Transfer-Encoding:
chunked), which tells the server to read the size of the next chunk,
followed by a <CRLF>, then that size bytes, repeated until a chunk of
zero bytes is read.

This was just a snippet from the RFC (from the top of my head, so don't
shoot me if I got some details wrong) - the point is that it is a
COMPLICATED standard, so trying to implement it without reading the
standard is just silly.

Hope this helps.

Rogan

P.S. Also keep in mind that the BufferedReader fills its buffer (duh!),
so it may also read bytes from the socket that are part of a subsequent
request. This becomes a problem when dealing with POST's which may
contain arbitrary binary data, that should really be read from a raw
InputStream, and not converted to characters by the InputStreamReader
and then buffered.

 

[Roedy Green]

According to JavaDoc
i cant find anything about readLine() blocking.

I would expect it to block. How could it give you a partial line?

 

[Lew]

roele said:

According to JavaDoc
i [sic] cant find anything about readLine() blocking.

You sometimes have to dig through the Javadocs to get to the answer.

In general, each read request made of a Reader causes a corresponding read request to be made of the underlying character or byte stream.

OK, what does the underlying Reader's read() do? For that matter,
BufferedReader extends Reader:

<http://java.sun.com/javase/6/docs/api/java/io/Reader.html>

This method will block until a character is available, an I/O error occurs, or the end of the stream is reached.

This method will block until some input is available, an I/O error occurs, or the end of the stream is reached.

This method will block until some input is available, an I/O error occurs, or the end of the stream is reached.

QED.

You have to think through - if it says it's wrapping another method, then the
behavior of the wrapped method is relevant.

 

[Patricia Shanahan]

Lew wrote:
....

You have to think through - if it says it's wrapping another method,
then the behavior of the wrapped method is relevant.

There is a more direct way of looking at this. readLine's own
documentation says it does one of three things: throws an IOException to
report an error, returns null to indicate end of file, or returns a line.

readLine's documentation does not permit it to return until it either
has detected end of file or it has a complete line.

Patricia