Detecting EOF

[Scott W Gifford]

Hello,

I'm working on a streaming data server which accepts a query, then
sends any matching data that comes in after the query was submitted.

The problem I'm having is that when a client sends a query then later
goes away, I'm not always able to detect the situation to cancel their
query. I can usually detect a client is gone when I get EOF when
reading or writing to it, but I have nothing to read after the initial
query, and as long as no data matches the query, nothing to write
either.

My implementation uses a BufferedReader and BufferedWriter to interact
with the client, so I can use readLine(). In general these will be
connected to a socket, but for testing they're connected to pipes.

In Perl or C, I can use select or a nonblocking file descriptor to
check if EOF has been reached. How can I do the same thing in Java?

I've tried using BufferedReader.ready() and BufferedReader.read(new
char[1], 0, 0), but neither indicates whether I've reached EOF. If I
instead use BufferedReader.read(new char[1], 0, 1), it tells me about
the EOF, but blocks. I tried creating a seperate thread to do a
blocking read, and that works fine for pipes, but for some reason it
breaks sockets (my writes all block). I've also looked at the nio
classes, but I'm reluctant to change my interfaces (currently I accept
a Reader/Writer pair or a Socket to communicate over), and anyways it
looks like massive overkill for detecting EOF.

Ideally I'd like a way to get interrupted when an EOF happens, by
interrupt() or an EOF exception or whatever. If that's not possible,
I can periodically poll the socket if I haven't written anything to
it.

Thanks for any ideas or hints!

----Scott.

 

[Thomas Hawtin]

The problem I'm having is that when a client sends a query then later
goes away, I'm not always able to detect the situation to cancel their
query. I can usually detect a client is gone when I get EOF when
reading or writing to it, but I have nothing to read after the initial
query, and as long as no data matches the query, nothing to write
either.

My implementation uses a BufferedReader and BufferedWriter to interact
with the client, so I can use readLine(). In general these will be
connected to a socket, but for testing they're connected to pipes.

In Perl or C, I can use select or a nonblocking file descriptor to
check if EOF has been reached. How can I do the same thing in Java?

@since 1.4, Socket has gained a few methods which you could poll. For
instance isOutputShutdown. I assume that does the obvious.

Presumably NIO wouldn't be too bad. Use Socket.getChannel, and register
that with a selector.

I've tried using BufferedReader.ready() and BufferedReader.read(new
char[1], 0, 0), but neither indicates whether I've reached EOF. If I
instead use BufferedReader.read(new char[1], 0, 1), it tells me about
the EOF, but blocks. I tried creating a seperate thread to do a
blocking read, and that works fine for pipes, but for some reason it
breaks sockets (my writes all block). I've also looked at the nio
classes, but I'm reluctant to change my interfaces (currently I accept
a Reader/Writer pair or a Socket to communicate over), and anyways it
looks like massive overkill for detecting EOF.

Using separate threads for reading and writing should work. If it hangs,
Ctrl-\ (or Ctrl-Break on Windows) from the console should show what the
problem is.

Tom Hawtin

 

[Scott W Gifford]

Hi Thomas,

@since 1.4, Socket has gained a few methods which you could poll. For
instance isOutputShutdown. I assume that does the obvious.

Thanks, I hadn't seen those methods. The problem is that the input
doesn't always come from a socket. My JUnit tests use pipes to test
parts of the system, and I'd like to keep that working if at all
possible. It doesn't look like PipedReader/PipedWriter implement
anything like isOutputShutdown; is there a way to convert them into
something that does?

I also don't see a way to create a non-Internet socket (like a Unix
domain socket, or a socketpair()-style socket) for testing. I guess I
could rig my test infrastructure up to use Internet sockets to
localhost, but that seems silly.

Presumably NIO wouldn't be too bad. Use Socket.getChannel, and
register that with a selector.

I haven't been able to figure out how to store an object that may be a
Socket or may be a Pipe with NIO. It looks like I can declare it as a
SelectableChannel, but that doesn't provide read or write methods; or
I can declare it as a Readable/WritableByteChannel, but that's not
selectable. Any ideas?

I've tried using BufferedReader.ready() and BufferedReader.read(new
char[1], 0, 0), but neither indicates whether I've reached EOF. If I
instead use BufferedReader.read(new char[1], 0, 1), it tells me about
the EOF, but blocks. I tried creating a seperate thread to do a
blocking read, and that works fine for pipes, but for some reason it
breaks sockets (my writes all block). I've also looked at the nio
classes, but I'm reluctant to change my interfaces (currently I accept
a Reader/Writer pair or a Socket to communicate over), and anyways it
looks like massive overkill for detecting EOF.

Using separate threads for reading and writing should work. If it
hangs, Ctrl-\ (or Ctrl-Break on Windows) from the console should show
what the problem is.

I tried; if I had a thread blocking on a read on the socket, then
writes to that socket blocked. When I don't create a reader thread,
everything works correctly. If that's not expected behavior, I can
probably put together a small test case to show the problem.

Thanks!

---Scott.

 

[Gordon Beaton]

The problem I'm having is that when a client sends a query then later
goes away, I'm not always able to detect the situation to cancel their
query. I can usually detect a client is gone when I get EOF when
reading or writing to it, but I have nothing to read after the initial
query, and as long as no data matches the query, nothing to write
either.

It is the nature of TCP that you can only detect EOF by actually
attempting to read from or write to the socket.

In Perl or C, I can use select or a nonblocking file descriptor to
check if EOF has been reached. How can I do the same thing in Java?

If you don't want to do a blocking read, then use a Selector and
expect OP_READ at EOF (then read to detect whether you're actually at
EOF). If it isn't feasible to use NIO, then use setSoTimeout() to set
a short read timeout on the Socket, then do a blocking read to poll
it.

Note that methods like ready() or available() can't be used to
reliably detect EOF because the state they report isn't unique to EOF.
The socket will be "not ready" any time there is no data currently
waiting to be read.

Methods like isInputShutdown(), isConnected() etc only tell you what
you've done with the Socket object itself (i.e. what methods have been
called), they don't say anything about the state of the underlying
socket.

/gordon

 

[Thomas Hawtin]

Thanks, I hadn't seen those methods. The problem is that the input
doesn't always come from a socket. My JUnit tests use pipes to test
parts of the system, and I'd like to keep that working if at all
possible. It doesn't look like PipedReader/PipedWriter implement
anything like isOutputShutdown; is there a way to convert them into
something that does?

You can always provide your own abstractions, but as this is just for
testing, we can note that Socket is not final...

I also don't see a way to create a non-Internet socket (like a Unix
domain socket, or a socketpair()-style socket) for testing. I guess I
could rig my test infrastructure up to use Internet sockets to
localhost, but that seems silly.

I don't think it's that silly.

I haven't been able to figure out how to store an object that may be a
Socket or may be a Pipe with NIO. It looks like I can declare it as a
SelectableChannel, but that doesn't provide read or write methods; or
I can declare it as a Readable/WritableByteChannel, but that's not
selectable. Any ideas?

Without an extra layer, you would need to get into the details of the
SPI, which wouldn't be fun. Or use a mocking tool.

I tried; if I had a thread blocking on a read on the socket, then
writes to that socket blocked. When I don't create a reader thread,
everything works correctly. If that's not expected behavior, I can
probably put together a small test case to show the problem.

Within Socket.socketWrite0 (or PlainSocketImpl.acquireFD)? Where and
what are both threads blocked on. It shouldn't behave like that. Here's
a little test program.

import java.io.*;
import java.net.*;
class SimpleServer {
public static void main(String[] args) throws Exception {
final ServerSocket server = new ServerSocket(6502);
final Socket socket = server.accept();
final OutputStream out = socket.getOutputStream();
final InputStream in = socket.getInputStream();
Thread inThread = new Thread(new Runnable() { public void run() {
System.out.println("in, start");
try {
while (in.read() != -1) {
System.out.println(":");
}
} catch (IOException exc) {
exc.printStackTrace();
} finally {
System.out.println("in, end");
}
}});
inThread.start();
for (; {
Thread.sleep(1000);
out.write('X');
}
}
}