Strange Socket problem?

Discussion in 'Java' started by Knute Johnson, Jul 18, 2007.

  1. Since upgrading to 1.6 I have seen on occasion a socket disconnect that
    isn't detected by the client end. No exception is thrown and the stream
    I/O stays blocked. The host end definitely shows the connection closed.
    I can't reproduce it which is going to be a problem if I want to file
    a bug report. Has anybody else seen this since upgrading to 1.6?

    --

    Knute Johnson
    email s/nospam/knute/
     
    Knute Johnson, Jul 18, 2007
    #1
    1. Advertising

  2. Knute Johnson

    Tom Hawtin Guest

    Knute Johnson wrote:
    > Since upgrading to 1.6 I have seen on occasion a socket disconnect that
    > isn't detected by the client end. No exception is thrown and the stream
    > I/O stays blocked. The host end definitely shows the connection closed.
    > I can't reproduce it which is going to be a problem if I want to file a
    > bug report. Has anybody else seen this since upgrading to 1.6?


    Are you sure it's not just coincidental that the occasional connection
    drops after you upgraded to 1.6? If there's a network problem it often
    wont show until sometime after a write from that socket, but that's
    nothing to do with Java.

    Tom Hawtin
     
    Tom Hawtin, Jul 18, 2007
    #2
    1. Advertising

  3. Tom Hawtin wrote:
    > Knute Johnson wrote:
    >> Since upgrading to 1.6 I have seen on occasion a socket disconnect
    >> that isn't detected by the client end. No exception is thrown and the
    >> stream I/O stays blocked. The host end definitely shows the
    >> connection closed. I can't reproduce it which is going to be a
    >> problem if I want to file a bug report. Has anybody else seen this
    >> since upgrading to 1.6?

    >
    > Are you sure it's not just coincidental that the occasional connection
    > drops after you upgraded to 1.6? If there's a network problem it often
    > wont show until sometime after a write from that socket, but that's
    > nothing to do with Java.
    >
    > Tom Hawtin


    I've got some software in the field that is now occasionally hanging
    when it didn't before. Although that problem is newer than the change
    to 1.6. I'm working on another program and the client in that pair has
    hung a few times on me while the host has definitely shown a disconnect.
    And it is very possible that it is all coincidental. The client end
    that has missed the disconnect in both cases has only been reading.
    Reads have always been a reliable way to detect disconnects in the past.

    I have put a timer into the software in the field to check to make sure
    that the read thread has terminated. We had an actual failure this
    morning and it was not corrected because the read thread did not
    terminate even when the socket was closed. No exception was thrown.
    This program is very complicated though and it is quite possible that my
    fix was not correct. I've got a new fix going in tonight (why is it
    that it always has to be late at night :).

    Oh and one more tidbit is that on the new project I'm working on the
    hang occurred with the host and client on the same machine.

    All the computers are running Windows XP which just adds to the
    entertainment value.

    Thanks very much for your interest.

    --

    Knute Johnson
    email s/nospam/knute/
     
    Knute Johnson, Jul 18, 2007
    #3
  4. Knute Johnson <> writes:
    :
    > Oh and one more tidbit is that on the new project I'm working on the
    > hang occurred with the host and client on the same machine.

    :

    This alone makes me suspect it's the application, the fact that it's XP
    notwithstanding (despite my misgivings, I would trust the average M$
    system programmer before the average application programmer, just :)

    Would it be difficult to introduce bidirectional heartbeats between
    the server and the client, with timeouts?

    m.
     
    Matt Atterbury, Jul 18, 2007
    #4
  5. Knute Johnson

    Tom Hawtin Guest

    Knute Johnson wrote:
    > Reads have always been a reliable way to detect disconnects in the past.


    Reads generally aren't great. They generate no traffic, so you wouldn't
    even expect the remote end to respond. You might get an ICMP no route to
    host message if you are lucky. What you can do is send some form of NOP
    periodically. (Can't remember if flush will actually send an empty packet.)

    > Oh and one more tidbit is that on the new project I'm working on the
    > hang occurred with the host and client on the same machine.


    So probably not a network failure then.

    Having a brief look at the code closing a socket close the file
    descriptor which then presumably that closes the streams. The path is
    direct, so I can't claim out of hand that it is your fault...

    Tom Hawtin
     
    Tom Hawtin, Jul 18, 2007
    #5
  6. Tom Hawtin wrote:
    > Knute Johnson wrote:
    >> Reads have always been a reliable way to detect disconnects in the past.

    >
    > Reads generally aren't great. They generate no traffic, so you wouldn't
    > even expect the remote end to respond. You might get an ICMP no route to
    > host message if you are lucky. What you can do is send some form of NOP
    > periodically. (Can't remember if flush will actually send an empty packet.)
    >
    >> Oh and one more tidbit is that on the new project I'm working on the
    >> hang occurred with the host and client on the same machine.

    >
    > So probably not a network failure then.
    >
    > Having a brief look at the code closing a socket close the file
    > descriptor which then presumably that closes the streams. The path is
    > direct, so I can't claim out of hand that it is your fault...
    >
    > Tom Hawtin


    Thanks Matt and Tom. I hate it when I have a lot of coincidental
    happenings. Makes me start to think that this stuff runs on voodoo.

    Two of my three programs in the field locked up again this morning so
    I've got another fix to put in tonight. One of the threads only reads
    so I'm going to try closing its socket instead of the output stream. I
    reorganized some other code so maybe I'll get it cured. I know that the
    server end has been having some problems as this code worked flawlessly
    for almost a year before it started giving me fits.

    Thanks again,

    --

    Knute Johnson
    email s/nospam/knute/
     
    Knute Johnson, Jul 19, 2007
    #6
  7. More in the continuing saga. It appears now that I'm really having a
    problem getting the BufferedInputStream.read() to throw an exception
    when the stream is closed. I've tried closing the input stream and the
    socket and it appears that neither is causing the exception to be thrown.

    --

    Knute Johnson
    email s/nospam/knute/
     
    Knute Johnson, Jul 19, 2007
    #7
  8. Knute Johnson

    Esmond Pitt Guest

    Knute Johnson wrote:
    > More in the continuing saga. It appears now that I'm really having a
    > problem getting the BufferedInputStream.read() to throw an exception
    > when the stream is closed. I've tried closing the input stream and the
    > socket and it appears that neither is causing the exception to be thrown.


    Are you closing the socket from another thread in the same JVM while a
    thread is blocked in the read? This isn't specified to work: it works
    on some platforms, not all. That's why
    java.nio.channels.InterruptibleChannel was introduced. IOW it works on
    Sockets that you get from SocketChannels but not (necessarily) on pure
    java.net.Sockets.
     
    Esmond Pitt, Jul 20, 2007
    #8
  9. Esmond Pitt wrote:
    > Are you closing the socket from another thread in the same JVM while a
    > thread is blocked in the read? This isn't specified to work: it works
    > on some platforms, not all. That's why
    > java.nio.channels.InterruptibleChannel was introduced. IOW it works on
    > Sockets that you get from SocketChannels but not (necessarily) on pure
    > java.net.Sockets.


    Yes. I'm not sure how you could do it from the same thread since it is
    blocked. Where is it specified not to work?

    Thanks,

    --

    Knute Johnson
    email s/nospam/knute/
     
    Knute Johnson, Jul 20, 2007
    #9
  10. Knute Johnson

    Esmond Pitt Guest

    Knute Johnson wrote:
    > Yes. I'm not sure how you could do it from the same thread since it is
    > blocked. Where is it specified not to work?


    It's not specified that it does work unless the object implements
    InterruptibleChannel.

    Use a read timeout of a couple of minutes and check a flag, or
    Thread.isInterrupted(), every time it triggers.
     
    Esmond Pitt, Jul 20, 2007
    #10
  11. Knute Johnson

    Graham Guest

    On 19 Jul, 19:29, Knute Johnson <>
    wrote:
    > More in the continuing saga. It appears now that I'm really having a
    > problem getting the BufferedInputStream.read() to throw an exception
    > when the stream is closed. I've tried closing the input stream and the
    > socket and it appears that neither is causing the exception to be thrown.
    >
    > --
    >
    > Knute Johnson
    > email s/nospam/knute/


    I've had problems with BufferedInputStream.read() on Windows XP
    machines with the problem you describe. Although a BufferedInputStream
    will not throw an exception, I seem to remember that it will reliably
    return a -1 to indicate that the end of the stream has been reached
    after the server has closed the socket (and if you perform another
    read it seems to block indefinitely).

    Are you definitely checking for this scenario? It was a while ago and
    unfortunately I don't have access to a Windows machine at the moment
    so I can't confirm it, but thought I would throw it in as a suggestion
    anyway.
     
    Graham, Jul 20, 2007
    #11
  12. Esmond Pitt wrote:
    > Knute Johnson wrote:
    >> Yes. I'm not sure how you could do it from the same thread since it
    >> is blocked. Where is it specified not to work?

    >
    > It's not specified that it does work unless the object implements
    > InterruptibleChannel.
    >
    > Use a read timeout of a couple of minutes and check a flag, or
    > Thread.isInterrupted(), every time it triggers.


    While that may be the case for NIO I don't think that is true for stream
    I/O. From the docs;

    Socket.close()

    "Any thread currently blocked in an I/O operation upon this socket will
    throw a SocketException.

    ....

    Closing this socket will also close the socket's InputStream and
    OutputStream"

    I don't know how you could call Socket.close() from the blocked thread
    so it would have to work from another.

    --

    Knute Johnson
    email s/nospam/knute/
     
    Knute Johnson, Jul 20, 2007
    #12
  13. Graham wrote:
    > On 19 Jul, 19:29, Knute Johnson <>
    > wrote:
    >> More in the continuing saga. It appears now that I'm really having a
    >> problem getting the BufferedInputStream.read() to throw an exception
    >> when the stream is closed. I've tried closing the input stream and the
    >> socket and it appears that neither is causing the exception to be thrown.
    >>
    >> --
    >>
    >> Knute Johnson
    >> email s/nospam/knute/

    >
    > I've had problems with BufferedInputStream.read() on Windows XP
    > machines with the problem you describe. Although a BufferedInputStream
    > will not throw an exception, I seem to remember that it will reliably
    > return a -1 to indicate that the end of the stream has been reached
    > after the server has closed the socket (and if you perform another
    > read it seems to block indefinitely).
    >
    > Are you definitely checking for this scenario? It was a while ago and
    > unfortunately I don't have access to a Windows machine at the moment
    > so I can't confirm it, but thought I would throw it in as a suggestion
    > anyway.
    >


    Graham:

    Thanks for the idea but yes I am checking for end of stream. I'm going
    to try some tests with that though on XP and Linux to see if there are
    differences.

    --

    Knute Johnson
    email s/nospam/knute/
     
    Knute Johnson, Jul 20, 2007
    #13
  14. Knute Johnson

    Esmond Pitt Guest

    Knute Johnson wrote:
    >> Use a read timeout of a couple of minutes and check a flag, or
    >> Thread.isInterrupted(), every time it triggers.


    When you've tried this let us know.
     
    Esmond Pitt, Jul 23, 2007
    #14
  15. Esmond Pitt wrote:
    > Knute Johnson wrote:
    >>> Use a read timeout of a couple of minutes and check a flag, or
    >>> Thread.isInterrupted(), every time it triggers.

    >
    > When you've tried this let us know.


    Then code has had read timeouts from the beginning. They work fine if
    there is a connection but when it messes up it never times out.

    --

    Knute Johnson
    email s/nospam/knute/
     
    Knute Johnson, Jul 23, 2007
    #15
  16. Knute Johnson

    Esmond Pitt Guest

    Knute Johnson wrote:
    > Then code has had read timeouts from the beginning. They work fine if
    > there is a connection but when it messes up it never times out.


    Sorry, I'm not getting this. How are you detecting the lost connection
    if not via a read timeout?
     
    Esmond Pitt, Jul 24, 2007
    #16
  17. Esmond Pitt wrote:
    > Knute Johnson wrote:
    >> Then code has had read timeouts from the beginning. They work fine if
    >> there is a connection but when it messes up it never times out.

    >
    > Sorry, I'm not getting this. How are you detecting the lost connection
    > if not via a read timeout?


    That's the point exactly. It isn't being detected. I put an alligator
    in to attempt to close the connection if it hangs. The alligator closes
    the socket. If it were working correctly, the blocked read would either
    return -1 or throw an IOException when the other end closes. It
    doesn't, that's what I've been wrestling with.

    --

    Knute Johnson
    email s/nospam/knute/
     
    Knute Johnson, Jul 24, 2007
    #17
  18. Knute Johnson

    Esmond Pitt Guest

    Knute Johnson wrote:
    > That's the point exactly. It isn't being detected. I put an alligator
    > in to attempt to close the connection if it hangs.


    'If it hangs' meaning what? Sorry, I'm not getting this.

    Anyway the read timeout should throw a SocketTimeoutException. I've
    never seen it not work. What platform?
     
    Esmond Pitt, Jul 24, 2007
    #18
  19. Esmond Pitt wrote:
    > Knute Johnson wrote:
    >> That's the point exactly. It isn't being detected. I put an
    >> alligator in to attempt to close the connection if it hangs.

    >
    > 'If it hangs' meaning what? Sorry, I'm not getting this.
    >
    > Anyway the read timeout should throw a SocketTimeoutException. I've
    > never seen it not work. What platform?


    The host end disconnects but the client end does not detect the
    disconnection. I know the complete data transfer will only last a few
    seconds so I wrote an alligator that closes the client socket to end the
    thread. But when this problem happens, closing the socket does not
    throw an exception or cause the read() to return and it will not
    timeout. The thread is hung.

    Platform is Windows XP.

    --

    Knute Johnson
    email s/nospam/knute/
     
    Knute Johnson, Jul 24, 2007
    #19
  20. Knute Johnson

    Esmond Pitt Guest

    and if the alligator isn't there does the read time out?
     
    Esmond Pitt, Jul 25, 2007
    #20
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Laszlo Nagy
    Replies:
    1
    Views:
    4,922
    Mark Wooding
    Jan 27, 2009
  2. Jean-Paul Calderone
    Replies:
    0
    Views:
    992
    Jean-Paul Calderone
    Jan 27, 2009
  3. Laszlo Nagy
    Replies:
    0
    Views:
    565
    Laszlo Nagy
    Feb 1, 2009
  4. Steve Holden
    Replies:
    0
    Views:
    683
    Steve Holden
    Feb 1, 2009
  5. Steve Holden
    Replies:
    1
    Views:
    730
Loading...

Share This Page