RMI connection refused... eventually

Discussion in 'Java' started by Qu0ll, Dec 10, 2007.

  1. Qu0ll

    Qu0ll Guest

    I have a simple RMI server and a stress testing application is able to
    connect to it about 400 times and then suddenly future connection attempts
    result in a connection refused exception.

    What would be the possible reasons for refusing connection when obviously
    connection is permitted by the security manager and firewall initially? Is
    there some parameter that controls the maximum number of RMI connections? I
    don't think it is memory related as the server is running with a 1GB heap
    and varying it makes no difference.

    This is the exception:

    java.rmi.ConnectException: Connection refused to host: 10.1.1.3; nested
    exception is:
    java.net.ConnectException: Connection refused: connect
    at sun.rmi.transport.tcp.TCPEndpoint.newSocket(TCPEndpoint.java:601)
    at
    sun.rmi.transport.tcp.TCPChannel.createConnection(TCPChannel.java:198)
    at
    sun.rmi.transport.tcp.TCPChannel.newConnection(TCPChannel.java:184)
    at sun.rmi.server.UnicastRef.invoke(UnicastRef.java:110)
    at com.qu0ll.ServerDaemon_Stub.registerApplet(Unknown Source)
    at com.qu0ll.StressTester$AppletThread.run(StressTester.java:73)
    Caused by: java.net.ConnectException: Connection refused: connect
    at java.net.PlainSocketImpl.socketConnect(Native Method)
    at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:333)
    at
    java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:195)
    at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:182)
    at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:366)
    at java.net.Socket.connect(Socket.java:519)
    at java.net.Socket.connect(Socket.java:469)
    at java.net.Socket.<init>(Socket.java:366)
    at java.net.Socket.<init>(Socket.java:180)
    at
    sun.rmi.transport.proxy.RMIDirectSocketFactory.createSocket(RMIDirectSocketFactory.java:22)
    at
    sun.rmi.transport.proxy.RMIMasterSocketFactory.createSocket(RMIMasterSocketFactory.java:128)
    at sun.rmi.transport.tcp.TCPEndpoint.newSocket(TCPEndpoint.java:595)
    ... 5 more

    --
    And loving it,

    -Q
    _________________________________________________

    (Replace the "SixFour" with numbers to email me)
    Qu0ll, Dec 10, 2007
    #1
    1. Advertising

  2. Qu0ll

    Esmond Pitt Guest

    Qu0ll wrote:
    > I have a simple RMI server and a stress testing application is able to
    > connect to it about 400 times and then suddenly future connection
    > attempts result in a connection refused exception.


    Is the server a Windows box and the client a Unix box? Windows has a
    nasty habit of issuing resets when the listen backlog queue fills, and a
    compensating behaviour in the Windows implementation of connect(). AFAIK
    Unix connects() don't have that so when they get a reset you get it.
    Esmond Pitt, Dec 10, 2007
    #2
    1. Advertising

  3. Qu0ll

    Nigel Wade Guest

    Qu0ll wrote:

    > I have a simple RMI server and a stress testing application is able to
    > connect to it about 400 times and then suddenly future connection attempts
    > result in a connection refused exception.
    >
    > What would be the possible reasons for refusing connection when obviously
    > connection is permitted by the security manager and firewall initially? Is
    > there some parameter that controls the maximum number of RMI connections? I
    > don't think it is memory related as the server is running with a 1GB heap
    > and varying it makes no difference.
    >
    > This is the exception:
    >
    > java.rmi.ConnectException: Connection refused to host: 10.1.1.3; nested
    > exception is:
    > java.net.ConnectException: Connection refused: connect



    I saw exactly the same thing when I was doing applet/servlet comms. My applet on
    startup had the option to load the last 24hrs of data. It was programmed to
    read each data record on a new socket. If I did this then watched the network
    connections using netstat I could see that Windows wasn't closing the sockets
    when they were actually closed, instead it seemed to "batch" the closes. There
    would be hundreds of sockets in the TIME_WAIT state, then a whole load of them
    would all close together.

    This caused problems when the sockets were being opened faster than the
    "batched" closes shut them down. The system quickly reached its limit on open
    sockets. Attempting a new connection whilst the system is in this state results
    in connection refused.

    This situation didn't occur on Linux, it shuts down sockets when they are
    actually closed. I presume it's just the way Microsoft have implemented their
    TCP/IP stack (or if you are believer in conspiracies, they've done it
    deliberately to encourage you to buy the much more expensive server versions of
    Windows).

    --
    Nigel Wade, System Administrator, Space Plasma Physics Group,
    University of Leicester, Leicester, LE1 7RH, UK
    E-mail :
    Phone : +44 (0)116 2523548, Fax : +44 (0)116 2523555
    Nigel Wade, Dec 11, 2007
    #3
  4. Qu0ll

    Qu0ll Guest

    "Nigel Wade" <> wrote in message
    news:fjlp5c$g2s$...
    > Qu0ll wrote:
    >
    >> I have a simple RMI server and a stress testing application is able to
    >> connect to it about 400 times and then suddenly future connection
    >> attempts
    >> result in a connection refused exception.
    >>
    >> What would be the possible reasons for refusing connection when obviously
    >> connection is permitted by the security manager and firewall initially?
    >> Is
    >> there some parameter that controls the maximum number of RMI connections?
    >> I
    >> don't think it is memory related as the server is running with a 1GB heap
    >> and varying it makes no difference.
    >>
    >> This is the exception:
    >>
    >> java.rmi.ConnectException: Connection refused to host: 10.1.1.3; nested
    >> exception is:
    >> java.net.ConnectException: Connection refused: connect

    >
    >
    > I saw exactly the same thing when I was doing applet/servlet comms. My
    > applet on
    > startup had the option to load the last 24hrs of data. It was programmed
    > to
    > read each data record on a new socket. If I did this then watched the
    > network
    > connections using netstat I could see that Windows wasn't closing the
    > sockets
    > when they were actually closed, instead it seemed to "batch" the closes.
    > There
    > would be hundreds of sockets in the TIME_WAIT state, then a whole load of
    > them
    > would all close together.
    >
    > This caused problems when the sockets were being opened faster than the
    > "batched" closes shut them down. The system quickly reached its limit on
    > open
    > sockets. Attempting a new connection whilst the system is in this state
    > results
    > in connection refused.
    >
    > This situation didn't occur on Linux, it shuts down sockets when they are
    > actually closed. I presume it's just the way Microsoft have implemented
    > their
    > TCP/IP stack (or if you are believer in conspiracies, they've done it
    > deliberately to encourage you to buy the much more expensive server
    > versions of
    > Windows).


    Thanks Nigel - I figured it had something to do with a limitation of
    Windows. It's actually a Vista x64 machine that I am running it on at the
    moment. So, I guess then I should be hosting this server on a Linux
    machine? Is that the only way around this problem?

    --
    And loving it,

    -Q
    _________________________________________________

    (Replace the "SixFour" with numbers to email me)
    Qu0ll, Dec 11, 2007
    #4
  5. Qu0ll

    Lew Guest

    Qu0ll wrote:
    > Thanks Nigel - I figured it had something to do with a limitation of
    > Windows. It's actually a Vista x64 machine that I am running it on at
    > the moment. So, I guess then I should be hosting this server on a Linux
    > machine? Is that the only way around this problem?


    Certainly not. There's Solaris, FreeBSD, ...

    --
    Lew
    Lew, Dec 11, 2007
    #5
  6. Qu0ll

    Nigel Wade Guest

    Qu0ll wrote:

    > "Nigel Wade" <> wrote in message
    > news:fjlp5c$g2s$...
    >> Qu0ll wrote:
    >>
    >>> I have a simple RMI server and a stress testing application is able to
    >>> connect to it about 400 times and then suddenly future connection
    >>> attempts
    >>> result in a connection refused exception.
    >>>
    >>> What would be the possible reasons for refusing connection when obviously
    >>> connection is permitted by the security manager and firewall initially?
    >>> Is
    >>> there some parameter that controls the maximum number of RMI connections?
    >>> I
    >>> don't think it is memory related as the server is running with a 1GB heap
    >>> and varying it makes no difference.
    >>>
    >>> This is the exception:
    >>>
    >>> java.rmi.ConnectException: Connection refused to host: 10.1.1.3; nested
    >>> exception is:
    >>> java.net.ConnectException: Connection refused: connect

    >>
    >>
    >> I saw exactly the same thing when I was doing applet/servlet comms. My
    >> applet on
    >> startup had the option to load the last 24hrs of data. It was programmed
    >> to
    >> read each data record on a new socket. If I did this then watched the
    >> network
    >> connections using netstat I could see that Windows wasn't closing the
    >> sockets
    >> when they were actually closed, instead it seemed to "batch" the closes.
    >> There
    >> would be hundreds of sockets in the TIME_WAIT state, then a whole load of
    >> them
    >> would all close together.
    >>
    >> This caused problems when the sockets were being opened faster than the
    >> "batched" closes shut them down. The system quickly reached its limit on
    >> open
    >> sockets. Attempting a new connection whilst the system is in this state
    >> results
    >> in connection refused.
    >>
    >> This situation didn't occur on Linux, it shuts down sockets when they are
    >> actually closed. I presume it's just the way Microsoft have implemented
    >> their
    >> TCP/IP stack (or if you are believer in conspiracies, they've done it
    >> deliberately to encourage you to buy the much more expensive server
    >> versions of
    >> Windows).

    >
    > Thanks Nigel - I figured it had something to do with a limitation of
    > Windows. It's actually a Vista x64 machine that I am running it on at the
    > moment. So, I guess then I should be hosting this server on a Linux
    > machine? Is that the only way around this problem?
    >


    The problem I had was on the client, not the server. The server was/is running
    Linux. I have no idea how a server process running under Windows (or Windows
    itself) would react to that level of socket creation/destruction. You can
    easily see if this is the problem by running netstat in a command shell. This
    should show you every network connection, and its current state. If you see
    lots of sockets in one of the shutdown states (TIMED_WAIT, CLOSE_WAIT,
    FIN_WAIT_1/2 etc.) then you have a similar problem to mine.

    Is your stress test a realistic load? Is there a way to change the load so that
    it doesn't require the creation/destruction of sockets at such a high rate? I
    thought one of the improvements in Vista was the TCP/IP stack, although it may
    well be that this method of closing sockets down is more efficient for a client
    which isn't expected to be opening/closing sockets at such a high rate. After
    all, this is not a normal load for a client and the Windows most people have is
    the client version.

    I worked around the problem by changing the protocol so the client didn't need
    to create (and destroy) so many sockets.

    --
    Nigel Wade, System Administrator, Space Plasma Physics Group,
    University of Leicester, Leicester, LE1 7RH, UK
    E-mail :
    Phone : +44 (0)116 2523548, Fax : +44 (0)116 2523555
    Nigel Wade, Dec 11, 2007
    #6
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. VisionSet

    RMI - connection refused listener

    VisionSet, Sep 6, 2004, in forum: Java
    Replies:
    7
    Views:
    2,812
    Sudsy
    Sep 6, 2004
  2. Daniel Klein
    Replies:
    1
    Views:
    290
    Fredrik Lundh
    Nov 16, 2006
  3. Duane Evenson

    RMI & connection refused

    Duane Evenson, Aug 2, 2009, in forum: Java
    Replies:
    10
    Views:
    8,481
    Nigel Wade
    Aug 5, 2009
  4. FutureScalper

    RMI connection refused

    FutureScalper, Apr 30, 2010, in forum: Java
    Replies:
    14
    Views:
    3,813
    Esmond Pitt
    Jun 24, 2010
  5. Jacks
    Replies:
    3
    Views:
    88
    Brian McCauley
    Jan 27, 2005
Loading...

Share This Page