Java networking, 50000 sockets

Discussion in 'Java' started by millerch, Apr 21, 2005.

  1. millerch

    millerch Guest

    Has anyone tried to monitor 50000 socket connections in java?
    I have an application that needs to do just that.

    I can open 50000 sockets with no problems, but things get unstable when
    I try to listen to 5000 to 6000 of them.

    My configuration is a combination of 50 threads to listen to 1 socket
    each, 500 threads pooled and waiting for the ServerSocket.accept(), and
    2500 listening for data on 20 sockets each. Tested on Red Hat Enterprise
    up to 6000 sockets. I am pretty sure there no deadlocks, or anything
    strange.

    Has anyone done something similar? Is this idea even sane?

    Thanks
     
    millerch, Apr 21, 2005
    #1
    1. Advertising

  2. millerch

    millerch Guest

    Esmond Pitt wrote:
    > millerch wrote:
    >
    >> Has anyone tried to monitor 50000 socket connections in java?
    >> I have an application that needs to do just that.
    >>
    >> I can open 50000 sockets with no problems, but things get unstable when
    >> I try to listen to 5000 to 6000 of them.
    >>
    >> My configuration is a combination of 50 threads to listen to 1 socket
    >> each, 500 threads pooled and waiting for the ServerSocket.accept(), and
    >> 2500 listening for data on 20 sockets each. Tested on Red Hat Enterprise
    >> up to 6000 sockets. I am pretty sure there no deadlocks, or anything
    >> strange.

    >
    >
    > I agree with Betty. You should most certainly be using NIO for this, but
    > in the above I don't see what you need the first 50 threads mentioned
    > for. What are they doing?
    >
    > Also why do you need 50,000 listening sockets? Wouldn't one do? If not
    > why not?


    Thanks guys I'll look into NIO. By the way its 2500 threads at 20
    sockets each(asynchronous) which makes 50,000 and 50 threads at 1 socket
    each (synchronous), that puts the total at 50,050 sockets.#]

    Thanks again!
    Millerch
     
    millerch, Apr 21, 2005
    #2
    1. Advertising

  3. millerch

    Betty Guest

    "millerch" <> wrote in message
    news:...
    > Has anyone tried to monitor 50000 socket connections in java?
    > I have an application that needs to do just that.
    >
    > I can open 50000 sockets with no problems, but things get unstable when
    > I try to listen to 5000 to 6000 of them.
    >
    > My configuration is a combination of 50 threads to listen to 1 socket
    > each, 500 threads pooled and waiting for the ServerSocket.accept(), and
    > 2500 listening for data on 20 sockets each. Tested on Red Hat Enterprise
    > up to 6000 sockets. I am pretty sure there no deadlocks, or anything
    > strange.
    >
    > Has anyone done something similar? Is this idea even sane?
    >
    > Thanks


    Have you tried NIO?
     
    Betty, Apr 21, 2005
    #3
  4. millerch

    Esmond Pitt Guest

    millerch wrote:

    > Has anyone tried to monitor 50000 socket connections in java?
    > I have an application that needs to do just that.
    >
    > I can open 50000 sockets with no problems, but things get unstable when
    > I try to listen to 5000 to 6000 of them.
    >
    > My configuration is a combination of 50 threads to listen to 1 socket
    > each, 500 threads pooled and waiting for the ServerSocket.accept(), and
    > 2500 listening for data on 20 sockets each. Tested on Red Hat Enterprise
    > up to 6000 sockets. I am pretty sure there no deadlocks, or anything
    > strange.


    I agree with Betty. You should most certainly be using NIO for this, but
    in the above I don't see what you need the first 50 threads mentioned
    for. What are they doing?

    Also why do you need 50,000 listening sockets? Wouldn't one do? If not
    why not?
     
    Esmond Pitt, Apr 22, 2005
    #4
  5. In article <>,
    millerch <> wrote:

    > Has anyone tried to monitor 50000 socket connections in java?
    > I have an application that needs to do just that.
    >
    > I can open 50000 sockets with no problems, but things get unstable when
    > I try to listen to 5000 to 6000 of them.
    >
    > My configuration is a combination of 50 threads to listen to 1 socket
    > each, 500 threads pooled and waiting for the ServerSocket.accept(), and
    > 2500 listening for data on 20 sockets each. Tested on Red Hat Enterprise
    > up to 6000 sockets. I am pretty sure there no deadlocks, or anything
    > strange.
    >
    > Has anyone done something similar? Is this idea even sane?
    >
    > Thanks


    There doesn't have to be a 1:1 relationship between Java threads and
    system threads/LWPs. When using extremely large numbers of threads,
    it's possible for the native threads to be depleted on blocking I/O.
    When and how that happens depends on the system configuration.

    As another mentioned, NIO will allow you to have just a few listeners
    that dispatch tasks to workers.

    If that still has problems, check the file descriptor limit and the OS's
    abilities. Some OSes (Solaris) croak when there are a lot of sockets.
     
    Kevin McMurtrie, Apr 23, 2005
    #5
  6. millerch

    Esmond Pitt Guest

    millerch wrote:

    > Esmond Pitt wrote:
    >
    >>millerch wrote:
    >>
    >>
    >>>Has anyone tried to monitor 50000 socket connections in java?
    >>>I have an application that needs to do just that.
    >>>
    >>>I can open 50000 sockets with no problems, but things get unstable when
    >>>I try to listen to 5000 to 6000 of them.
    >>>
    >>>My configuration is a combination of 50 threads to listen to 1 socket
    >>>each, 500 threads pooled and waiting for the ServerSocket.accept(), and
    >>>2500 listening for data on 20 sockets each. Tested on Red Hat Enterprise
    >>>up to 6000 sockets. I am pretty sure there no deadlocks, or anything
    >>>strange.

    >>
    >>
    >>I agree with Betty. You should most certainly be using NIO for this, but
    >>in the above I don't see what you need the first 50 threads mentioned
    >>for. What are they doing?
    >>
    >>Also why do you need 50,000 listening sockets? Wouldn't one do? If not
    >>why not?

    >
    >
    > Thanks guys I'll look into NIO. By the way its 2500 threads at 20
    > sockets each(asynchronous) which makes 50,000 and 50 threads at 1 socket
    > each (synchronous), that puts the total at 50,050 sockets.#]


    I'm still curious why you need 50,000 listening sockets. Does this mean
    you are providing 50,000 different services in one JVM?

    Or is it 50 listening sockets and up to 1000 connections each on
    average? in which case my question is still why the 50? - i.e. are you
    really implementing 50 different servics in one JVM?
     
    Esmond Pitt, Apr 23, 2005
    #6
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Rahul Sharma

    Is there Java.Networking group

    Rahul Sharma, Jul 23, 2003, in forum: Java
    Replies:
    6
    Views:
    412
    Jezuch
    Jul 23, 2003
  2. Robert Olofsson

    Re: Java I/O for Networking

    Robert Olofsson, Aug 22, 2003, in forum: Java
    Replies:
    2
    Views:
    407
    Robert Olofsson
    Aug 24, 2003
  3. VisionSet

    java to java networking

    VisionSet, Nov 19, 2003, in forum: Java
    Replies:
    0
    Views:
    281
    VisionSet
    Nov 19, 2003
  4. Steve R. Burrus

    Need help w. Java Networking.

    Steve R. Burrus, Jun 5, 2004, in forum: Java
    Replies:
    12
    Views:
    768
    Bryce
    Jun 7, 2004
  5. Ju Hui
    Replies:
    12
    Views:
    417
    Ju Hui
    May 6, 2006
Loading...

Share This Page