Maximum connections/sockets/threads? What's going on!

[Joshua Jung]

We are trying to test the maximum socket/connection load on our brand new
Mac Pro.

I have set up on the Mac Pro server a basic server application that waits
for connections and receives them.

On the client end, I have a program that can generate any number of
clients I desire and make them all connect at whatever interval I choose
(this way they aren't trying to connect too quickly one after another).

I am running into the weirdest problem. The server can receive ROUGHLY 376
connections and then, despite the fact that netstat says the port is
listening on the server, no clients (on any computers) can connect. They
spit out a

java.net.ConnectException: Operation Timed Out

error.

I have plenty of memory and CPU left on the server when it stops
receiving connections, and I know it probably has nothing to do with a
maximum socket setting because sometimes it will allow 376, sometimes
more, and sometimes fewer connections... so it can't be a limit on the
number of threads/sockets etc.

I am racking my brain trying to figure this out... and what the heck does
it mean when an operation times out rather than the connection timing out?

Josh <><

 

[opalpa]

How many file descriptors per process?

Things other than sockets use file descriptors (like open files) so the
number of socket connections you can have would be variable.

Mac Pro is a mac os X machine, correct? In which case it's a unix
machine, in which case ulimit -n tells you current file descriptor
setting.

Maybe this is the issue. All the best,
Opalinski
(e-mail address removed)
http://www.geocities.com/opalpaweb/

 

[Joshua Jung]

How many file descriptors per process?

Things other than sockets use file descriptors (like open files) so the
number of socket connections you can have would be variable.

Makes sense.

Mac Pro is a mac os X machine, correct? In which case it's a unix
machine, in which case ulimit -n tells you current file descriptor
setting.

It reports 256, which is nowhere near my maximum connection number of ~370
I'm willing to test this idea out, but how? Can I change this limit for
the entire operating system? I saw from documentation that changing the
limit only works for the current shell and its descendents, whatever
that means.

How do I see how many file descriptors a program is using?

Maybe this is the issue. All the best, Opalinski (e-mail address removed)

Thanks so much!

 

[opalpa]

It reports 256, which is nowhere near my maximum connection number of ~370
I'm willing to test this idea out, but how? Can I change this limit for
the entire operating system? I saw from documentation that changing the
limit only works for the current shell and its descendents, whatever
that means.

Changing the limit for the shell and its descendents means that when
running your app from a command prompt you can first set the limit for
the shell that is giving you the command prompt and the same limit will
be given to all apps started from that command prompt.

ulimit -S -n 1000 # set limit for shell
java -jar YourApp.jar # start your app which is a descendent of this
shell

Re: change it for OS

I don't know off top for sure on Mac OX. Maybe it is an /etc/system
setting. This should not be too hard to locate on the internet. It's
a common need.

How do I see how many file descriptors a program is using?

Don't know. Best I can think of:
1) google it
or
2) Check ulimit on computer you have and see if any flags tell you how
many are left. Then you can call that routine from within java, parse
output, and get yourself an answer


Another approach could be that when limit is reached you ask for more.

All the best,
Opalinski
(e-mail address removed)
http://www.geocities.com/opalpaweb/

 

[EJP]

Are you actually accepting the connections with ServerSocket.accept()?
If not, you've just hit the listen backlog limit.

 

[Joshua Jung]

I figured it out. Apparently two things were happening:

1. I was running out of memory, and had to up the stack heap size when
running my server from the command prompt (using -Xmx command)

2. I had to up the file descriptor limit to some huge number in the
thousands.

Then everything worked fine and I got like 1600 connections, all with
independent threads to work just fine!

Josh <><

On Mon, 02