thread limits

[zero]

As a hypothetical question (not something I am working on, just something I
was thinking about last night), suppose you're implementing a simple
threaded server, for example a chat or game server. The server creates a
new thread for every user/client that connects to it. This thread then
handles all the incoming messages, and sends them to all other clients (for
example using a multicast address).

Now my question is, how many users - and hence how many threads - would you
allow at the same time? Do you select a limit based on the platform (eg
available memory, bandwidth), or are there other things to consider? Or,
would you not impose a limit at all, trusting that the system has enough
resources for your number of users (sounds like a bad plan IMO).

Looking forward to your replies,

zero

 

[iamfractal]

zero skrev:

The server creates a
new thread for every user/client that connects to it.
Looking forward to your replies,

zero

(Presuming you have just one processor, or are at least talking about
threads-per-processor.)

Threads are not fundamentally concerned with muti-user-ness. That
threads can help to serve multiple users is a derivative. Threads are
fundamentally a blocking-management mechanism.

So ignore how many users you have for a start, and think about which
processes will block, for example:

A) Receiving a request (and perhaps putting it into a queue).
B) On-chip processing of the request.
C) Off-chip processing of the request (e.g., database access, perhaps).
D) Sending response.

Straight off the top: you should only have four threads, and I the
limit will probably be the size of the input-queue, and/or a reasonable
respons-time.


..ed

 

[Ben_]

A pattern "1 user = 1 thread" does not scale very well.

One is that each thread has a heap space reserved, and an application can
run out of memory when there are too many threads.

See for example a discussion I had here in the past:
http://groups.google.com/group/comp.lang.java.machine/browse_thread/thread/bd8c8229ce19ffd7

You can also Google for terms like too many threads, out of threads,
threads scalability, etc.

You'll see that a thousand thread is considered very high already.

That's why J2EE Containers for example implement pool of threads.

Basically, the idea is that a user does not act so rapidly that a thread
needs to be reserved all the time to him. So, you can reuse the thread to
work for another user.

HTH.

 

[Daniel Dyer]

As a hypothetical question (not something I am working on, just
something I
was thinking about last night), suppose you're implementing a simple
threaded server, for example a chat or game server. The server creates a
new thread for every user/client that connects to it. This thread then
handles all the incoming messages, and sends them to all other clients
(for
example using a multicast address).

Now my question is, how many users - and hence how many threads - would
you
allow at the same time? Do you select a limit based on the platform (eg
available memory, bandwidth), or are there other things to consider? Or,
would you not impose a limit at all, trusting that the system has enough
resources for your number of users (sounds like a bad plan IMO).

Looking forward to your replies,

This is quite a good article about implementing multi-threaded servers.
The java.util.concurrent package in Java 5 originates from the the author
and concepts of this article.

http://www.awprofessional.com/articles/article.asp?p=167821&rl=1

The NIO packages introduced in Java 1.4 are also worth looking at,
particularly the concepts of channels and selectors. Combined with the
ThreadPoolExecutor in java.util.concurrent, this will allow you to write a
scalable server using a minimal number of threads.

Given enough RAM (for all the stacks), a clever enough JVM and the right
OS, you can have a huge number of Java threads (as many as 10,000), even
on a single-processor server. This does not necessarily correspond to a
similar number of native threads as the JVM may map several Java threads
onto fewer native threads. Whatever approach you take, you are mapping
multiple concurrent tasks onto a few (probably only one) physical
processors. This can be done at the OS level (JVM just maps Java threads
one-to-one with native threads and leaves the OS to do the context
switching), the JVM level (such as the JRockit thin threads
implementation) or the application level (you write Java code to share a
few Java threads between many clients).

You'll probably have to experiment a bit since it depends on the patterns
of behaviour of the clients (how much data they are sending/receiving and
how often, what kind of latency you require etc.).

Dan.

 

[Daniel Dyer]

Given enough RAM (for all the stacks), a clever enough JVM and the right
OS, you can have a huge number of Java threads (as many as 10,000), even
on a single-processor server. This does not necessarily correspond to a
similar number of native threads as the JVM may map several Java threads
onto fewer native threads.

Of course, I should add that just because you can do it, doesn't mean you
should. This approach (thousands of threads per processor) only really
works when the clients are reasonably idle for most of the time and is
heavily dependent on the cost of context-switching with your OS/JVM
combination.

Dan.

 

[Walter Mitty]

A pattern "1 user = 1 thread" does not scale very well.

One is that each thread has a heap space reserved, and an application can
run out of memory when there are too many threads.

See for example a discussion I had here in the past:
http://groups.google.com/group/comp.lang.java.machine/browse_thread/thread/bd8c8229ce19ffd7

You can also Google for terms like too many threads, out of threads,
threads scalability, etc.

You'll see that a thousand thread is considered very high already.

That's why J2EE Containers for example implement pool of threads.

Basically, the idea is that a user does not act so rapidly that a thread
needs to be reserved all the time to him. So, you can reuse the thread to
work for another user.

HTH.

Ditto for handles to a database connection.

 

[Babu Kalakrishnan]

A pattern "1 user = 1 thread" does not scale very well.

One is that each thread has a heap space reserved, and an application can
run out of memory when there are too many threads.

Agree with your statement on line 1, but a correction to the first part
of the next line.

Each thread DOES NOT have a separate heap space reserved for it - it is
actually stack space that is reserved as private to each thread. The
heap is shared by all threads.

BK

 

[Ben_]

Agreed. Thanks for correcting.