Unlimited threads being created

[Hugo]

Hi:

I have a three servlet classes in a Tomcat where I use the
ExetorServices to create a thread pool. The thread pool is used to
execute tasks so that the main thread does not slow down.

Problem: Tomcat crashes with an out of memory error:
"<snip>java.lang.OutOfMemoryError: unable to create new native thread)
executing org.apache.tomcat.util.net.TcpWorkerThread</snip>"

From looking at the logs, each time a servlet was called a new thread
was created. I am guessing that the Tomcat ran until the process ran
out of threads. I thought that the maximum number of threads that can
be created by the thread pool will be listed by the
newFixedThreadPool() method.

Sample of the code is below, can anyone help me out on where I have
gone wrong?


Public Class Foo extends HTTPServlet
{

ExecutorService pool = Executors.newFixedThreadPool(1000);

public void doGet{

//web form

}

public void doPost{

pool.execute(bar);

}


}

 

[Eric Sosman]

Hi:

I have a three servlet classes in a Tomcat where I use the
ExetorServices to create a thread pool. The thread pool is used to
execute tasks so that the main thread does not slow down.

Problem: Tomcat crashes with an out of memory error:
"<snip>java.lang.OutOfMemoryError: unable to create new native thread)
executing org.apache.tomcat.util.net.TcpWorkerThread</snip>"

From looking at the logs, each time a servlet was called a new thread
was created. I am guessing that the Tomcat ran until the process ran
out of threads. I thought that the maximum number of threads that can
be created by the thread pool will be listed by the
newFixedThreadPool() method.

Sample of the code is below, can anyone help me out on where I have
gone wrong?


Public Class Foo extends HTTPServlet
{

ExecutorService pool = Executors.newFixedThreadPool(1000);

A thousand threads? That's a LOT of threads! I'm not
surprised you ran out of memory.

Also, how many hardware execution units ("CPU's," "cores,"
"strands") does your system have, and how does that number
compare to a thousand? Unless your threads are spending a
whole lot of time doing nothing (e.g., waiting for something
slow like I/O), they'll thrash the scheduler to death. Ever
seen a thousand people scramble for the same elevator?

Reduce the thread count to something sane and try again.

 

[Hugo]

Thank you for your post.

Also, how many hardware execution units ("CPU's," "cores,"
"strands") does your system have,

CPUs: 2 x 1002Mhz
Memory size: 4GB
Operating system: Solaris 9

Why didn't the Executor recycle through the threads created via:
ExecutorService pool = Executors.newFixedThreadPool(1000)?

The ExecutorService is declared in the class field and not in the
doPost() method, will this be an issue?

 

[Eric Sosman]

Thank you for your post.

CPUs: 2 x 1002Mhz
Memory size: 4GB
Operating system: Solaris 9

You don't mention the CPU type, which matters a lot,
but in any event the number of threads that can execute
simultaneously on the hardware will not be close to a
thousand. You could likely count them on your fingers;
taking your shoes off is probably unnecessary.

Why didn't the Executor recycle through the threads created via:
ExecutorService pool = Executors.newFixedThreadPool(1000)?

Why do you think it didn't? My suspicion (until proven
otherwise, and proofs are certainly possible) is that it
never got to a thousand in the first place, but ran out of
memory before it could create that many threads.

The ExecutorService is declared in the class field and not in the
doPost() method, will this be an issue?

It's in the Foo constructor (well, sort of: what you
showed is obviously a paraphrase -- it wouldn't compile,
for starters -- so I can't be sure just where the pool
is created). But let's suppose it's in Foo's constructor,
and that Foo is an HTTPServlet. That means you'll get a
new pool each time an HTTPServlet/Foo instance is created.
I don't know how frequently Tomcat does so -- I seem to
recall that it ordinarily creates a given servlet type
only once, but that there may be ways of unloading and
reloading them, or ways of starting multiple instances
under different aliases. So you'll have at least one pool
with a "ceiling" of a thousand threads, possibly N such
pools.

I say again: Reduce the thread count, maybe to ten or
twenty-ish. What benefit do you expect from a thousand
threads all trying to elbow each other out of the way for
the attention of two processor chips?

 

[Mark Space]

ExecutorService pool = Executors.newFixedThreadPool(1000);

I completely agree with Eric, espcialy considering that your hardware is
very, very modest.

Change the 1000 to 20, see if that works better. You may get a
different error, but at least it should run.

 

[Alex.From.Ohio.Java]

You could likely count them on your fingers;
taking your shoes off is probably unnecessary.

Eric! You are Sun guy! Open JConsole and look how many threads simple
application has! I never saw less then 20. Should I mention all of
them?

Alex.
http://www.myjavaserver.com/~alexfromohio/

 

[Knute Johnson]

Eric! You are Sun guy! Open JConsole and look how many threads simple
application has! I never saw less then 20. Should I mention all of
them?

Alex.
http://www.myjavaserver.com/~alexfromohio/

On my Windows machine, more than about 100 threads will bring it to a
halt. Running Linux I can get two or three times that many but there is
no real performance advantage. You are probably better of with 10 or 20
threads in your Executor.

 

[Owen Jacobson]

Eric! You are Sun guy! Open JConsole and look how many threads simple
application has! I never saw less then 20. Should I mention all of
them?

Alex.http://www.myjavaserver.com/~alexfromohio/

Threads are _not free_. Threads that are "ready to run" are
especially not free.

Each thread has a stack, which requires memory from the process's
address space (which is largely taken up by Java's heap). Each ready
thread must be considered by the scheduler every time the scheduler
runs. Thread context switches save the state of the processor to the
stack and load the new thread's state from the new stack, effectively
resetting the CPU cache.

The ideal application has exactly one ready-to-run thread per
execution unit, not hundreds. In practice it's hard to hit this ideal
goal, so often a small integer number of threads are used per core
instead - two is a fairly popular rule of thumb.

Yes, your system is probably running close to a hundred threads
already. The vast majority of them are waiting most of the time -
either for IO, or for user input - and are therefore not causing
context switches nor being considered by the scheduler. Furthermore,
these threads are scattered across many processes, so the address
space contention threads can cause in a single process is a much
smaller concern.

One app trying to run a thousand threads simultaneously is grossly
wasteful. I'm not at all surprised that it doesn't work on your
system.

 

[Eric Sosman]

Eric! You are Sun guy! Open JConsole and look how many threads simple
application has! I never saw less then 20. Should I mention all of
them?

In the material you snipped, I made it clear that I was
referring to "hardware execution units," to "threads that
can execute simultaneously on the hardware." Some people
call these "strands" or "pipelines," but neither term seems
entirely satisfactory to me. What I was getting at was:
How many threads could possibly be in the act of executing
instructions at a given moment? This is not the same as:
How many threads could exist on the system and be eligible
to compete for the right to execute instructions?

The two notions -- "strands" and threads -- are different
and to some degree independent (the system's scheduler maintains
the illusion that all eligible threads run simultaneously, even
though there may be more threads than "strands"). However, it
is usually a sign that something is wrong if the number of
eligible threads vastly exceeds the number of "strands" that
can run them.

 

[Hugo]

Thank you for all your posts - they are very helpful.

I am running the application on a V240 machine.

I ran a load test today and logged the thread count
[Thread.activeCount()], the max count obtained was ~2900 threads
before the JVM complained about 'Out of Memory'. The load test was run
for the setting: Executors.newFixedThreadPool(1000).

I will rerun the load test was lower Executors.newFixedThreadPool(n).

Would you recommend staying away from using
Executors.newFixedThreadPool(n) in servlets?

 

[Mark Space]

Would you recommend staying away from using
Executors.newFixedThreadPool(n) in servlets?

How many servlets do you have that declare an Executor?

 

[Kenneth P. Turvey]

On Wed, 19 Mar 2008 10:52:11 -0700, Hugo wrote:

[Snip]

Would you recommend staying away from using
Executors.newFixedThreadPool(n) in servlets?

[Snip]

Using them with a reasonable number of threads is probably fine.

 

[Hugo]

How many servlets do you have that declare an Executor?

6 servlets using Executor (all from one web app).

 

[Mark Space]

6 servlets using Executor (all from one web app).

6 isn't too bad, I think, but for future scalability it might be better
to put one Executor in the servlet context and have all your servlets
use that one object.