Threading costs

[Thomas Hawtin]

Has anyone measured how expensive creating threads really is in Java?

It is highly dependent upon JVM. IIRC, BEA JRockit takes very little
time to create and start a Thread (presumably they've made Thread a
facade over a thread-pool).

And how much memory does an Executor's thread pool consume?

About as much as a Thread multiplied by the number of threads.

I know that forking a new thread (basically a runnable) takes less
than a millisecond. I haven't measured how much resources does a
thread pool consume per thread, or how big can thread pool become
before thread switching starts taking too much time.

For computationally heavy threads, you generally want to avoid switches.
So you want approximately as many threads as you have hardware-threads
(Runtime.availableProcessors since 1.4). It's not just the actual switch
that costs, your caches will hurt immediately afterwards. Also any
thread-local pools will be duplicated, and you can have problems with
lock contention.

You can benchmark how scalable your application is by running it with
increasing numbers of threads. If it doesn't scale very well, you can
then look into it further. Or if it does scale well, you can do
something more productive instead.

Tom Hawtin

 

[Domagoj Klepac]

Has anyone measured how expensive creating threads really is in Java?
And how much memory does an Executor's thread pool consume?

I know that forking a new thread (basically a runnable) takes less
than a millisecond. I haven't measured how much resources does a
thread pool consume per thread, or how big can thread pool become
before thread switching starts taking too much time.

Has anyone benchmarked those things, so that I don't have to?

My typical worker thread completes its task in under 50-250 ms. So a
one-millisecond to fork a new thread is not a big deal. But I haven't
measured thread pool memory consumption, and it's pretty hard to
benchmark a time it takes to switch between two threads.

Domchi

 

[David Gourley]

Has anyone measured how expensive creating threads really is in Java?
And how much memory does an Executor's thread pool consume?

I know that forking a new thread (basically a runnable) takes less
than a millisecond. I haven't measured how much resources does a
thread pool consume per thread, or how big can thread pool become
before thread switching starts taking too much time.

There may be a number of OS-specific issues associated with the cost of
creating a thread. On some OSs, a thread may have an associated kernel
entry; so when you create a thread, it causes contention for a kernel
level lock. There's also going to be some cost associated with
destroying a thread... (both within the JVM and at the OS level).

If you're running on a single processor system, you won't see the impact
of this; however if your code is part of a much larger system with many
parts of the system creating threads, this lock can become a bottleneck.
Similarly, there may be locks associated with creation of the stack
for the thread.

Another element associated with the number of threads created is that as
threads are the typical unit of scheduling on modern OSs, the threading
policy you take within an individual JVM can actually effectively
dictate the priority of a JVM... potentially causing starvation of any
single threaded processes running on your system (because if you've a
lot of Java threads, it may be a long time before a single threaded
process gets scheduled).

Dave

 

[Chris Uppal]

My typical worker thread completes its task in under 50-250 ms.

If the tasks complete in that time, why do you have to fork a thread at all ?
Perhaps sometimes they take much longer and you can't predict in advance when
that will happen ?

Anyway, as others have said, the time and resources consumed per thread
startup/death are very system-dependent. Personally, if I wanted such short
lived threads, then I would have used a thread-pool without even thinking about
it (easier to write a thread-pool implementation than to analyse/measure the
overheads of not using one).

-- chris

 

[blmblm]

[ snip ]

For computationally heavy threads, you generally want to avoid switches.
So you want approximately as many threads as you have hardware-threads
(Runtime.availableProcessors since 1.4).

One might want to use caution here -- experiment suggests that each
hyperthreaded processor counts as two in the count returned by
availableProcessors, but for computationally intensive threads all
doing roughly the same work, hyperthreading may be of little or no
benefit, so you might want just one thread for each actual processor.

FWIW, maybe YMMV.

[ snip ]

 

[Domagoj Klepac]

If the tasks complete in that time, why do you have to fork a thread at all ?
Perhaps sometimes they take much longer and you can't predict in advance when
that will happen ?

In my concrete example, a worker task consists of several checks and
modifications which involve several database lookups. The database
runs on the other computer, so in that 250ms timeframe, most of the
time is spent waiting for the database. Meanwhile, it's critical that
requests are served as fast as possible, and requests are
asynchronous.

So if a system gets two requests in the same time, not using threads
runs second task two times slower than the first, while threading runs
them simultaneously without any delays.

Anyway, as others have said, the time and resources consumed per thread
startup/death are very system-dependent. Personally, if I wanted such short
lived threads, then I would have used a thread-pool without even thinking about
it (easier to write a thread-pool implementation than to analyse/measure the
overheads of not using one).

It really depends on the application you're writing. But if you're
using thread pool, it means that you're maybe allocating more memory.
If you're forking threads, you're producing more garbage (and leaving
that to garbage collector, which in turn burdens processor), but
you're mostly using less memory.

But does it take more (memory and/or processor) to have a pool of 50
threads, or does it take more (processor) to fork threads and gc them?

I have a hunch that today's JVMs and OS-es are pretty good with
forking threads, and that forking a thread is not time-consuming
operation it once was. But the expenses of garbage collection of many
forked threads might justify the use of thread pools. So it's
basically memory (forking wins) vs. processor (pool wins).

But I'm just guessing - and I would have already benchmarked that if
it was simple.

Thanks everyone for answers - though I'm still looking for some
numbers. How much time does it take to fork a thread, and how much
time does it take to retrieve a thread from a pool? How big are your
thread pools? Do you worry about memory limits when allocating thread
pools?

Domchi

 

[Chris Uppal]

Domagoj Klepac wrote:

[me:]

In my concrete example, a worker task consists of several checks and
modifications which involve several database lookups. The database
runs on the other computer, so in that 250ms timeframe, most of the
time is spent waiting for the database. Meanwhile, it's critical that
requests are served as fast as possible, and requests are
asynchronous.

That makes sense -- assuming that the DB can service, say, 10 tasks
simultaneously faster than the same 10 sequentially.

It really depends on the application you're writing. But if you're
using thread pool, it means that you're maybe allocating more memory.
If you're forking threads, you're producing more garbage (and leaving
that to garbage collector, which in turn burdens processor), but
you're mostly using less memory.

Agreed, but I suspect that you are thinking of a thread pool where threads hang
around for long periods if unused. I'd use a timer to retire threads that had
been idle for <some small period of time>. At a first, completely arbitrary,
number to plug into the design, I would probably choose 5 seconds.

But does it take more (memory and/or processor) to have a pool of 50
threads, or does it take more (processor) to fork threads and gc them?

How many DB queries are typically in-flight at any one time, and how much does
that figure vary with time ? You don't have to make your pool big enough that
any request can be satisfied from the pool, since there is nothing to stop you
creating a new thread /if/ there isn't one already available. Given the figure
of 50 that you mention, and given the short time taken for each request, it
seems that you anticipate large numbers of simultaneous DB queries, and that
they must be fired off rather often. That being the case, an even shorter
time-out might make the pool tune itself more effectively.

But I'm just guessing - and I would have already benchmarked that if
it was simple.

Agreed ;-)

What I would do is write a pool implementation (including the timeout), and add
instrumentation to it to monitor how many idle threads existed at any one time.
Then run it under a production load[*] and see directly what the costs were.

([*] Assuming that's possible -- and if you can't at least simulate a rough
equivalent to a production load then you're reduced to guessing anyway...)

-- chris

 

[Eric Sosman]

Chris Uppal wrote On 04/20/06 08:24,:

Agreed, but I suspect that you are thinking of a thread pool where threads hang
around for long periods if unused. I'd use a timer to retire threads that had
been idle for <some small period of time>. At a first, completely arbitrary,
number to plug into the design, I would probably choose 5 seconds.

If a Thread is sitting idle, it doesn't have much of an
effect on the operation of the program. Yes, it'll tie up
some memory -- but reclaiming the memory is only useful if
you have reason to believe some other Thread will need it.
Since the first Thread is sitting idle, chances are that the
overall load is fairly light at the moment and the memory
demand is probably also at a low ebb ...

So I'd vote for letting the idle Thread just sit there:
you needed it once, which suggests you might need it again
when the load ramps back up, so why not have it handy and
ready to roll?

 

[Chris Uppal]

Eric Sosman wrote:

[me:]

So I'd vote for letting the idle Thread just sit there:
you needed it once, which suggests you might need it again
when the load ramps back up, so why not have it handy and
ready to roll?

I suspect that we are both approaching this from the same direction: "hack
something simple together and see if it works; only get complicated if the
simple version is inadequate", but that we differ in our ideas of what is
"simple" when applied to a thread pool.

You position appears to be something like: don't worry about memory (or address
space) consumption unless there's a known problem. Mine is more like, just
make the damn thing kill themselves, and you won't even need to /think/ about
whether there's a resource problem.

I'd rather write code than think ;-)

(BTW, on a more practical note, 5 seconds is an enormously long period by
computer standards, so if the thread has been idle that long, I'd consider that
it was /not/ likely to be needed again -- not on a timescale comparable with
the thread-creation overhead anyway.)

-- chris

 

[Domagoj Klepac]

That makes sense -- assuming that the DB can service, say, 10 tasks
simultaneously faster than the same 10 sequentially.

Yes, but I'm not worried about the DB. There is a number of ways to
solve DB bottlenecks, including clustering, but that doesn't have much
to do with my application.

But I'm just guessing - and I would have already benchmarked that if
it was simple.

Agreed ;-)

What I would do is write a pool implementation (including the timeout), and add
instrumentation to it to monitor how many idle threads existed at any one time.
Then run it under a production load[*] and see directly what the costs were.

Yep, it seems that's what I'll have to do.

Domchi