B
bob smith
I'm looking at this method in the Executors class:
J
Joerg Meier
While modern operating systems support thousands of threads, there are
really not many advantages in using more threads than you have CPUs/cores,
and the additional context switching of having more threads than cores
comes with its own performance cost.
Liebe Gruesse,
Joerg
M
markspace
It depends on your system and your application, and you should measure
your system performance (and other metrics) to determine what is best
for your particular case. I'm sorry I don't know much about practical
ways to go about those measurements.
However, 100 to 300 threads is trivial for most modern system. Since
most tasks will be IO-bound, it make sense to have lots of threads so
some can be running while others wait for data to be returned. (Sorry
to contradict Mr. Meier but I'm pretty sure he's off base.) Given a
"typical" application I'd probably use 300 threads to start off and
measure from there.
3000 is probably approaching some sort of upper limit on most "average"
servers. Don't neglect to measure actual running performance, but I
don't think commodity hardware can handle that many practically.
J
Joerg Meier
Obviously, you can pretty much have Integer.MAX_VALUE threads if all of
them are waiting without negatively impacting your waiting performance -
that's not really a sensible way to look at that question
I said, and stand by, that context switching is not free and is the cost of
having too many threads. Threads that sleep or otherwise wait, such as for
IO, don't typically cause much context switching.
That being said, I would expect the case of 100s of threads waiting for IO
probably an edge case (for use cases such as web servers), and the majority
of cases would have IO restricted to a small number of the threads. YMMV.
We once ran (by accident) something that triggered 100,000 threads (+ GC
and whatever) on a normal Linux server with a quad core CPU and it still
worked (although much slower than with the intended 100). As far as I know,
neither Linux nor the hardware was anything unusual.
Liebe Gruesse,
Joerg
L
Lew
Joerg said:Obviously, you can pretty much have Integer.MAX_VALUE threads if all of
them are waiting without negatively impacting your waiting performance -
that's not really a sensible way to look at that question
Perhaps not, but see below.
But how "not free" is it? Let's look at your experience in a minute.
The question is how much slower, and whether 1,000 threads would have experienced
a severe slowdown for that setup. Or 10,000.
Quite often resource limits show up as a discontinuity in the scaling graph. Service time
degrades gracefully up to some n, then catastrophically above that.
In your case there might have been such a knee in that 10^3- 10^4 range. The numbers
would tell.
And that's the point for the OP. You cannot know ahead of time what the right number is,
really. You need to measure, under realistic loads. (And be pretty good at simulating "realistic".)
Notice that answers here are in terms of "if you're I/O bound", "if you're waiting", "if this", "if that".
You need to know which ifs apply to your case.
R
Roedy Green
The last time I looked, each thread ate up 1MB.
I think the best thing to do is run some benchmarks. It will depend
on how much RAM you have, how many CPU's how much time each thread
spends sleeping. It also depends on how busy the machine is with
other work.
See http://mindprod.com/project/tweakable.html for a proposal to solve
this class of problem.
--
Roedy Green Canadian Mind Products http://mindprod.com
The first 90% of the code accounts for the first 90% of the development time.
The remaining 10% of the code accounts for the other 90% of the development
time.
~ Tom Cargill Ninety-ninety Law
A
Arved Sandstrom
Just a cautionary, and that's not saying you're ignoring this aspect:
the question "how many" in terms of units of concurrent execution is a
question that should be answered by looking at logic requirements first,
API and platform performance capabilities second.
For example, given N units of work, described by tasks (and ultimately
executed by threads under the control of some Executor, say), the
business logic for one particular use case might require you to process
these tasks 1 by 1, in some order, in another use case the business
logic might dictate that you have to process in groups of M < N, and in
yet another use case the business logic places no _logical_ bound on how
many tasks can be executed at the same time.
Also, consider whether you have a concurrent problem or a parallel
problem. If your application is doing a bunch of things, among them
handling continual requests (user or external system inputs), that's
really concurrency. But if you've got one problem that benefits from
being split up, that's parallelism and you may want to look at ForkJoin
instead.
Task design also is driven by business logic and comes before choosing
things like thread pools and sizes thereof. Task granularity in a
concurrent problem should be whatever logic dictates. An important
consideration here too is synchronous versus asynchronous, and how you
plan to get task results (if any). Runnables vs. Callables, Futures, etc.
Addressing number of threads for pools, a bit, a lot of small tasks can
be worse than a moderate number of medium-sized ones. As others have
said, only performance testing will tell. But don't artificially chop up
your logic in a _concurrent_ problem thinking that you can process a
passel of these things with umpteen threads, because that may backfire.
Overall point being, about the last thing I'd be worrying about is how
many threads to assign to a fixed size thread pool. *Eventually* I'd
think about it, sure. After lots of testing.
AHS
R
Roedy Green
It pays to have more threads than cores if your threads are probing
websites. You want to have enough waiting to keep your port busy. You
can also do that in more efficient but complicated way with a single
thread that processes the next response in turn.
Your advice would apply for CPU intensive tasks.
--
Roedy Green Canadian Mind Products http://mindprod.com
The first 90% of the code accounts for the first 90% of the development time.
The remaining 10% of the code accounts for the other 90% of the development
time.
~ Tom Cargill Ninety-ninety Law
D
Daniel Pitts
As you have read in the rest of this thread, the answer is "it depends".
You can ask for many many threads (thousands). The real question is "how
many threads *should* I for task X?"
Which, of course, depends on the task.
A rule of thumb I use for CPU bound tasks (ray-tracing for example) is
N+1 where N is the number of CPU cores. The +1 is to have an extra
thread available to pick up slack if any one of the other threads ends
up blocked by something, such as synchronization, paging, etc...
For IO bound tasks, I tend not to use a fixed pool, but use a pool that
automatically adjusts its size to meet demand. You also need to be aware
of the effect that you might have down-stream. If your IO is all local
disk access on a spinning media (eg, non SDD hard disk), too many
threads *might* cause undue head shifting, slowing down the overall
operation.
For Memory (or other allocated resource) Intensive tasks, you want to
have an upper bound of threads that prevents any thread from failing due
to running out of resources.
For mixed tasks (some tasks which block on IO for a portion of time,
then process the result in a CPU intensive manor), you'll have to make a
guess and then fine-tune to each machine. Provide sensible defaults but
let the end-user configure it.
As a matter of fact, many applications have configuration options for
thread-pool size. Let the end-user (or administrator at least)
decide/figure out what is best for their particular use-cases,
workflows, and systems.
HTH,
Daniel.
R
Robert Klemme
Obviously, you can pretty much have Integer.MAX_VALUE threads if all of
them are waiting without negatively impacting your waiting performance -
that's not really a sensible way to look at that question
Obviously.
I said, and stand by, that context switching is not free and is the cost of
having too many threads.
Nobody questioned that.
Yes, but in a pool with a fixed upper limit of threads they consume a
slot while sitting there doing nothing (and causing no CPU cost). In
this case a core will be idle if you limit pool size to number of cores.
If the application is doing IO (or anything else which might cause
intermediate blocking) you are unnecessarily restricting throughput of
the application if you take number of cores (or threads as they are
called in some architectures) as limit. What the best limit is
obviously depends on the nature of the application.
There you go.
Cheers
robert
A
Arne Vajhoej
It depends on the problem domain.
With CPU intensive tasks you do not want many threads. Somewhere
1-2 x number of thread contexts in CPU (for x86-64 that is number of
cores without HT and twice number of cores with HT).
With tasks that mostly wait, then you may want a lot of threads. But at
some point the overhead of having many threads becomes significant. I
would try and keep number of threads <500 on smaller systems.
Arne
A
Arne Vajhoej
That is only for CPU intensive tasks.
If tasks are waiting then it is a very different story.
Arne
A
Arne Vajhoej
Obviously, you can pretty much have Integer.MAX_VALUE threads if all of
them are waiting without negatively impacting your waiting performance -
that's not really a sensible way to look at that question
I said, and stand by, that context switching is not free and is the cost of
having too many threads. Threads that sleep or otherwise wait, such as for
IO, don't typically cause much context switching.
That being said, I would expect the case of 100s of threads waiting for IO
probably an edge case (for use cases such as web servers), and the majority
of cases would have IO restricted to a small number of the threads. YMMV.
I would expect >95% of Java developers to work in Java EE space and
<5% of Java developers to work outside Java EE space.
In Java EE space you do have threads that are waiting and thread pools
to manage threads (both for web container and EJB container).
So I would reverse what is edge and what is normal.
Arne
A
Arne Vajhoej
That is typical configurable.
And the default depends on OS, Java vendor, versions etc..
Arne