nThreads

Discussion in 'Java' started by bob smith, Feb 13, 2013.

  1. bob smith

    bob smith Guest

    I'm looking at this method in the Executors class:

    --------------------------------
    newFixedThreadPool
    public static ExecutorService newFixedThreadPool(int nThreads)

    Creates a thread pool that reuses a fixed set of threads operating off a shared unbounded queue. If any thread terminates due to a failure during execution prior to shutdown, a new one will take its place if needed to executesubsequent tasks.

    --------------------------------

    How much is too much in terms of me telling it how many threads I want? Isit okay to ask for 100 threads? What about 256?

    Thanks.
     
    bob smith, Feb 13, 2013
    #1
    1. Advertising

  2. bob smith

    Joerg Meier Guest

    On Wed, 13 Feb 2013 15:25:33 -0800 (PST), bob smith wrote:

    > How much is too much in terms of me telling it how many threads I want? Is it okay to ask for 100 threads? What about 256?


    While modern operating systems support thousands of threads, there are
    really not many advantages in using more threads than you have CPUs/cores,
    and the additional context switching of having more threads than cores
    comes with its own performance cost.

    Liebe Gruesse,
    Joerg

    --
    Ich lese meine Emails nicht, replies to Email bleiben also leider
    ungelesen.
     
    Joerg Meier, Feb 13, 2013
    #2
    1. Advertising

  3. bob smith

    markspace Guest

    On 2/13/2013 3:25 PM, bob smith wrote:
    > How much is too much in terms of me telling it how many threads I
    > want? Is it okay to ask for 100 threads? What about 256?


    It depends on your system and your application, and you should measure
    your system performance (and other metrics) to determine what is best
    for your particular case. I'm sorry I don't know much about practical
    ways to go about those measurements.

    However, 100 to 300 threads is trivial for most modern system. Since
    most tasks will be IO-bound, it make sense to have lots of threads so
    some can be running while others wait for data to be returned. (Sorry
    to contradict Mr. Meier but I'm pretty sure he's off base.) Given a
    "typical" application I'd probably use 300 threads to start off and
    measure from there.

    3000 is probably approaching some sort of upper limit on most "average"
    servers. Don't neglect to measure actual running performance, but I
    don't think commodity hardware can handle that many practically.
     
    markspace, Feb 14, 2013
    #3
  4. bob smith

    Joerg Meier Guest

    On Wed, 13 Feb 2013 16:06:37 -0800, markspace wrote:

    > However, 100 to 300 threads is trivial for most modern system. Since
    > most tasks will be IO-bound, it make sense to have lots of threads so
    > some can be running while others wait for data to be returned. (Sorry
    > to contradict Mr. Meier but I'm pretty sure he's off base.) Given a


    Obviously, you can pretty much have Integer.MAX_VALUE threads if all of
    them are waiting without negatively impacting your waiting performance -
    that's not really a sensible way to look at that question ;)

    I said, and stand by, that context switching is not free and is the cost of
    having too many threads. Threads that sleep or otherwise wait, such as for
    IO, don't typically cause much context switching.

    That being said, I would expect the case of 100s of threads waiting for IO
    probably an edge case (for use cases such as web servers), and the majority
    of cases would have IO restricted to a small number of the threads. YMMV.

    > 3000 is probably approaching some sort of upper limit on most "average"
    > servers. Don't neglect to measure actual running performance, but I
    > don't think commodity hardware can handle that many practically.


    We once ran (by accident) something that triggered 100,000 threads (+ GC
    and whatever) on a normal Linux server with a quad core CPU and it still
    worked (although much slower than with the intended 100). As far as I know,
    neither Linux nor the hardware was anything unusual.

    Liebe Gruesse,
    Joerg

    --
    Ich lese meine Emails nicht, replies to Email bleiben also leider
    ungelesen.
     
    Joerg Meier, Feb 14, 2013
    #4
  5. bob smith

    Lew Guest

    Joerg Meier wrote:
    > markspace wrote:
    >> However, 100 to 300 threads is trivial for most modern system. Since
    >> most tasks will be IO-bound, it make sense to have lots of threads so
    >> some can be running while others wait for data to be returned. (Sorry
    >> to contradict Mr. Meier but I'm pretty sure he's off base.) Given a

    >
    > Obviously, you can pretty much have Integer.MAX_VALUE threads if all of
    > them are waiting without negatively impacting your waiting performance -
    > that's not really a sensible way to look at that question ;)


    Perhaps not, but see below.

    > I said, and stand by, that context switching is not free and is the cost of
    > having too many threads. Threads that sleep or otherwise wait, such as for
    > IO, don't typically cause much context switching.


    But how "not free" is it? Let's look at your experience in a minute.

    > That being said, I would expect the case of 100s of threads waiting for IO
    > probably an edge case (for use cases such as web servers), and the majority
    > of cases would have IO restricted to a small number of the threads. YMMV.
    >
    >> 3000 is probably approaching some sort of upper limit on most "average"
    >> servers. Don't neglect to measure actual running performance, but I
    >> don't think commodity hardware can handle that many practically.

    >
    > We once ran (by accident) something that triggered 100,000 threads (+ GC
    > and whatever) on a normal Linux server with a quad core CPU and it still
    > worked (although much slower than with the intended 100). As far as I know,
    > neither Linux nor the hardware was anything unusual.


    The question is how much slower, and whether 1,000 threads would have experienced
    a severe slowdown for that setup. Or 10,000.

    Quite often resource limits show up as a discontinuity in the scaling graph. Service time
    degrades gracefully up to some n, then catastrophically above that.

    In your case there might have been such a knee in that 10^3- 10^4 range. The numbers
    would tell.

    And that's the point for the OP. You cannot know ahead of time what the right number is,
    really. You need to measure, under realistic loads. (And be pretty good at simulating "realistic".)

    Notice that answers here are in terms of "if you're I/O bound", "if you're waiting", "if this", "if that".

    You need to know which ifs apply to your case.

    --
    Lew
     
    Lew, Feb 14, 2013
    #5
  6. bob smith

    Roedy Green Guest

    On Wed, 13 Feb 2013 15:25:33 -0800 (PST), bob smith
    <> wrote, quoted or indirectly quoted someone
    who said :

    >How much is too much in terms of me telling it how many threads I want? Is=
    > it okay to ask for 100 threads? What about 256?


    The last time I looked, each thread ate up 1MB.

    I think the best thing to do is run some benchmarks. It will depend
    on how much RAM you have, how many CPU's how much time each thread
    spends sleeping. It also depends on how busy the machine is with
    other work.

    See http://mindprod.com/project/tweakable.html for a proposal to solve
    this class of problem.
    --
    Roedy Green Canadian Mind Products http://mindprod.com
    The first 90% of the code accounts for the first 90% of the development time.
    The remaining 10% of the code accounts for the other 90% of the development
    time.
    ~ Tom Cargill Ninety-ninety Law
     
    Roedy Green, Feb 14, 2013
    #6
  7. On 02/13/2013 07:25 PM, bob smith wrote:
    > I'm looking at this method in the Executors class:
    >
    > --------------------------------
    > newFixedThreadPool
    > public static ExecutorService newFixedThreadPool(int nThreads)
    >
    > Creates a thread pool that reuses a fixed set of threads operating off a shared unbounded queue. If any thread terminates due to a failure during execution prior to shutdown, a new one will take its place if needed to execute subsequent tasks.
    >
    > --------------------------------
    >
    > How much is too much in terms of me telling it how many threads I want? Is it okay to ask for 100 threads? What about 256?
    >
    > Thanks.
    >

    Just a cautionary, and that's not saying you're ignoring this aspect:
    the question "how many" in terms of units of concurrent execution is a
    question that should be answered by looking at logic requirements first,
    API and platform performance capabilities second.

    For example, given N units of work, described by tasks (and ultimately
    executed by threads under the control of some Executor, say), the
    business logic for one particular use case might require you to process
    these tasks 1 by 1, in some order, in another use case the business
    logic might dictate that you have to process in groups of M < N, and in
    yet another use case the business logic places no _logical_ bound on how
    many tasks can be executed at the same time.

    Also, consider whether you have a concurrent problem or a parallel
    problem. If your application is doing a bunch of things, among them
    handling continual requests (user or external system inputs), that's
    really concurrency. But if you've got one problem that benefits from
    being split up, that's parallelism and you may want to look at ForkJoin
    instead.

    Task design also is driven by business logic and comes before choosing
    things like thread pools and sizes thereof. Task granularity in a
    concurrent problem should be whatever logic dictates. An important
    consideration here too is synchronous versus asynchronous, and how you
    plan to get task results (if any). Runnables vs. Callables, Futures, etc.

    Addressing number of threads for pools, a bit, a lot of small tasks can
    be worse than a moderate number of medium-sized ones. As others have
    said, only performance testing will tell. But don't artificially chop up
    your logic in a _concurrent_ problem thinking that you can process a
    passel of these things with umpteen threads, because that may backfire.

    Overall point being, about the last thing I'd be worrying about is how
    many threads to assign to a fixed size thread pool. *Eventually* I'd
    think about it, sure. After lots of testing.

    AHS
     
    Arved Sandstrom, Feb 14, 2013
    #7
  8. bob smith

    Roedy Green Guest

    On Thu, 14 Feb 2013 00:46:50 +0100, Joerg Meier <>
    wrote, quoted or indirectly quoted someone who said :

    >While modern operating systems support thousands of threads, there are
    >really not many advantages in using more threads than you have CPUs/cores,


    It pays to have more threads than cores if your threads are probing
    websites. You want to have enough waiting to keep your port busy. You
    can also do that in more efficient but complicated way with a single
    thread that processes the next response in turn.

    Your advice would apply for CPU intensive tasks.
    --
    Roedy Green Canadian Mind Products http://mindprod.com
    The first 90% of the code accounts for the first 90% of the development time.
    The remaining 10% of the code accounts for the other 90% of the development
    time.
    ~ Tom Cargill Ninety-ninety Law
     
    Roedy Green, Feb 14, 2013
    #8
  9. bob smith

    Daniel Pitts Guest

    On 2/13/13 3:25 PM, bob smith wrote:
    > I'm looking at this method in the Executors class:
    >
    > --------------------------------
    > newFixedThreadPool
    > public static ExecutorService newFixedThreadPool(int nThreads)
    >
    > Creates a thread pool that reuses a fixed set of threads operating off a shared unbounded queue. If any thread terminates due to a failure during execution prior to shutdown, a new one will take its place if needed to execute subsequent tasks.
    >
    > --------------------------------
    >
    > How much is too much in terms of me telling it how many threads I want? Is it okay to ask for 100 threads? What about 256?
    >
    > Thanks.
    >
    >

    As you have read in the rest of this thread, the answer is "it depends".

    You can ask for many many threads (thousands). The real question is "how
    many threads *should* I for task X?"

    Which, of course, depends on the task.

    A rule of thumb I use for CPU bound tasks (ray-tracing for example) is
    N+1 where N is the number of CPU cores. The +1 is to have an extra
    thread available to pick up slack if any one of the other threads ends
    up blocked by something, such as synchronization, paging, etc...

    For IO bound tasks, I tend not to use a fixed pool, but use a pool that
    automatically adjusts its size to meet demand. You also need to be aware
    of the effect that you might have down-stream. If your IO is all local
    disk access on a spinning media (eg, non SDD hard disk), too many
    threads *might* cause undue head shifting, slowing down the overall
    operation.

    For Memory (or other allocated resource) Intensive tasks, you want to
    have an upper bound of threads that prevents any thread from failing due
    to running out of resources.

    For mixed tasks (some tasks which block on IO for a portion of time,
    then process the result in a CPU intensive manor), you'll have to make a
    guess and then fine-tune to each machine. Provide sensible defaults but
    let the end-user configure it.

    As a matter of fact, many applications have configuration options for
    thread-pool size. Let the end-user (or administrator at least)
    decide/figure out what is best for their particular use-cases,
    workflows, and systems.

    HTH,
    Daniel.
     
    Daniel Pitts, Feb 14, 2013
    #9
  10. On 14.02.2013 01:46, Joerg Meier wrote:
    > On Wed, 13 Feb 2013 16:06:37 -0800, markspace wrote:
    >
    >> However, 100 to 300 threads is trivial for most modern system. Since
    >> most tasks will be IO-bound, it make sense to have lots of threads so
    >> some can be running while others wait for data to be returned. (Sorry
    >> to contradict Mr. Meier but I'm pretty sure he's off base.) Given a

    >
    > Obviously, you can pretty much have Integer.MAX_VALUE threads if all of
    > them are waiting without negatively impacting your waiting performance -
    > that's not really a sensible way to look at that question ;)


    Obviously.

    > I said, and stand by, that context switching is not free and is the cost of
    > having too many threads.


    Nobody questioned that.

    > Threads that sleep or otherwise wait, such as for
    > IO, don't typically cause much context switching.


    Yes, but in a pool with a fixed upper limit of threads they consume a
    slot while sitting there doing nothing (and causing no CPU cost). In
    this case a core will be idle if you limit pool size to number of cores.
    If the application is doing IO (or anything else which might cause
    intermediate blocking) you are unnecessarily restricting throughput of
    the application if you take number of cores (or threads as they are
    called in some architectures) as limit. What the best limit is
    obviously depends on the nature of the application.

    > We once ran (by accident) something that triggered 100,000 threads (+ GC
    > and whatever) on a normal Linux server with a quad core CPU and it still
    > worked (although much slower than with the intended 100). As far as I know,
    > neither Linux nor the hardware was anything unusual.


    There you go.

    Cheers

    robert

    --
    remember.guy do |as, often| as.you_can - without end
    http://blog.rubybestpractices.com/
     
    Robert Klemme, Feb 14, 2013
    #10
  11. bob smith

    Arne Vajhoej Guest

    On 2/13/2013 6:25 PM, bob smith wrote:
    > I'm looking at this method in the Executors class:
    >
    > --------------------------------
    > newFixedThreadPool
    > public static ExecutorService newFixedThreadPool(int nThreads)
    >
    > Creates a thread pool that reuses a fixed set of threads operating off a shared unbounded queue. If any thread terminates due to a failure during execution prior to shutdown, a new one will take its place if needed to execute subsequent tasks.
    >
    > --------------------------------
    >
    > How much is too much in terms of me telling it how many threads I want? Is it okay to ask for 100 threads? What about 256?


    It depends on the problem domain.

    With CPU intensive tasks you do not want many threads. Somewhere
    1-2 x number of thread contexts in CPU (for x86-64 that is number of
    cores without HT and twice number of cores with HT).

    With tasks that mostly wait, then you may want a lot of threads. But at
    some point the overhead of having many threads becomes significant. I
    would try and keep number of threads <500 on smaller systems.

    Arne
     
    Arne Vajhoej, Feb 15, 2013
    #11
  12. bob smith

    Arne Vajhoej Guest

    On 2/13/2013 6:46 PM, Joerg Meier wrote:
    > On Wed, 13 Feb 2013 15:25:33 -0800 (PST), bob smith wrote:
    >
    >> How much is too much in terms of me telling it how many threads I want? Is it okay to ask for 100 threads? What about 256?

    >
    > While modern operating systems support thousands of threads, there are
    > really not many advantages in using more threads than you have CPUs/cores,
    > and the additional context switching of having more threads than cores
    > comes with its own performance cost.


    That is only for CPU intensive tasks.

    If tasks are waiting then it is a very different story.

    Arne
     
    Arne Vajhoej, Feb 15, 2013
    #12
  13. bob smith

    Arne Vajhoej Guest

    On 2/13/2013 7:46 PM, Joerg Meier wrote:
    > On Wed, 13 Feb 2013 16:06:37 -0800, markspace wrote:
    >> However, 100 to 300 threads is trivial for most modern system. Since
    >> most tasks will be IO-bound, it make sense to have lots of threads so
    >> some can be running while others wait for data to be returned. (Sorry
    >> to contradict Mr. Meier but I'm pretty sure he's off base.) Given a

    >
    > Obviously, you can pretty much have Integer.MAX_VALUE threads if all of
    > them are waiting without negatively impacting your waiting performance -
    > that's not really a sensible way to look at that question ;)
    >
    > I said, and stand by, that context switching is not free and is the cost of
    > having too many threads. Threads that sleep or otherwise wait, such as for
    > IO, don't typically cause much context switching.
    >
    > That being said, I would expect the case of 100s of threads waiting for IO
    > probably an edge case (for use cases such as web servers), and the majority
    > of cases would have IO restricted to a small number of the threads. YMMV.


    I would expect >95% of Java developers to work in Java EE space and
    <5% of Java developers to work outside Java EE space.

    In Java EE space you do have threads that are waiting and thread pools
    to manage threads (both for web container and EJB container).

    So I would reverse what is edge and what is normal.

    Arne
     
    Arne Vajhoej, Feb 15, 2013
    #13
  14. bob smith

    Arne Vajhoej Guest

    On 2/13/2013 9:37 PM, Roedy Green wrote:
    > On Wed, 13 Feb 2013 15:25:33 -0800 (PST), bob smith
    > <> wrote, quoted or indirectly quoted someone
    > who said :
    >
    >> How much is too much in terms of me telling it how many threads I want? Is=
    >> it okay to ask for 100 threads? What about 256?

    >
    > The last time I looked, each thread ate up 1MB.


    That is typical configurable.

    And the default depends on OS, Java vendor, versions etc..

    Arne
     
    Arne Vajhoej, Feb 15, 2013
    #14
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.

Share This Page