Limit number of concurrent running threads in pool

Discussion in 'Ruby' started by Joe Martin, Mar 4, 2010.

  1. Joe Martin

    Joe Martin Guest

    Hi
    I created a pool of threads (say, 500 threads) to process. However, due
    to the weight of each thread, I want to limit the number of threads that
    run concurrently.

    So how would I go about putting a limit on the number of threads that
    run at any given time? I would like to take, say, 5 threads from the
    pool and run them, and as each one completes, it is removed from the
    pool and is replaced with a new thread from the pool.

    Could this be done with a "spy" thread, in that it constantly loops to
    check how many threads are running at once, and if the number of running
    threads falls below the limit of 5, it takes the next thread out of the
    pool and runs it? Not sure how I would go about doing this, pretty new
    to multithreading.

    Thanks!
    --
    Posted via http://www.ruby-forum.com/.
     
    Joe Martin, Mar 4, 2010
    #1
    1. Advertising

  2. Joe Martin

    Roger Pack Guest

    > I created a pool of threads (say, 500 threads) to process. However, due
    > to the weight of each thread, I want to limit the number of threads that
    > run concurrently.
    >
    > So how would I go about putting a limit on the number of threads that
    > run at any given time? I would like to take, say, 5 threads from the
    > pool and run them, and as each one completes, it is removed from the
    > pool and is replaced with a new thread from the pool.


    http://github.com/spox/actionpool

    might help.
    -r
    --
    Posted via http://www.ruby-forum.com/.
     
    Roger Pack, Mar 4, 2010
    #2
    1. Advertising

  3. Joe Martin

    Joe Martin Guest

    Roger Pack wrote:
    >> I created a pool of threads (say, 500 threads) to process. However, due
    >> to the weight of each thread, I want to limit the number of threads that
    >> run concurrently.
    >>
    >> So how would I go about putting a limit on the number of threads that
    >> run at any given time? I would like to take, say, 5 threads from the
    >> pool and run them, and as each one completes, it is removed from the
    >> pool and is replaced with a new thread from the pool.

    >
    > http://github.com/spox/actionpool
    >
    > might help.
    > -r


    Thank you very much, Roger. I found this link
    (http://snippets.dzone.com/posts/show/3276) which it looks like you had
    a part in as well and just got that code working shortly after posting
    this thread. But looking into ActionPool, it definitely offers expanded
    functionality so I will probably implement that solution instead.

    Cheers!
    --
    Posted via http://www.ruby-forum.com/.
     
    Joe Martin, Mar 4, 2010
    #3
  4. 2010/3/4 Joe Martin <>:
    > I created a pool of threads (say, 500 threads) to process. =A0However, du=

    e
    > to the weight of each thread, I want to limit the number of threads that
    > run concurrently.
    >
    > So how would I go about putting a limit on the number of threads that
    > run at any given time? =A0I would like to take, say, 5 threads from the
    > pool and run them, and as each one completes, it is removed from the
    > pool and is replaced with a new thread from the pool.
    >
    > Could this be done with a "spy" thread, in that it constantly loops to
    > check how many threads are running at once, and if the number of running
    > threads falls below the limit of 5, it takes the next thread out of the
    > pool and runs it? =A0Not sure how I would go about doing this, pretty new
    > to multithreading.


    Why do you create a pool much larger than the load you want to accept?
    Usually the pool size is used to limit concurrency. Actually that is
    the main purpose of thread pools.

    If you have different tasks for which you want to have different
    limits on concurrency you could also create several pools with
    different sizes.

    Kind regards

    robert

    --=20
    remember.guy do |as, often| as.you_can - without end
    http://blog.rubybestpractices.com/
     
    Robert Klemme, Mar 5, 2010
    #4
  5. Joe Martin

    Chuck Remes Guest

    On Mar 4, 2010, at 12:49 PM, Joe Martin wrote:

    > Hi
    > I created a pool of threads (say, 500 threads) to process. However, due
    > to the weight of each thread, I want to limit the number of threads that
    > run concurrently.
    >
    > So how would I go about putting a limit on the number of threads that
    > run at any given time? I would like to take, say, 5 threads from the
    > pool and run them, and as each one completes, it is removed from the
    > pool and is replaced with a new thread from the pool.
    >
    > Could this be done with a "spy" thread, in that it constantly loops to
    > check how many threads are running at once, and if the number of running
    > threads falls below the limit of 5, it takes the next thread out of the
    > pool and runs it? Not sure how I would go about doing this, pretty new
    > to multithreading.


    I've had very good success using the Threadz gem.

    http://github.com/nanodeath/threadz

    It's quite easy to understand and works very well with MRI and JRuby.

    cr
     
    Chuck Remes, Mar 5, 2010
    #5
  6. Joe Martin

    Joe Martin Guest

    Robert Klemme wrote:
    > 2010/3/4 Joe Martin <>:
    >> check how many threads are running at once, and if the number of running
    >> threads falls below the limit of 5, it takes the next thread out of the
    >> pool and runs it? �Not sure how I would go about doing this, pretty new
    >> to multithreading.

    >
    > Why do you create a pool much larger than the load you want to accept?
    > Usually the pool size is used to limit concurrency. Actually that is
    > the main purpose of thread pools.
    >
    > If you have different tasks for which you want to have different
    > limits on concurrency you could also create several pools with
    > different sizes.
    >
    > Kind regards
    >
    > robert


    When running this program, I will provide a list of items that need
    processing. In some cases, this list can be as long as 250 items, in
    other cases well over 50,000. The processing of each item can take
    anywhere from 15 to 60 seconds per item, so you can see there is a
    benefit to multithreading here. In processing each item, there are also
    a number of database calls that occur, so I would like to put a cap on
    the number of actively running threads to avoid overwhelming the
    database. Am I going about this the wrong way? Is there a more
    effecient more suitable way of doing this?
    --
    Posted via http://www.ruby-forum.com/.
     
    Joe Martin, Mar 5, 2010
    #6
  7. 2010/3/5 Joe Martin <>:
    > Robert Klemme wrote:
    >> 2010/3/4 Joe Martin <>:
    >>> check how many threads are running at once, and if the number of runnin=

    g
    >>> threads falls below the limit of 5, it takes the next thread out of the
    >>> pool and runs it? =EF=BF=BDNot sure how I would go about doing this, pr=

    etty new
    >>> to multithreading.

    >>
    >> Why do you create a pool much larger than the load you want to accept?
    >> =C2=A0Usually the pool size is used to limit concurrency. =C2=A0Actually=

    that is
    >> the main purpose of thread pools.
    >>
    >> If you have different tasks for which you want to have different
    >> limits on concurrency you could also create several pools with
    >> different sizes.
    >>
    >> Kind regards
    >>
    >> robert

    >
    > When running this program, I will provide a list of items that need
    > processing. =C2=A0In some cases, this list can be as long as 250 items, i=

    n
    > other cases well over 50,000. =C2=A0The processing of each item can take
    > anywhere from 15 to 60 seconds per item, so you can see there is a
    > benefit to multithreading here. =C2=A0In processing each item, there are =

    also
    > a number of database calls that occur, so I would like to put a cap on
    > the number of actively running threads to avoid overwhelming the
    > database. =C2=A0Am I going about this the wrong way? =C2=A0Is there a mor=

    e
    > effecient more suitable way of doing this?


    For this scenario a thread pool with fixed size seems sufficient.

    queue =3D Queue.new # or bounded queue

    def cont(item) !item.nil? end

    threads =3D (1..10).map do
    Thread.new do
    while (cont(item =3D queue.deq))
    # .. process
    end
    end
    end

    queue.enc "Task"

    threads.size.times do
    queue.enq nil # terminate
    end

    threads.each {|th| th.join}

    As simple as that.

    Kind regards

    robert


    --=20
    remember.guy do |as, often| as.you_can - without end
    http://blog.rubybestpractices.com/
     
    Robert Klemme, Mar 5, 2010
    #7
  8. Joe Martin

    Chuck Remes Guest

    On Mar 5, 2010, at 7:06 AM, Joe Martin wrote:

    > Robert Klemme wrote:
    >> 2010/3/4 Joe Martin <>:
    >>> check how many threads are running at once, and if the number of =

    running
    >>> threads falls below the limit of 5, it takes the next thread out of =

    the
    >>> pool and runs it? =EF=BF=BDNot sure how I would go about doing this, =

    pretty new
    >>> to multithreading.

    >>=20
    >> Why do you create a pool much larger than the load you want to =

    accept?
    >> Usually the pool size is used to limit concurrency. Actually that is
    >> the main purpose of thread pools.
    >>=20
    >> If you have different tasks for which you want to have different
    >> limits on concurrency you could also create several pools with
    >> different sizes.
    >>=20
    >> Kind regards
    >>=20
    >> robert

    >=20
    > When running this program, I will provide a list of items that need=20
    > processing. In some cases, this list can be as long as 250 items, in=20=


    > other cases well over 50,000. The processing of each item can take=20
    > anywhere from 15 to 60 seconds per item, so you can see there is a=20
    > benefit to multithreading here. In processing each item, there are =

    also=20
    > a number of database calls that occur, so I would like to put a cap on=20=


    > the number of actively running threads to avoid overwhelming the=20
    > database. Am I going about this the wrong way? Is there a more=20
    > effecient more suitable way of doing this?


    The Threadz gem let's you create a thread pool and then wait on its =
    completion before you add more to it. This mechanism will help you cap =
    the number of threads making database calls.

    cr
     
    Chuck Remes, Mar 5, 2010
    #8
  9. Joe Martin

    Joe Martin Guest

    Robert Klemme wrote:
    > For this scenario a thread pool with fixed size seems sufficient.
    >


    Very good. This works quite nicely as well.

    Just wondering, are there any performance benefits of using one method
    over the other?
    --
    Posted via http://www.ruby-forum.com/.
     
    Joe Martin, Mar 5, 2010
    #9
  10. On 3/5/10, Joe Martin <> wrote:
    > Robert Klemme wrote:
    >> For this scenario a thread pool with fixed size seems sufficient.
    >>

    >
    > Very good. This works quite nicely as well.
    >
    > Just wondering, are there any performance benefits of using one method
    > over the other?


    Yes. Threads use memory (quite a lot of it, in fact). Mostly this goes
    to the thread's stack. Limiting the number of threads saves quite a
    bit of memory. I'm not sure there's any improvement in the amount of
    cpu time either way, other than perhaps some fewer cache misses
    resulting from using less memory.
     
    Caleb Clausen, Mar 5, 2010
    #10
  11. On 03/05/2010 09:59 PM, Joe Martin wrote:
    > Robert Klemme wrote:
    >> For this scenario a thread pool with fixed size seems sufficient.
    >>

    >
    > Very good. This works quite nicely as well.
    >
    > Just wondering, are there any performance benefits of using one method
    > over the other?


    Which other method are you referring to?

    Kind regards

    robert

    --
    remember.guy do |as, often| as.you_can - without end
    http://blog.rubybestpractices.com/
     
    Robert Klemme, Mar 6, 2010
    #11
  12. Joe Martin

    Joe Martin Guest

    Caleb Clausen wrote:
    > Yes. Threads use memory (quite a lot of it, in fact). Mostly this goes
    > to the thread's stack. Limiting the number of threads saves quite a
    > bit of memory. I'm not sure there's any improvement in the amount of
    > cpu time either way, other than perhaps some fewer cache misses
    > resulting from using less memory.


    Thanks! I figured this was the case.
    --
    Posted via http://www.ruby-forum.com/.
     
    Joe Martin, Mar 8, 2010
    #12
  13. Joe Martin

    Joe Martin Guest

    Robert Klemme wrote:
    > Which other method are you referring to?


    Well my original thinking was to create a thread for every single item I
    pass to the program. Caleb answered this for me in that threads
    themselves are quite heavy, so it further leads me to go with your
    solution which creates a low number of threads and re-uses them until
    the queue is empty. It seems much more simple and effecient than my
    original plan.
    --
    Posted via http://www.ruby-forum.com/.
     
    Joe Martin, Mar 8, 2010
    #13
  14. 2010/3/8 Joe Martin <>:
    > Robert Klemme wrote:
    >> Which other method are you referring to?

    >
    > Well my original thinking was to create a thread for every single item I
    > pass to the program. =A0Caleb answered this for me in that threads
    > themselves are quite heavy, so it further leads me to go with your
    > solution which creates a low number of threads and re-uses them until
    > the queue is empty. =A0It seems much more simple and effecient than my
    > original plan.


    Ah, OK. Then I was just confused because you had used the term
    "thread pool" in your original posting. That term is commonly use for
    exactly the type of solution I posted (fixed or at least limited
    number of threads which get their tasks from some form of queue). You
    probably weren't aware of this.

    Kind regards

    robert


    --=20
    remember.guy do |as, often| as.you_can - without end
    http://blog.rubybestpractices.com/
     
    Robert Klemme, Mar 8, 2010
    #14
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Pep
    Replies:
    6
    Views:
    836
  2. Paul

    Limit # of concurrent logins.

    Paul, May 4, 2006, in forum: ASP .Net
    Replies:
    4
    Views:
    687
    Rajesh CKR
    May 5, 2006
  3. TroutyJoe
    Replies:
    1
    Views:
    989
    =?Utf-8?B?Y2xpY2tvbg==?=
    Aug 9, 2006
  4. Replies:
    2
    Views:
    922
    Carramba
    Oct 27, 2006
  5. Rick Lawson
    Replies:
    8
    Views:
    823
    Graham Dumpleton
    Jul 17, 2009
Loading...

Share This Page