Thread pool issue

S

Sriram Varahan

Hello,


I was just trying out the thread pool code form Ruby Cookbook.

I created a script to test it out. The complete code is as follows:
-----------------------------------------------------------------------------
require 'thread'

class ThreadPool
def initialize(max_size)
@pool = []
@max_size = max_size
@pool_mutex = Mutex.new
@pool_cv = ConditionVariable.new
end

def dispatch(*args)
Thread.new do
# Wait for space in the pool.
@pool_mutex.synchronize do
while @pool.size >= @max_size
puts "Pool is full; waiting to run #{args.join(',')}…\n"
# Sleep until some other thread calls @pool_cv.signal.
@pool_cv.wait(@pool_mutex)
end
end
@pool << Thread.current
begin
yield(*args)
rescue => e
exception(self, e, *args)
ensure
@pool_mutex.synchronize do
# Remove the thread from the pool.
@pool.delete(Thread.current)
# Signal the next waiting thread that there's a space in the
pool.
@pool_cv.signal
end
end
end
end

def shutdown
@pool_mutex.synchronize { @pool_cv.wait(@pool_mutex) until
@pool.empty? }
end

def exception(thread, exception, *original_args)
# Subclass this method to handle an exception within a thread.
puts "Exception in thread #{thread}: #{exception}"
end
end

@work = []
def no_work_only_sleep
sleep(2)
@work << "Finished"
end

pool = ThreadPool.new(3)

10.times do |i|
pool.dispatch(i) do
puts "Job #{i} started"
no_work_only_sleep
end
end

pool.shutdown
puts @work.length

-------------------------------------------------------------------------
What I am doing here is creating 10 jobs and within a job calling a
method.
Inside the method I am inserting a string "Finished" to an array @work.
So for 10 jobs this array should contain 10 such strings.

The issue I have here is : For a thread pool of size 3 it gives an
output of 9 where it should have been 10.

For any other thread pool value it gives 10 as required.

Any reason as to why this is happening?

Thanks,
Sriram.
 
R

Robert Klemme

2010/3/18 Sriram Varahan said:
Hello,


=A0 =A0 I was just trying out the thread pool code form Ruby Cookbook.

I created a script to test it out. The complete code is as follows:
-------------------------------------------------------------------------= ----
require 'thread'

class ThreadPool
=A0def initialize(max_size)
=A0 =A0@pool =A0 =A0 =A0 =3D []
=A0 =A0@max_size =A0 =3D max_size
=A0 =A0@pool_mutex =3D Mutex.new
=A0 =A0@pool_cv =A0 =A0=3D ConditionVariable.new
=A0end

=A0def dispatch(*args)
=A0 =A0Thread.new do
=A0 =A0 =A0# Wait for space in the pool.
=A0 =A0 =A0@pool_mutex.synchronize do
=A0 =A0 =A0 =A0while @pool.size >=3D @max_size
=A0 =A0 =A0 =A0 =A0puts "Pool is full; waiting to run #{args.join(',')}= =85\n"
=A0 =A0 =A0 =A0 =A0# Sleep until some other thread calls @pool_cv.signal.
=A0 =A0 =A0 =A0 =A0@pool_cv.wait(@pool_mutex)
=A0 =A0 =A0 =A0end
=A0 =A0 =A0end
=A0 =A0 =A0@pool << Thread.current
=A0 =A0 =A0begin
=A0 =A0 =A0 =A0yield(*args)
=A0 =A0 =A0rescue =3D> e
=A0 =A0 =A0 =A0exception(self, e, *args)
=A0 =A0 =A0ensure
=A0 =A0 =A0 =A0@pool_mutex.synchronize do
=A0 =A0 =A0 =A0 =A0# Remove the thread from the pool.
=A0 =A0 =A0 =A0 [email protected](Thread.current)
=A0 =A0 =A0 =A0 =A0# Signal the next waiting thread that there's a space = in the
pool.
=A0 =A0 =A0 =A0 =A0@pool_cv.signal
=A0 =A0 =A0 =A0end
=A0 =A0 =A0end
=A0 =A0end
=A0end

=A0def shutdown
=A0 =A0@pool_mutex.synchronize { @pool_cv.wait(@pool_mutex) until
@pool.empty? }
=A0end

=A0def exception(thread, exception, *original_args)
=A0 =A0# Subclass this method to handle an exception within a thread.
=A0 =A0puts "Exception in thread #{thread}: #{exception}"
=A0end
end

@work =3D []
def no_work_only_sleep
=A0sleep(2)
=A0@work << "Finished"
end

pool =3D ThreadPool.new(3)

10.times do |i|
=A0pool.dispatch(i) do
=A0 =A0puts "Job #{i} started"
=A0 =A0no_work_only_sleep
=A0end
end

pool.shutdown
puts @work.length

-------------------------------------------------------------------------
What I am doing here is creating 10 jobs and within a job calling a
method.
Inside the method I am inserting a string "Finished" to an array @work.
So for 10 jobs this array should contain 10 such strings.

The issue I have here is : For a thread pool of size 3 it gives an
output of 9 where it should have been 10.

For any other thread pool value it gives 10 as required.

Any reason as to why this is happening?

Two things stand out:

1. as far as I can see you are not properly synchronizing access to @Work.

2. IMHO the check of the current thread pool size should be done in
the calling thread in order to block it - not in the newly created
background thread.

Issue 2 is a bit tricky to understand but I believe you have created a
situation where background threads are started and waiting for room in
the pool to become available but the shutdown may get in the way so
you might terminate the pool although there are still threads waiting
to start their work.

Kind regards

robert

--=20
remember.guy do |as, often| as.you_can - without end
http://blog.rubybestpractices.com/
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

Forum statistics

Threads
473,821
Messages
2,569,748
Members
45,726
Latest member
RaleighAll

Latest Threads

Top