D
David Karr
I had a nested loop where each iteration doesn't depend on any other iteration, so I concluded I could use Thread:ool to queue up the iterations to make the loop go faster. Each iteration makes an HTTP connection to a server, so it likely would be faster using a thread pool.
So, I moved the body of the iteration to a new sub, and added a sub parameter for the two loop indices (which the iteration body needs). The body needs to reference other data defined earlier in the script, which I left as is in the new sub.
I ended up replacing the body with the following approximate line:
$pool->enqueue(\&mynewsub, $rindex, $dindex);
The first thing I noticed were warnings like the following:
pragma "attrs" is deprecated, use "sub NAME : ATTRS" instead at /usr/lib/perl5/site_perl/5.10/Thread/Pool.pm line 98.
Fortunately, this was an easy fix. Pool.pm was using the deprecated "use attrs". so I changed it to the newer format. I also filed a bug report on Thread:ool for this.
The next problem I don't have a fix for yet. When it hit the loop and started enqueuing tasks, I saw the following on the console:
Thread 343 terminated abnormally: cond_wait can only be used on shared values at /usr/lib/perl5/site_perl/5.10/Thread/Pool.pm line 100.
This is in the following method:
sub _next_job : locked : method {
my $pool = shift;
cond_wait($pool) while $pool->{num} <= $pool->{min} && !@{$pool->{jobs}};
my $job = $pool->{jobs}->[0];
shift @{$pool->{jobs}} if defined $job;
return $job;
}
I don't know if this represents a bug in Thread:ool or some problem with my script. If it matters, these are my first statements in the script:
use threads;
use threads::shared;
use Thread:ool;
The other thing I'm wondering about with Thread:ool is concerning what happens at the end of my script? Do I have to do some sort of "join()" to ensure all the jobs are completed? I didn't see any way to do this in the Thread:ool documentation.
So, I moved the body of the iteration to a new sub, and added a sub parameter for the two loop indices (which the iteration body needs). The body needs to reference other data defined earlier in the script, which I left as is in the new sub.
I ended up replacing the body with the following approximate line:
$pool->enqueue(\&mynewsub, $rindex, $dindex);
The first thing I noticed were warnings like the following:
pragma "attrs" is deprecated, use "sub NAME : ATTRS" instead at /usr/lib/perl5/site_perl/5.10/Thread/Pool.pm line 98.
Fortunately, this was an easy fix. Pool.pm was using the deprecated "use attrs". so I changed it to the newer format. I also filed a bug report on Thread:ool for this.
The next problem I don't have a fix for yet. When it hit the loop and started enqueuing tasks, I saw the following on the console:
Thread 343 terminated abnormally: cond_wait can only be used on shared values at /usr/lib/perl5/site_perl/5.10/Thread/Pool.pm line 100.
This is in the following method:
sub _next_job : locked : method {
my $pool = shift;
cond_wait($pool) while $pool->{num} <= $pool->{min} && !@{$pool->{jobs}};
my $job = $pool->{jobs}->[0];
shift @{$pool->{jobs}} if defined $job;
return $job;
}
I don't know if this represents a bug in Thread:ool or some problem with my script. If it matters, these are my first statements in the script:
use threads;
use threads::shared;
use Thread:ool;
The other thing I'm wondering about with Thread:ool is concerning what happens at the end of my script? Do I have to do some sort of "join()" to ensure all the jobs are completed? I didn't see any way to do this in the Thread:ool documentation.