Trouble with throttling fork()

[Jim]

I'm having a hard time using fork() and throttling it. Perhaps it is
because I am using the fork() emulation in Activestate Perl for
Windows. Basically, what I want to do is loop through an array and run
a process on each element. I need to fork each process but the array
is hundreds of elements big and I can only run 8 processes at a time.

my $parent = $$;
$StartProc = 1;
$TotProcs = 8;

# Trap signals from child processes and run this subroutine.
$SIG{CHLD} = \&ChldStopped;

foreach $x (@array) {
if ($parent == $$){ # this is parent process
while ($StartProc > $TotProcs) { # limit to 8 at a time
sleep 1;
}
$StartProc ++;
my $child = fork();
if ($child == 0) { # This is a child process.
# run process on $x
exit 1;
}
}

sub ChldStopped {
$StartProc --;
}

 

[J. Gleixner]

I'm having a hard time using fork() and throttling it. [...]

Take a look at Proc::Queue
http://search.cpan.org/~salva/Proc-Queue-1.13/Queue.pm

 

[Salvador Fandino]

I'm having a hard time using fork() and throttling it. [...]

Take a look at Proc::Queue
http://search.cpan.org/~salva/Proc-Queue-1.13/Queue.pm

Hi,

Proc::Queue doesn't work on Windows, sorry :-(

Bye,

- Salva

 

[Salvador Fandino]

Proc::Queue doesn't work on Windows, sorry :-(

Well, it does now ;-)

I have uploaded a new beta version 1.14_01 with windows support to CPAN.

I will appreciate feedback about how it works for "real" windows apps.

Bye,

- Salva

 

[Jim]

Jim wrote:

(snipped)





Reset your alias value for your interrupt signal.


sub ChldStopped
{
$StartProc --;
$SIG{CHLD} = \&ChldStopped;
}

Once an interrupt signal is used, the system default value
for that interrupt siganl is reset. You need to reset your
custom value each time your sub-routine is called.

Your current code will only deincrement your loop control
value once, then stop. Reset your $SIG{CHLD} custom value
in your sub-routine and your code should work, at least
for this part of your code.

You will find a few articles written by me, I think four to
six months back, addressing this problem. I cannot remember
the subject title but you will find $SIG(INT} within the
body of my articles, perhaps in the subject title as well.

Here is a link which provides a similar case as yours
and examples of using and resetting interrupt signals:

http://www.purlgurl.net/~purlgurl/perl/sig_int/sig_int.html


Purl Gurl

I made the change you suggested and I have the same problem. Right
now, the script spits out the following and just hangs (substituting
print "jim - $x"; for # run process on $x).

jim - 0
jim - 1
jim - 2
jim - 3

Jim

 

[Steve East]

Another claims resetting of an interrupt signal to a custom
value with each usage is not needed. This is untrue.

Solaris 8 and perl 5.8.0:

$SIG{HUP} = \&handleHUP;

kill HUP, $$;
sleep 2;
kill HUP, $$;
sleep 2;
kill HUP, $$;
sleep 2;

sub handleHUP {
print "Hey, hey, we're the Monkees...\n";
}


produces:

Hey, hey, we're the Monkees...
Hey, hey, we're the Monkees...
Hey, hey, we're the Monkees...


This doesn't mean that some systems don't still use the unreliable Sys V
signals that did need to be re-installed.

 

[Steve East]

(snipped East support of not resetting an interrupt signal)

Once you educate yourself in this topic area, you will realize
resetting an interrupt signal for a custom value is, de facto,
required for all systems, as a good programming practice.

Way too much pomposity for me. Killfiled.