Marc the Reaper

[Marc Girod]

Hello,

I have a nightly build script which forks a lot of processes which may
last long or not.
I wanted to reap them before the end of the script.
This addition made, I lose the exit code of my builds: they all
pretend to fail, which I suspect to be false.
So, I guess I was not able to read the perlipc page correctly.

Here is a short script which behaves in the same way as my nightly
build;

foo:

#!/usr/bin/perl -w
use strict;
use POSIX ":sys_wait_h";

sub child {
system('date');
my $ec = $? >> 8;
warn "Child exit code: $?\n";
exit $ec;
}
my %family;
sub reaper { # from perlipc
local ($!, $?); # don't let waitpid() overwrite current error
my ($prd, $child) = '';
while (($child = waitpid(-1, WNOHANG)) > 0) {
if (exists $family{$child}) {
$prd = $family{$child};
delete $family{$child};
warn "Reaped $prd($child) with exit $?\n";
}
}
$SIG{CHLD} = \&reaper;
}
$SIG{CHLD} = \&reaper;

if (my $pid = fork) {
$family{$pid} = 'date';
} else {
die "cannot fork: $!" unless defined($pid);
child;
}
while (%family) {
foreach my $kid (keys %family) {
delete $family{$kid} if waitpid($kid, WNOHANG);
}
sleep 1;
}
warn "Parent exit\n";
exit 0;


And an example run:

$ ./foo
Wed Sep 15 14:15:17 BST 2010
Child exit code: -1
Reaped date(21287) with exit 65280
Parent exit

So, how to protect the return code?
Thanks,
Marc

 

[Marc Girod]

Just two additions to my post, to clarify:

So, how to protect the return code?

- I did 2 minors improvements inside the reaper function:
1. removed the useless initialization
2. shifted the error code there as I had done in the child function

sub reaper { # from perlipc
local ($!, $?); # don't let waitpid() overwrite current error
my ($prd, $child) = '';
while (($child = waitpid(-1, WNOHANG)) > 0) {
if (exists $family{$child}) {
$prd = $family{$child};
delete $family{$child};
warn "Reaped $prd($child) with exit @{[$?>>8]}\n";
}
}
$SIG{CHLD} = \&reaper;
}

- I checked that if I comment away the signal handler setting, date
starts to report success again:

#$SIG{CHLD} = \&reaper;


$ ./foo
Wed Sep 15 14:51:08 BST 2010
Child exit code: 0
Parent exit

Marc

 

[Marc Girod]

On Sep 15, 2:22 pm, Marc Girod

So, how to protect the return code?

I found this in perlvar:

If you have installed a signal handler for "SIGCHLD", the value
of $? will usually be wrong outside that handler.

Marc

 

[Alan Curry]

Hello,

I have a nightly build script which forks a lot of processes which may
last long or not.
I wanted to reap them before the end of the script.
This addition made, I lose the exit code of my builds: they all
pretend to fail, which I suspect to be false.
So, I guess I was not able to read the perlipc page correctly.

Here is a short script which behaves in the same way as my nightly
build;

Your child process inherited the signal handler from its parent. The signal
handler is stealing the child's exitcode before system() can get it. Since
your signal handler isn't necessary in the child process, you should be able
to fix this by setting $SIG{CHLD}='DEFAULT' before calling system().

 

[Marc Girod]

Your child process inherited the signal handler from its parent. The signal
handler is stealing the child's exitcode before system() can get it. Since
your signal handler isn't necessary in the child process, you should be able
to fix this by setting $SIG{CHLD}='DEFAULT' before calling system().

Thanks.
Marc

 

[Marc Girod]

Hi,

Maybe this issue with $?, and even resetting the handler in child
processes, should be mentioned in the perlipc chapter on reapers?
I know it is pretty long already.

I believe btw that I found one minor glitch there: $child was renamed
(or so I guess) to $waitedpid (with a change in scope) from one
example to an other, but the 'my' definition was left (forgotten)
useless.

Marc

 

[C.DeRykus]

...

Another suggestion on the code. The parent's
sleep loop before exiting has a redundant
waitpid. The reaper handler is asynchronous
and is still reaping during the sleep loop.

So you could change the handler slightly and
eliminate the waitpid in the sleep loop.

...

sub reaper {
  while ((my $child = waitpid(-1, WNOHANG)) > 0) {
    if (exists $family{$child}) {
      my $prd = delete $family{$child};
      warn "Reaped $prd($child) with exit $?\n";
    }
$SIG{CHLD} = \&reaper;
  }

...

sleep 1 while keys %family; # eliminate redundant waitpid

 

[Marc Girod]

So you could change the handler slightly and
eliminate the waitpid in the sleep loop.

Thanks. You are right, of course.
What is your reason to prefer 'while keys %family' to 'while %family'?

Marc

 

[Dr.Ruud]

Thanks. You are right, of course.
What is your reason to prefer 'while keys %family' to 'while %family'?

Benchmark! (keys() is normally lighter and faster)

 

[C.DeRykus]

Benchmark! (keys() is normally lighter and faster)

A quick benchmark seems to favor keys over HISC
(Hash In Scalar Context) but better benchmarks
would likely include variously sized hashes.

perldata also hints HISC is of limited use:

If you evaluate a hash in scalar context,
it returns false if the hash is empty. If
there are any key/value pairs, it returns true;
... This is pretty much useful only to find out
whether Perl's internal hashing algorithm is
performing poorly on your data set.

There's also an interesting internals discussion on
this topic which seemed somewhat inconclusive:

http://www.perlmonks.org/?node_id=760534

Some may like keys because it's more familiar;
HISC could give someone pause to remember what it
does or how it works. However, perldata documents
HISC to return false for an empty hash. A case of
TIMTOWTDI.