fork and blocking...

T

Tom

Greetings,

I need a little help with forking, I got the following code off
the web and modified for doing "df" on file systems. I hope to change
this to "rsync" at some point. The problem is I don't want to run any
more that 5 rsyncs at a time. The forking is working fine, however my
blocking isn't... It gets the first 5 off fine but never decrements
the counter I'm using for blocking. Is there a better way than using a
counter?

Thanks in Advanced!
Tom



my @childs;
my @FILESYSTEMS=("Alienbrain_Proxy", "programs", "cae", "codesign",
"eng", "Gold_Build_Patches", "Gold_Builds", "Gold_Patches", "hwdev",
"Media_Shares");

$blockcount = 0;
foreach $item ( @FILESYSTEMS ) {
$blockcount++;
print "$item\n";
while ( $blockcount > 4 ) {
print "$blockcount\n";
sleep 1;
}
my $pid = fork();
if ($pid) {
# parent
# print "pid is $pid, parent $$\n";
push(@childs, $pid);
} elsif ($pid == 0) {
# child
fsize ("$item");
$blockcount--;
exit 0;
} else {
die "Couldn't Fork!: $!\n";
}
}

foreach (@childs) {
my $tmp = waitpid($_, 0);
print "Done with pid $tmp\n";
}

print "End of main program\n";


sub fsize {
my $input = shift;
system ("df /net/server1/$input/$input > /tmp/$input.log");
}
 
T

Ted Zlatanov

T> I need a little help with forking, I got the following code off
T> the web and modified for doing "df" on file systems. I hope to change
T> this to "rsync" at some point. The problem is I don't want to run any
T> more that 5 rsyncs at a time. The forking is working fine, however my
T> blocking isn't... It gets the first 5 off fine but never decrements
T> the counter I'm using for blocking. Is there a better way than using a
T> counter?

Have you looked at Parallel::ForkManager
(http://search.cpan.org/~dlux/Parallel-ForkManager-0.7.5/ForkManager.pm)?

Ted
 
J

Jim Gibson

Tom said:
Greetings,

I need a little help with forking, I got the following code off
the web and modified for doing "df" on file systems. I hope to change
this to "rsync" at some point. The problem is I don't want to run any
more that 5 rsyncs at a time. The forking is working fine, however my
blocking isn't... It gets the first 5 off fine but never decrements
the counter I'm using for blocking. Is there a better way than using a
counter?

Thanks in Advanced!
Tom



my @childs;
my @FILESYSTEMS=("Alienbrain_Proxy", "programs", "cae", "codesign",
"eng", "Gold_Build_Patches", "Gold_Builds", "Gold_Patches", "hwdev",
"Media_Shares");

$blockcount = 0;
foreach $item ( @FILESYSTEMS ) {
$blockcount++;
print "$item\n";
while ( $blockcount > 4 ) {
print "$blockcount\n";
sleep 1;
}

Once you get into this loop, when $blockcount > 4, you never get out.
You are in an infinite loop. Decrementing $blockcount is done by lines
below, but they never get executed. You will need to add a wait
statement here, waiting until one of your child processes has
completed, then decrementing $blockcount and removing the child process
from the array of PIDs.
 
J

Jens Thoms Toerring

Tom said:
... The forking is working fine, however my
blocking isn't... It gets the first 5 off fine but never decrements
the counter I'm using for blocking. Is there a better way than using a
counter?
my @childs;
my @FILESYSTEMS=("Alienbrain_Proxy", "programs", "cae", "codesign",
"eng", "Gold_Build_Patches", "Gold_Builds", "Gold_Patches", "hwdev",
"Media_Shares");
$blockcount = 0;
foreach $item ( @FILESYSTEMS ) {
$blockcount++;
print "$item\n";
while ( $blockcount > 4 ) {
print "$blockcount\n";
sleep 1;
}
my $pid = fork();
if ($pid) {
# parent
# print "pid is $pid, parent $$\n";
push(@childs, $pid);
} elsif ($pid == 0) {
# child
fsize ("$item");
$blockcount--;

Decrementing '$blockcount' is done in the child process,
which is a completely independend process with its own
memory. So the '$blockcount' variabe within this pro-
cess has no connection to that in the parent process.
Thus '$blockcount' gets only incremented in the parent
and never decremented.

One way this could be handled would be to install a signal
handler in the parent process for SIGCHLD signals and de-
crement '$blockcount' in there. I.e. add a line like

$SIG{ CHLD } = sub { $blockcount-- };

before you start the loop. That should do the trick (at least
on UNIX, I have no idea how Windows works in this respect).

And unless you want to avoid mixing the output for childs
created and exited, you could do the waitpid() call directly
in the signal handler

$SIG{ CHLD } = sub { while ( ( my $tmp = waitpid( -1, WNOHANG ) ) > 0 ) {
print "Done with pid $tmp\n";
exit 0 if --$blockcount == 0 ;
}
};

If you do that you need to 'use POSIX;' for the definition
of 'WNOHANG' and you need to replace the loop where you now
wait for all childs to finally exit with something like

pause while 1;

Don't do

pause while $blockcount,

but exit from the signal handler or you risk a so-called
race condition. That's because between the check here if
'$blockcount' is still larger than 0 and the start of the
pause() function a SIGCHLD signal could arrive that results
in '$blockcount' finally becoming 0. In that (not verly
likely but still possible) case the program would hang
forever.
Regards, Jens
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Similar Threads

Linux: using "clone3" and "waitid" 0
problem with fork 8
fork() & pipe() 3
Using fork() 3
fork it 11
fork and taint 5
issue with multiprocess - fork 2
How to resolve funky sync issues with fork here. 12

Members online

Forum statistics

Threads
473,744
Messages
2,569,482
Members
44,901
Latest member
Noble71S45

Latest Threads

Top