Child processes don't get the close on pipe

[Lothar Braun]

Hi,

I have a problem with pipes that I do not understand at all. My code
forks a number of child processes, and creates a pipe for every one of
them. The parent then writes data to the child's pipes, finishing the
whole process by closing the pipe.

The children consume to data and write results back to the parent
through the other side of the pipe. The problem is that the children
do not get an EOF on the pipes and expect more data, resulting in a
deadlock. I can't see what the problem with my code could be and
therefore extracted the relevant parts into the script pasted below
(you can also find it at pastebin: http://pastebin.com/KarksDXf )

Can someone point me to the problematic parts of the code?

#!/usr/bin/env perl -w

use strict;
use POSIX;

my $num_processes = 2;
my %process_data;

sub worker_thread {
my $id = shift;
my $reader = shift;
my $writer = shift;

print STDERR "Child $id: Starting to consume input ...\n";
while(<$reader>) {
chomp;
print STDERR "$id: Got line \"$_\"\"\n";
}

print STDERR "Finished consuming packets ...";
print $writer "$id: this is my result\n";
}

sub parent_reader {
for (0 .. $num_processes - 1) {
my $writer = ${$process_data{$_}}[1];
print $writer "Got some work\n";
}
}
################ main

# start the worker processes
for my $pnum (0 .. $num_processes - 1) {
print $pnum, "\n";
local *READER;
local *WRITER;
pipe(*READER, *WRITER);
my $pid = fork();
if (defined $pid && $pid == 0) {
worker_thread($pnum, *READER, *WRITER);
exit -1;
} elsif (defined $pid && $pid > 0) {
$process_data{$pnum} = [ *READER, *WRITER, $pnum];
}
}

# run the main reader loop, and pass data to the readers
parent_reader();

print "Parent: Finished reading input data from stdin...\n";

# close the pipes to indicate that there will be no more data
for my $i (keys %process_data) {
my $writer = $process_data{$i}[1];
print scalar $writer, "\n";
close($writer) or warn "Error closing pipe: $!";
}

# collect the results
for my $i (keys %process_data) {
my $reader = ${process_data{$i}}[0];
while (<$reader>) {
print "$_";
}
}

# wait for child processes to end
for my $i (keys %process_data) {
my $ret = waitpid(-1, 0);

if ($ret<0 || $ret>0) {
print "process $ret finished..\n";
delete $process_data{$ret};
}
}

 

[Rainer Weikusat]

I have a problem with pipes that I do not understand at all. My code
forks a number of child processes, and creates a pipe for every one of
them. The parent then writes data to the child's pipes, finishing the
whole process by closing the pipe.

The children consume to data and write results back to the parent
through the other side of the pipe. The problem is that the children
do not get an EOF on the pipes
[...]

sub worker_thread {
my $id = shift;
my $reader = shift;
my $writer = shift;

print STDERR "Child $id: Starting to consume input ...\n";
while(<$reader>) {
chomp;
print STDERR "$id: Got line \"$_\"\"\n";
}

print STDERR "Finished consuming packets ...";
print $writer "$id: this is my result\n";
}
[...]

for my $pnum (0 .. $num_processes - 1) {
print $pnum, "\n";
local *READER;
local *WRITER;
pipe(*READER, *WRITER);
my $pid = fork();
if (defined $pid && $pid == 0) {
worker_thread($pnum, *READER, *WRITER);

[...]

You'll get an EOF on a pipe as soon as one side of it has been
closed. Since the child inherits the 'writer' handle but doesn't close
it, this will never happen. You could use two pipes instead but it is
probably easier to use an AF_UNIX SOCK_STREAM socketpair (=> perldoc
-f socketpair), closing one of the handles in the parent process and
the other in the child.

 

[Lothar Braun]

Hi,

You'll get an EOF on a pipe as soon as one side of it has been
closed. Since the child inherits the 'writer' handle but doesn't close
it, this will never happen.

Ok, I can now see the problem.

You could use two pipes instead but it is
probably easier to use an AF_UNIX SOCK_STREAM socketpair (=> perldoc
-f socketpair), closing one of the handles in the parent process and
the other in the child.

Thank you. I was able to fix my problem with this one

- Lothar

 

[Rainer Weikusat]

[...]

You could use two pipes instead but it is
probably easier to use an AF_UNIX SOCK_STREAM socketpair (=> perldoc
-f socketpair), closing one of the handles in the parent process and
the other in the child.

Thank you. I was able to fix my problem with this one

Possibly useful remark: Sockets also support asymmetric communication
shutdown. The parent can use a suitable shutdown call to announce that
it is done with writing data. The child will then see an EOF but can
still use its file descriptor to send data to the parent.