Child processes don't get the close on pipe

Discussion in 'Perl Misc' started by Lothar Braun, Jun 2, 2012.

  1. Lothar Braun

    Lothar Braun Guest

    Hi,

    I have a problem with pipes that I do not understand at all. My code
    forks a number of child processes, and creates a pipe for every one of
    them. The parent then writes data to the child's pipes, finishing the
    whole process by closing the pipe.

    The children consume to data and write results back to the parent
    through the other side of the pipe. The problem is that the children
    do not get an EOF on the pipes and expect more data, resulting in a
    deadlock. I can't see what the problem with my code could be and
    therefore extracted the relevant parts into the script pasted below
    (you can also find it at pastebin: http://pastebin.com/KarksDXf )

    Can someone point me to the problematic parts of the code?

    #!/usr/bin/env perl -w

    use strict;
    use POSIX;

    my $num_processes = 2;
    my %process_data;

    sub worker_thread {
    my $id = shift;
    my $reader = shift;
    my $writer = shift;

    print STDERR "Child $id: Starting to consume input ...\n";
    while(<$reader>) {
    chomp;
    print STDERR "$id: Got line \"$_\"\"\n";
    }

    print STDERR "Finished consuming packets ...";
    print $writer "$id: this is my result\n";
    }

    sub parent_reader {
    for (0 .. $num_processes - 1) {
    my $writer = ${$process_data{$_}}[1];
    print $writer "Got some work\n";
    }
    }
    ################ main

    # start the worker processes
    for my $pnum (0 .. $num_processes - 1) {
    print $pnum, "\n";
    local *READER;
    local *WRITER;
    pipe(*READER, *WRITER);
    my $pid = fork();
    if (defined $pid && $pid == 0) {
    worker_thread($pnum, *READER, *WRITER);
    exit -1;
    } elsif (defined $pid && $pid > 0) {
    $process_data{$pnum} = [ *READER, *WRITER, $pnum];
    }
    }

    # run the main reader loop, and pass data to the readers
    parent_reader();

    print "Parent: Finished reading input data from stdin...\n";

    # close the pipes to indicate that there will be no more data
    for my $i (keys %process_data) {
    my $writer = $process_data{$i}[1];
    print scalar $writer, "\n";
    close($writer) or warn "Error closing pipe: $!";
    }

    # collect the results
    for my $i (keys %process_data) {
    my $reader = ${process_data{$i}}[0];
    while (<$reader>) {
    print "$_";
    }
    }

    # wait for child processes to end
    for my $i (keys %process_data) {
    my $ret = waitpid(-1, 0);

    if ($ret<0 || $ret>0) {
    print "process $ret finished..\n";
    delete $process_data{$ret};
    }
    }
     
    Lothar Braun, Jun 2, 2012
    #1
    1. Advertising

  2. Lothar Braun <> writes:
    >
    > I have a problem with pipes that I do not understand at all. My code
    > forks a number of child processes, and creates a pipe for every one of
    > them. The parent then writes data to the child's pipes, finishing the
    > whole process by closing the pipe.
    >
    > The children consume to data and write results back to the parent
    > through the other side of the pipe. The problem is that the children
    > do not get an EOF on the pipes


    [...]

    > sub worker_thread {
    > my $id = shift;
    > my $reader = shift;
    > my $writer = shift;
    >
    > print STDERR "Child $id: Starting to consume input ...\n";
    > while(<$reader>) {
    > chomp;
    > print STDERR "$id: Got line \"$_\"\"\n";
    > }
    >
    > print STDERR "Finished consuming packets ...";
    > print $writer "$id: this is my result\n";
    > }


    [...]

    > for my $pnum (0 .. $num_processes - 1) {
    > print $pnum, "\n";
    > local *READER;
    > local *WRITER;
    > pipe(*READER, *WRITER);
    > my $pid = fork();
    > if (defined $pid && $pid == 0) {
    > worker_thread($pnum, *READER, *WRITER);


    [...]

    You'll get an EOF on a pipe as soon as one side of it has been
    closed. Since the child inherits the 'writer' handle but doesn't close
    it, this will never happen. You could use two pipes instead but it is
    probably easier to use an AF_UNIX SOCK_STREAM socketpair (=> perldoc
    -f socketpair), closing one of the handles in the parent process and
    the other in the child.
     
    Rainer Weikusat, Jun 2, 2012
    #2
    1. Advertising

  3. Lothar Braun

    Lothar Braun Guest

    Hi,

    On Jun 2, 4:55 pm, Rainer Weikusat <> wrote:
    > You'll get an EOF on a pipe as soon as one side of it has been
    > closed. Since the child inherits the 'writer' handle but doesn't close
    > it, this will never happen.


    Ok, I can now see the problem.

    > You could use two pipes instead but it is
    > probably easier to use an AF_UNIX SOCK_STREAM socketpair (=> perldoc
    > -f socketpair), closing one of the handles in the parent process and
    > the other in the child.


    Thank you. I was able to fix my problem with this one :)

    - Lothar
     
    Lothar Braun, Jun 3, 2012
    #3
  4. Lothar Braun <> writes:
    > On Jun 2, 4:55 pm, Rainer Weikusat <> wrote:


    [...]

    >> You could use two pipes instead but it is
    >> probably easier to use an AF_UNIX SOCK_STREAM socketpair (=> perldoc
    >> -f socketpair), closing one of the handles in the parent process and
    >> the other in the child.

    >
    > Thank you. I was able to fix my problem with this one :)


    Possibly useful remark: Sockets also support asymmetric communication
    shutdown. The parent can use a suitable shutdown call to announce that
    it is done with writing data. The child will then see an EOF but can
    still use its file descriptor to send data to the parent.
     
    Rainer Weikusat, Jun 3, 2012
    #4
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Denon
    Replies:
    1
    Views:
    603
    Saravana [MVP]
    Nov 14, 2003
  2. Replies:
    6
    Views:
    8,937
    sateeshranga
    Aug 7, 2008
  3. Jeff Rodriguez
    Replies:
    23
    Views:
    1,204
    David Schwartz
    Dec 9, 2003
  4. Iñaki Baz Castillo
    Replies:
    7
    Views:
    945
    Iñaki Baz Castillo
    Jan 12, 2010
  5. Replies:
    3
    Views:
    445
    Chung Leong
    Jan 16, 2005
Loading...

Share This Page