Windows fork emulation (and buffering?) problem

Discussion in 'Perl Misc' started by Henry Townsend, Apr 7, 2005.

  1. This problem involves forks and pipes. Take a look at this code:

    pipe(OREADER, OWRITER) || die "$0: pipe(): $!";
    my $pid = fork();
    if ($pid) {
    print "parent\n";
    close OWRITER;
    print while(<OREADER>);
    print "done\n";
    wait;
    } else {
    die "$0: Error: fork(): $!" if !defined($pid);
    print "child\n";
    close OREADER;
    open(STDOUT, ">&OWRITER") || die "$0: Error: STDOUT: $!";
    print "data to stdout\n";
    }

    On Unix it works correctly, printing:

    child
    parent
    data to stdout
    done

    On Windows, using the emulated fork semantics it prints:

    T:\>perl -w moo.pl
    parent
    child

    and hangs. Upon Ctrl-C the message "Terminating on signal SIGINT(2)" is
    printed.

    This is with Perl 5.8.6, ActiveState build 811. I've read "perldoc
    perlfork" and do not see a reason this should not work on Windows. I'm
    guessing there's a buffering problem. Pointers to help?

    --
    Thanks,
    Henry Townsend
    Henry Townsend, Apr 7, 2005
    #1
    1. Advertising

  2. Henry Townsend <> wrote in
    news::

    > This problem involves forks and pipes. Take a look at this code:
    >
    > pipe(OREADER, OWRITER) || die "$0: pipe(): $!";
    > my $pid = fork();
    > if ($pid) {
    > print "parent\n";
    > close OWRITER;
    > print while(<OREADER>);
    > print "done\n";
    > wait;
    > } else {
    > die "$0: Error: fork(): $!" if !defined($pid);
    > print "child\n";
    > close OREADER;
    > open(STDOUT, ">&OWRITER") || die "$0: Error: STDOUT: $!";
    > print "data to stdout\n";
    > }


    Not an expert on how these things work, especially the whole dup thing,
    but I just checked,

    #! /usr/bin/perl

    use strict;
    use warnings;

    pipe my $reader, my $writer or die "Pipe failed: $!";

    my $pid = fork();
    if ($pid) {
    print "parent\n";
    close $writer;
    print "received: $_" while(<$reader>);
    print "done\n";
    wait;
    } elsif(defined $pid) {
    print "child\n";
    close $reader;
    local(*STDOUT) = $writer;
    print "data to writer\n";
    } else {
    die "Cannot fork: $!";
    }
    __END__

    seems to do what you want on Windows.

    That doesn't mean there is no bug, it might also be related to the
    warning against potential for deadlock in

    perldoc -f pipe

    Dunno if this helps any.

    Sinan

    --
    A. Sinan Unur <>
    (reverse each component and remove .invalid for email address)

    comp.lang.perl.misc guidelines on the WWW:
    http://mail.augustmail.com/~tadmc/clpmisc/clpmisc_guidelines.html
    A. Sinan Unur, Apr 7, 2005
    #2
    1. Advertising

  3. A. Sinan Unur wrote:

    > Henry Townsend <> wrote in
    > news::
    > Not an expert on how these things work, especially the whole dup thing,
    > but I just checked,
    >
    > #! /usr/bin/perl
    >
    > use strict;
    > use warnings;
    >
    > pipe my $reader, my $writer or die "Pipe failed: $!";
    >
    > my $pid = fork();
    > if ($pid) {
    > print "parent\n";
    > close $writer;
    > print "received: $_" while(<$reader>);
    > print "done\n";
    > wait;
    > } elsif(defined $pid) {
    > print "child\n";
    > close $reader;
    > local(*STDOUT) = $writer;
    > print "data to writer\n";
    > } else {
    > die "Cannot fork: $!";
    > }
    > __END__
    >
    > seems to do what you want on Windows.
    >
    > That doesn't mean there is no bug, it might also be related to the
    > warning against potential for deadlock in
    >
    > perldoc -f pipe


    Yes, this works on Windows as well as Unix. Unfortunately I didn't go
    into enough detail about what I'm trying to do. I need to capture stdout
    and apply a regular expression to it, and when I say stdout I mean data
    flowing from subprocesses as well as that originating from the perl
    program. My fault for not making the distinction in the first place, but
    the original code did an open() of STDOUT and thus operates on the file
    *descriptor*, whereas your version operates on the file *handle*. The
    difference being in what they do with (say)

    system qw(cat file);

    I'm moving on, playing with 4-arg select() and will report progress.

    --
    Henry Townsend
    Henry Townsend, Apr 8, 2005
    #3
  4. Henry Townsend <> wrote in
    news::

    > A. Sinan Unur wrote:
    >
    >> Henry Townsend <> wrote in
    >> news::


    >> Not an expert on how these things work, especially the whole dup
    >> thing, but I just checked,


    ....

    > *handle*. The difference being in what they do with (say)
    >
    > system qw(cat file);


    I understand. Now, the question is, are you in control of the system
    call above? If you are, and you need to process the output of an
    external program, say cat, you could use backticks to launch it, or open
    a pipe to it. The former would return all the output, the latter would
    allow you to process the output of the external command line by line, as
    in:

    #! /usr/bin/perl

    use strict;
    use warnings;

    open my $ls, 'ls -l |'
    or die "Cannot pipe ls: $!";

    while(<$ls>) {
    next unless /\.pl$/;
    print "Perl file: $_";
    }
    __END__


    D:\Home\asu1\UseNet\clpmisc> myls
    Perl file: -rwxr-xr-x 1 asu1 None 919 Jan 16 11:33 050116-a.pl
    Perl file: -rwxr-xr-x 1 asu1 None 919 Jan 16 11:33 050116-b.pl
    Perl file: -rwxr-xr-x 1 asu1 None 691 Jan 17 12:56 050117-b.pl
    Perl file: -rwxr-xr-x 1 asu1 None 295 Jan 17 09:05 051117-a.pl

    I am just not sure what you constraints are, so maybe I am suggesting
    somehting that is not relevant to the particular problem you are trying
    to solve.

    Sinan

    --
    A. Sinan Unur <>
    (reverse each component and remove .invalid for email address)

    comp.lang.perl.misc guidelines on the WWW:
    http://mail.augustmail.com/~tadmc/clpmisc/clpmisc_guidelines.html
    A. Sinan Unur, Apr 8, 2005
    #4
  5. A. Sinan Unur wrote:
    > I understand. Now, the question is, are you in control of the system
    > call above? If you are, and you need to process the output of an
    > external program, say cat, you could use backticks to launch it, or open
    > a pipe to it. The former would return all the output, the latter would
    > allow you to process the output of the external command line by line, as
    > in:


    Well ... internally we have a LOT of old scripts. I'm hoping to solve
    the problem generically in a module (you could go back just a little
    ways in clpm to a thread called "output-monitoring module" to see what
    I'm trying to achieve) without having to modify the scripts. And of
    course I'd like to come out of it with a module I could contribute back
    too. That's why I'm trying to find an OS-level solution which happens to
    be implemented in Perl rather than a Perl solution.

    I could go a bit farther with your idea and override system(), replacing
    it with a sub that uses qx(). But I'm not sure how reliably I could
    capture every method of creating subprocesses that might be in use down
    deep in some script.

    --
    Henry Townsend
    Henry Townsend, Apr 8, 2005
    #5
  6. Henry Townsend <> wrote in
    news::

    > Well ... internally we have a LOT of old scripts. I'm hoping to solve
    > the problem generically in a module (you could go back just a little
    > ways in clpm to a thread called "output-monitoring module" to see what
    > I'm trying to achieve) without having to modify the scripts.


    ....

    > I could go a bit farther with your idea and override system(),
    > replacing it with a sub that uses qx(). But I'm not sure how reliably
    > I could capture every method of creating subprocesses that might be in
    > use down deep in some script.


    Well, I am probably missing something here, but this is what I would
    have done:

    #! /usr/bin/perl
    # harness.pl

    use strict;
    use warnings;

    my $oldscript = shift;
    $oldscript or die "Please supply the path to script to be run\n";

    open my $h, '-|', "perl $oldscript"
    or die "Cannot pipe $oldscript: $!";

    while(<$h>) {
    print "FILTERED: $_";
    }
    __END__

    #! /usr/bin/perl
    # tada.pl
    use strict;
    use warnings;

    my @commands = (q{cat test.txt}, q{grep use *.pl});

    for my $command (@commands) {
    print "Executing $cmd\n";
    system $cmd;
    }
    __END__

    D:\Home> harness tada.pl
    FILTERED: Executing cat test.txt
    FILTERED: lkdsjflkajdflkn
    FILTERED:
    FILTERED: safdsdflf;keww;dsf
    ....
    FILTERED: Executing grep use *.pl
    ....
    FILTERED: z.pl:use Archive::Zip ':ERROR_CODES';
    FILTERED: z.pl:use Fcntl ':seek';
    FILTERED: z.pl:use File::Temp 'tempfile';

    Does this help?

    Sinan

    --
    A. Sinan Unur <>
    (reverse each component and remove .invalid for email address)

    comp.lang.perl.misc guidelines on the WWW:
    http://mail.augustmail.com/~tadmc/clpmisc/clpmisc_guidelines.html
    A. Sinan Unur, Apr 8, 2005
    #6
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. samindla venkateswar rao
    Replies:
    0
    Views:
    357
    samindla venkateswar rao
    Nov 27, 2004
  2. Replies:
    1
    Views:
    247
  3. Ethan Furman

    numeric emulation and __pos__

    Ethan Furman, Jul 8, 2008, in forum: Python
    Replies:
    8
    Views:
    312
    Sion Arrowsmith
    Jul 9, 2008
  4. Eric Snow

    os.fork and pty.fork

    Eric Snow, Jan 8, 2009, in forum: Python
    Replies:
    0
    Views:
    563
    Eric Snow
    Jan 8, 2009
  5. Henry Townsend

    pipes in Windows perlfork emulation

    Henry Townsend, Apr 10, 2005, in forum: Perl Misc
    Replies:
    0
    Views:
    92
    Henry Townsend
    Apr 10, 2005
Loading...

Share This Page