IPC::Open3 : Why can't I catch program output here?

Discussion in 'Perl Misc' started by Ronny, May 5, 2008.

  1. Ronny

    Ronny Guest

    I try to use the program below to catch standard output and standard
    error
    of a process. The program itself runs well, and it *does* write at
    least to
    standard output, but in the code below, the line saying "### READING"
    is never printed and @msg is still empty afterwards. This means that
    $selector->can_read() must have returned false already the first time
    it was called. Any suggestion on what could I have done wrong here?

    my @msg;
    my $pid=open3(*CMD_IN,*CMD_OUT,*CMD_ERR,@cmd)
    or confess "Open3 error: $!\n";;
    $SIG{CHLD} = sub {
    waitpid($pid,0)>0 &&
    print "SIGCHILD status=$? pid=$pid\n";
    };
    my $selector=IO::Select->new();
    $selector->add(*CMD_ERR,*CMD_OUT);
    while(my @ready=$selector->can_read()) {
    foreach my $fh (@ready) {
    print "### READING\n";
    push @msg,scalar(<$fh>);
    $selector->remove($fh) if eof($fh);
    }
    }
    close CMD_OUT;
    close CMD_ERR;
    Ronny, May 5, 2008
    #1
    1. Advertising

  2. Ronny

    Ben Morrow Guest

    Quoth Ronny <>:
    > I try to use the program below to catch standard output and standard
    > error of a process. The program itself runs well, and it *does* write
    > at least to standard output, but in the code below, the line saying
    > "### READING" is never printed and @msg is still empty afterwards.
    > This means that $selector->can_read() must have returned false already
    > the first time it was called. Any suggestion on what could I have done
    > wrong here?


    Works for me here (i686-freebsd). What platform are you on, and what
    command are you trying to run? In general, if you can provide a complete
    program people can copy/paste/run, you're more likely to get help: in
    this case, you omitted the 'use' lines and the contents of @cmd.

    > my @msg;
    > my $pid=open3(*CMD_IN,*CMD_OUT,*CMD_ERR,@cmd)
    > or confess "Open3 error: $!\n";;


    I would normally say 'Use lexical filehandles!' at this point;
    unfortunately, IPC::Open3 was written before they existed and the
    obvious way

    my $pid = open3(my $CMD_IN, my $CMD_OUT, my $CMD_ERR, @cmd)...

    doesn't work (and can't be made to since undef is already meaningful).
    So you have to fall back to the old method of using Symbol::gensym:

    use Symbol qw/gensym/;

    my ($CMD_IN, $CMD_OUT, $CMD_ERR) = (gensym, gensym, gensym);
    my $pid = open3($CMD_IN, $CMD_OUT, $CMD_ERR, @cmd)...

    which is definitely ugly, but IMHO worth it (if you must use IPC::Open3
    at all: I'd use IPC::Run instead). For one thing, these gensymmed
    filehandles close when they go out of scope, just like more usual
    lexical filehandles. Of course, I'd use a hash to hold the filehandles:

    my %CMD = map {($_, gensym)} qw/IN OUT ERR/;
    my $pid = open3(@CMD{qw/IN OUT ERR/}, @cmd)...

    > $SIG{CHLD} = sub {
    > waitpid($pid,0)>0 &&
    > print "SIGCHILD status=$? pid=$pid\n";
    > };
    > my $selector=IO::Select->new();
    > $selector->add(*CMD_ERR,*CMD_OUT);
    > while(my @ready=$selector->can_read()) {
    > foreach my $fh (@ready) {
    > print "### READING\n";
    > push @msg,scalar(<$fh>);
    > $selector->remove($fh) if eof($fh);


    Using eof is nearly always a mistake: it's almost always better to check
    the return value of your read statement instead. Something like

    my $msg = <$fh>;
    defined $msg
    ? push @msg, $msg
    : $selector->remove($fh);

    This also means you don't get a spurious undef on the end of @msg.

    Ben

    --
    I must not fear. Fear is the mind-killer. I will face my fear and
    I will let it pass through me. When the fear is gone there will be
    nothing. Only I will remain.
    Frank Herbert, 'Dune'
    Ben Morrow, May 5, 2008
    #2
    1. Advertising

  3. [A complimentary Cc of this posting was NOT [per weedlist] sent to
    Ben Morrow
    <>], who wrote in article <>:
    > I would normally say 'Use lexical filehandles!' at this point;
    > unfortunately, IPC::Open3 was written before they existed and the
    > obvious way
    >
    > my $pid = open3(my $CMD_IN, my $CMD_OUT, my $CMD_ERR, @cmd)...
    >
    > doesn't work (and can't be made to since undef is already meaningful).


    I have no idea what you mean here.

    sub open3 ($$$$@) {
    $_[0] = IO::Handle->new unless defined $_[0]; # Or whatever is THE initializer
    ...
    }

    Yours,
    Ilya
    Ilya Zakharevich, May 6, 2008
    #3
  4. Ronny

    Ben Morrow Guest

    Quoth Ilya Zakharevich <>:
    > [A complimentary Cc of this posting was NOT [per weedlist] sent to
    > Ben Morrow
    > <>], who wrote in article
    > <>:
    > > I would normally say 'Use lexical filehandles!' at this point;
    > > unfortunately, IPC::Open3 was written before they existed and the
    > > obvious way
    > >
    > > my $pid = open3(my $CMD_IN, my $CMD_OUT, my $CMD_ERR, @cmd)...
    > >
    > > doesn't work (and can't be made to since undef is already meaningful).

    >
    > I have no idea what you mean here.
    >
    > sub open3 ($$$$@) {
    > $_[0] = IO::Handle->new unless defined $_[0]; # Or whatever is THE
    > initializer
    > ...
    > }


    Yeah, in general you can do that; however, for the specific case of
    open3, the docs say

    If CHLD_ERR is false, or the same file descriptor as CHLD_OUT, then
    STDOUT and STDERR of the child are on the same filehandle.

    so at least the CHLD_ERR filehandle can't be autovivified. Presumably it
    is felt that this behaviour can't be changed.

    Ben

    --
    "If a book is worth reading when you are six, *
    it is worth reading when you are sixty." [C.S.Lewis]
    Ben Morrow, May 6, 2008
    #4
  5. Ben Morrow <> wrote:
    > I would normally say 'Use lexical filehandles!' at this point;
    > unfortunately, IPC::Open3 was written before they existed and the
    > obvious way
    >
    > my $pid = open3(my $CMD_IN, my $CMD_OUT, my $CMD_ERR, @cmd)...
    >
    > doesn't work (and can't be made to since undef is already meaningful).


    What's wrong with the above?

    The only issue I see is that Open3 will not autogenerate a filehandle
    for stderr (instead it will combine stdout and stderr and place them
    both on the $CMD_OUT filehandle). I actually use that method so I don't
    have to select between two filehandles and can just read one at a time.

    # capture STDOUT and STDERR, on a single filehandle
    use IPC::Open3;
    open3(my $cmd_in, my $cmd_out, undef, @cmd);
    while (<$cmd_out>)
    {
    ...
    }

    --
    Darren Dunham
    Senior Technical Consultant TAOS http://www.taos.com/
    Got some Dr Pepper? San Francisco, CA bay area
    < This line left intentionally blank to confuse you. >
    Darren Dunham, May 6, 2008
    #5
  6. Ronny

    Guest

    (Darren Dunham) wrote:
    > Ben Morrow <> wrote:
    > > I would normally say 'Use lexical filehandles!' at this point;
    > > unfortunately, IPC::Open3 was written before they existed and the
    > > obvious way
    > >
    > > my $pid = open3(my $CMD_IN, my $CMD_OUT, my $CMD_ERR, @cmd)...
    > >
    > > doesn't work (and can't be made to since undef is already meaningful).

    >
    > What's wrong with the above?


    It causes cmd's stderr to go to $CMD_OUT rather than the obviously
    intended $CMD_ERR.

    >
    > The only issue I see is that Open3 will not autogenerate a filehandle
    > for stderr (instead it will combine stdout and stderr and place them
    > both on the $CMD_OUT filehandle).


    And this is obviously a problem if that is not what you want to happen.

    Xho

    --
    -------------------- http://NewsReader.Com/ --------------------
    The costs of publication of this article were defrayed in part by the
    payment of page charges. This article must therefore be hereby marked
    advertisement in accordance with 18 U.S.C. Section 1734 solely to indicate
    this fact.
    , May 6, 2008
    #6
  7. wrote:
    > (Darren Dunham) wrote:
    >> Ben Morrow <> wrote:
    >> > I would normally say 'Use lexical filehandles!' at this point;
    >> > unfortunately, IPC::Open3 was written before they existed and the
    >> > obvious way
    >> >
    >> > my $pid = open3(my $CMD_IN, my $CMD_OUT, my $CMD_ERR, @cmd)...
    >> >
    >> > doesn't work (and can't be made to since undef is already meaningful).

    >>
    >> What's wrong with the above?

    >
    > It causes cmd's stderr to go to $CMD_OUT rather than the obviously
    > intended $CMD_ERR.


    I don't know that it's obviously intended. It took me quite a while to
    realize that I even had an option to return output and error on a single
    filehandle. I mention that explicitly because the OP might have been
    having issues with selecting between the two, when reading them combined
    may have been preferable.

    However, if you do want them separated, then the other choices offered
    seem quite verbose. Instead of them, I would pass in a (defined)
    filehandle. That takes one more line of code (two if you count bringing
    in the FileHandle module.

    my $CMD_ERR = FileHandle->new();
    my $pid = open3(my $CMD_IN, my $CMD_OUT, $CMD_ERR, @cmd)...

    --
    Darren Dunham
    Senior Technical Consultant TAOS http://www.taos.com/
    Got some Dr Pepper? San Francisco, CA bay area
    < This line left intentionally blank to confuse you. >
    Darren Dunham, May 6, 2008
    #7
  8. Ronny

    Ben Morrow Guest

    Quoth (Darren Dunham):
    > wrote:
    > > (Darren Dunham) wrote:
    > >> Ben Morrow <> wrote:
    > >> > I would normally say 'Use lexical filehandles!' at this point;
    > >> > unfortunately, IPC::Open3 was written before they existed and the
    > >> > obvious way
    > >> >
    > >> > my $pid = open3(my $CMD_IN, my $CMD_OUT, my $CMD_ERR, @cmd)...
    > >> >
    > >> > doesn't work (and can't be made to since undef is already meaningful).
    > >>
    > >> What's wrong with the above?

    > >
    > > It causes cmd's stderr to go to $CMD_OUT rather than the obviously
    > > intended $CMD_ERR.

    >
    > I don't know that it's obviously intended. It took me quite a while to
    > realize that I even had an option to return output and error on a single
    > filehandle. I mention that explicitly because the OP might have been
    > having issues with selecting between the two, when reading them combined
    > may have been preferable.
    >
    > However, if you do want them separated, then the other choices offered
    > seem quite verbose. Instead of them, I would pass in a (defined)
    > filehandle. That takes one more line of code (two if you count bringing
    > in the FileHandle module.
    >
    > my $CMD_ERR = FileHandle->new();
    > my $pid = open3(my $CMD_IN, my $CMD_OUT, $CMD_ERR, @cmd)...


    ....which is essentially the same as what I proposed, except that I
    prefer to avoid FileHandle (and IO::Handle) and use Symbol::gensym
    directly. That's just a matter of taste, of course.

    Ben
    Ben Morrow, May 6, 2008
    #8
  9. [A complimentary Cc of this posting was NOT [per weedlist] sent to
    Ben Morrow
    <>], who wrote in article <>:
    > > sub open3 ($$$$@) {
    > > $_[0] = IO::Handle->new unless defined $_[0]; # Or whatever is THE
    > > initializer
    > > ...
    > > }


    > Yeah, in general you can do that; however, for the specific case of
    > open3, the docs say
    >
    > If CHLD_ERR is false, or the same file descriptor as CHLD_OUT, then
    > STDOUT and STDERR of the child are on the same filehandle.


    Thanks, I forgot about this semantic. Which does not mean that one
    could not write ALSO open3_autovivify... ;-)

    Yours,
    Ilya
    Ilya Zakharevich, May 7, 2008
    #9
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Dave Newberry
    Replies:
    0
    Views:
    641
    Dave Newberry
    Oct 17, 2004
  2. Mr. SweatyFinger
    Replies:
    2
    Views:
    1,769
    Smokey Grindel
    Dec 2, 2006
  3. J. Romano
    Replies:
    0
    Views:
    150
    J. Romano
    Feb 10, 2004
  4. Mike Hunter

    Scary IPC::Open3 filehandle difference

    Mike Hunter, Oct 28, 2004, in forum: Perl Misc
    Replies:
    1
    Views:
    103
    Ilmari Karonen
    Dec 14, 2004
  5. Rocky Allen

    IPC::Open3

    Rocky Allen, Aug 24, 2005, in forum: Perl Misc
    Replies:
    4
    Views:
    109
    Joe Smith
    Aug 24, 2005
Loading...

Share This Page