Broken Pipe

Discussion in 'Perl Misc' started by Jon Landenburer, Feb 11, 2004.

  1. We have these large files which are compressed. I need to read the
    first 50 lines and make various calcs. In sh I been doing
    zcat dog.Z |head -n50 > dog.dat
    In perl I been gettuing broken pipes. I get all of the data but I get
    a message (stderr)
    dog: Broken pipe

    to isolate the probelm I wrote
    @temp = `zcat $filename | head -n50|`;
    print @temp;
    print "wait\n"; $C= <STDIN>;

    it will sit for about 30 seconds then give that message .
    I will get similar results if I do

    open (scnfile, "zcat dog.z|") or die...
    while ($line = <scnfile>) {
    if ($i++ > 50 ) {last;}
    }
    print "wait\n"; $C= <STDIN>;

    same thing happens. a pause of 30 secs then the message dog:
    broken pipe


    Why and where I dont know.
    Any ideas?




    In perl I been did
    open (scnfile, "zcat dog.Z |head -n50 |") or die ....
    while ($line = <scnfile>){
    .......
    }
    close file;
    get next file ......

    when I do this it completes the first file, shows the calce but then
    displays a message (stderr)
    dog: Broken pipe

    so then I replaced the open ,read,close with a simpler
    @lines = `zcat dog.Z |head -n50 `;
    and still get the same thing.
    Its a harmless thing but worrysome.
    the zcat in sh gives no broken pipe
    Jon Landenburer, Feb 11, 2004
    #1
    1. Advertising

  2. Jon Landenburer

    Big and Blue Guest

    Jon Landenburer wrote:
    >
    > In perl I been gettuing broken pipes. I get all of the data but I get
    > a message (stderr)
    > dog: Broken pipe
    >
    > to isolate the probelm I wrote
    > @temp = `zcat $filename | head -n50|`;


    So, after it has read 50 lines the head command exits.

    If $filename is large the zcat command may still be pushing the contents
    into the now deceased head, in which case it will be writing to a closed
    pipe. Hence a SIGPIPE is sent.

    Just put:

    $SIG{PIPE} = _IGNORE_;

    in your code (ideally scoped to the extent of the `` command).

    --
    -*- Just because I've written it here doesn't -*-
    -*- mean that you should, or I do, believe it. -*-
    Big and Blue, Feb 12, 2004
    #2
    1. Advertising

  3. Jon Landenburer

    Big and Blue Guest

    Big and Blue wrote:
    >
    > If $filename is large the zcat command may still be pushing the
    > contents into the now deceased head, in which case it will be writing to
    > a closed pipe. Hence a SIGPIPE is sent.
    >
    > Just put:
    >
    > $SIG{PIPE} = _IGNORE_;
    >
    > in your code (ideally scoped to the extent of the `` command).


    Err...could be wrong there. The "Broken Pipe" message is from the
    shell running the `` command, so a perl SIG handler isn't going to help.

    So, you'll need to trap the signal in the shell....

    @temp = `trap "" 13; zcat $filename | head -n50`;

    (although the comment made about the trailing | is also true - it shouldn't
    be there: I'm assuming it isn't).

    Anyway, try all of the suggestions in turn until the problem is gone.

    (Another suggestion would be to install Compress::Zlib and just read
    the first 50 lines entirely in Perl).

    --
    -*- Just because I've written it here doesn't -*-
    -*- mean that you should, or I do, believe it. -*-
    Big and Blue, Feb 12, 2004
    #3
  4. Thanks.
    As mentioned later the $SIG{PIPE} = _IGNORE_; would not work
    and it did not work
    but the trap "" 13 did
    Thanks much.
    I have'nt done anything with either SIG or the trapping of errs but
    can see its utility

    thanks again
    JONL
    Jon Landenburer, Feb 13, 2004
    #4
  5. Big and Blue wrote:

    > Big and Blue wrote:
    > >
    >> If $filename is large the zcat command may still be pushing the
    >> contents into the now deceased head, in which case it will be writing to
    >> a closed pipe. Hence a SIGPIPE is sent.
    >>
    >> Just put:
    >>
    >> $SIG{PIPE} = _IGNORE_;
    >>
    >> in your code (ideally scoped to the extent of the `` command).

    >
    > Err...could be wrong there. The "Broken Pipe" message is from the
    > shell running the `` command, so a perl SIG handler isn't going to help.
    >
    > So, you'll need to trap the signal in the shell....
    >
    > @temp = `trap "" 13; zcat $filename | head -n50`;


    Nope, don't wanna do this. Once zcat gets the SIGPIPE, you *want*
    it to terminate; you don't have any need for the rest of the file,
    so why make zcat uncompress it all? This is actually a bit sticky.
    You can get rid of the error message by throwing away stderr:

    @temp = `zcat $filename 2>/dev/null | head -n50`;

    but then you won't can't any *other* error messages, either...

    > (Another suggestion would be to install Compress::Zlib and just read
    > the first 50 lines entirely in Perl).
    >

    This is actually probably the best suggestion. More efficient and
    elegant than anything else I can think of .

    Chris mattern
    Christopher Mattern, Feb 13, 2004
    #5
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. funtoosh

    how to detect broken pipe ?

    funtoosh, Dec 20, 2003, in forum: Perl
    Replies:
    1
    Views:
    901
    Ben Morrow
    Dec 21, 2003
  2. Clive
    Replies:
    0
    Views:
    2,992
    Clive
    Aug 12, 2003
  3. Tomas Penc
    Replies:
    3
    Views:
    35,891
  4. lee, wonsun
    Replies:
    1
    Views:
    478
    Jack Klein
    Nov 2, 2004
  5. Replies:
    1
    Views:
    215
    Ben Morrow
    Jun 2, 2004
Loading...

Share This Page