Broken Pipe

[Jon Landenburer]

We have these large files which are compressed. I need to read the
first 50 lines and make various calcs. In sh I been doing
zcat dog.Z |head -n50 > dog.dat
In perl I been gettuing broken pipes. I get all of the data but I get
a message (stderr)
dog: Broken pipe

to isolate the probelm I wrote
@temp = `zcat $filename | head -n50|`;
print @temp;
print "wait\n"; $C= <STDIN>;

it will sit for about 30 seconds then give that message .
I will get similar results if I do

open (scnfile, "zcat dog.z|") or die...
while ($line = <scnfile>) {
if ($i++ > 50 ) {last;}
}
print "wait\n"; $C= <STDIN>;

same thing happens. a pause of 30 secs then the message dog:
broken pipe


Why and where I dont know.
Any ideas?




In perl I been did
open (scnfile, "zcat dog.Z |head -n50 |") or die ....
while ($line = <scnfile>){
.......
}
close file;
get next file ......

when I do this it completes the first file, shows the calce but then
displays a message (stderr)
dog: Broken pipe

so then I replaced the open ,read,close with a simpler
@lines = `zcat dog.Z |head -n50 `;
and still get the same thing.
Its a harmless thing but worrysome.
the zcat in sh gives no broken pipe

 

[Big and Blue]

In perl I been gettuing broken pipes. I get all of the data but I get
a message (stderr)
dog: Broken pipe

to isolate the probelm I wrote
@temp = `zcat $filename | head -n50|`;

So, after it has read 50 lines the head command exits.

If $filename is large the zcat command may still be pushing the contents
into the now deceased head, in which case it will be writing to a closed
pipe. Hence a SIGPIPE is sent.

Just put:

$SIG{PIPE} = _IGNORE_;

in your code (ideally scoped to the extent of the `` command).

 

[Big and Blue]

If $filename is large the zcat command may still be pushing the
contents into the now deceased head, in which case it will be writing to
a closed pipe. Hence a SIGPIPE is sent.

Just put:

$SIG{PIPE} = _IGNORE_;

in your code (ideally scoped to the extent of the `` command).

Err...could be wrong there. The "Broken Pipe" message is from the
shell running the `` command, so a perl SIG handler isn't going to help.

So, you'll need to trap the signal in the shell....

@temp = `trap "" 13; zcat $filename | head -n50`;

(although the comment made about the trailing | is also true - it shouldn't
be there: I'm assuming it isn't).

Anyway, try all of the suggestions in turn until the problem is gone.

(Another suggestion would be to install Compress::Zlib and just read
the first 50 lines entirely in Perl).

 

[Jon Landenburer]

Thanks.
As mentioned later the $SIG{PIPE} = _IGNORE_; would not work
and it did not work
but the trap "" 13 did
Thanks much.
I have'nt done anything with either SIG or the trapping of errs but
can see its utility

thanks again
JONL

 

[Christopher Mattern]

Err...could be wrong there. The "Broken Pipe" message is from the
shell running the `` command, so a perl SIG handler isn't going to help.

So, you'll need to trap the signal in the shell....

@temp = `trap "" 13; zcat $filename | head -n50`;

Nope, don't wanna do this. Once zcat gets the SIGPIPE, you *want*
it to terminate; you don't have any need for the rest of the file,
so why make zcat uncompress it all? This is actually a bit sticky.
You can get rid of the error message by throwing away stderr:

@temp = `zcat $filename 2>/dev/null | head -n50`;

but then you won't can't any *other* error messages, either...

(Another suggestion would be to install Compress::Zlib and just read
the first 50 lines entirely in Perl).

This is actually probably the best suggestion. More efficient and
elegant than anything else I can think of .

Chris mattern