ISO: What is the "best practice" for getting error info from a pipeline?

[Larry W. Virden]

Background:

On a scale of 0 to 10, where a person at 0 says "what's a programming
language" and a 10 is Larry Wall, I would rate myself a 2 or maybe a 3
in relationship to perl.

I provide maintenance support on a huge number of programs, written in
a dozen or so languages. I don't do much original programming, so
there is little practice.

In the thousands of files that I support is a rather largish (in the
200+k lines of code range) system that deals with installation
metadata (where does item version 1.2 get installed, with what
permissions, what mode, etc. - that kind of metadata).

Within this code is some perl code that does basically this sort of
thing:

# Email request
my $send = IO:ipe->new();
my $arch = `/bin/arch`;
chomp $arch;
$send->writer("/path/bin/$arch/psend", 'REQUEST');
my $a = $self->{'queue'};
for (my $indx = 0; $indx < @{$a}; $indx++) {
my $r = $a->[$indx];
$send->print(":$r->[0]");
for (my $args = 1; $args < @{$r}; $args++) {
(my $arg = $r->[$args]) =~ s!([\\"])!\\$1!g;
$send->print(',"', $arg, '"');
}
$send->print("\n");
}
if ($send->close()) {
my $msg = @{$a};
delete $self->{'queue'};
$self->{'queue'} = [ $sys ];
return "$msg steps submitted";
}
$self->error('error: psend failed');
return undef;

The problem is that psend turns out to possibly exit with a non-zero
exit code if it has problems, and this calling program doesn't appear
to notice the fact that psend has failed.

So I have been asked to update it to :

a. recognize that the program at the end of the IO:ipe has failed
and to exit with an error msg
b. pass any stderr messages back as part of the psend failed error
message, so that the user has some chance of fixing the problem.

In reading the IO:ipe and IO::Handle docs, I don't see a lot of
detail about handling the remote program's error "exits".

Does anyone have suggestions on what needs to happen? For instance,
the info in IO::Handle mentions $self->error, but says that it is a
boolean indicator; that doesn't seem like it is going to help me with
accessing the error messages theirselves. I suppose the specific exit
code won't be as big a deal as long as the error messages are unique.

Anyways, I'd love any pointers, etc. that you might have. I did google
for some things and tried to solve this, but failed to turn up
anything that looked remotely useful.

 

[Larry W. Virden]

IO::Handle::close calls close() on the pipe and returns the value
returned by close(). From 'perldoc close':

"If the file handle came from a piped open, "close" will addi-
tionally return false if one of the other system calls involved
fails, or if the program exits with non-zero status. (If the
only problem was that the program exited non-zero, $! will be
set to 0.) Closing a pipe also waits for the process executing
on the pipe to complete, in case you want to look at the output
of the pipe afterwards, and implicitly puts the exit status
value of that command into $?."

So this program is already checking for the error return from close().
There may not be much more that you can do except check the values of
$! and $? if close returns an error.

More detail - and confusion.

Our perl is 5.8.4

Here's a smaller, hopefully more coherent example:

file 1
$ cat /tmp/displayerr.ksh
#! /bin/ksh

echo "This is stderr" >&2
echo "This is stdout"
cat
exit 3
$ cat tstio.pl
#! /bin/perl -w

use strict;
use IO:ipe;

my $send = IO:ipe->new();
$send->writer("/tmp/displayerr.ksh", 'REQUEST');
$send->print("This is just some output\n");
print "error 1 returns " . $send->error() . "\n";
if ($send->error()) {
print "io error\n";
}
print "close returns " . $send->close() . "\n";
print "error 2 returns " . $send->error() . "\n";


And here's the peculiar thing:
$ ~/tstio.pl
error 1 returns 0
This is stderr
This is stdout
This is just some output
close returns 1
error 2 returns -1

$

This is the same output as I get if the ksh script exits with a 0 . So
I see no indication that the piped command has exited with a non-zero
return code.

It seems like the error() method isn't working as expected - or I am
totally misunderstanding its purpose.

 

[xhoster]

IO::Handle::close calls close() on the pipe and returns the value
returned by close(). From 'perldoc close':

"If the file handle came from a piped open, "close" will addi-
tionally return false if one of the other system calls involved
fails, or if the program exits with non-zero status.

But the handle here is *not* derived from a pipe open. It is derived
from IO:ipe->new() . So that part of the docs do not apply.

Xho

 

[xhoster]

# Email request
my $send = IO:ipe->new();
my $arch = `/bin/arch`;
chomp $arch;
$send->writer("/path/bin/$arch/psend", 'REQUEST');
my $a = $self->{'queue'};
for (my $indx = 0; $indx < @{$a}; $indx++) {
my $r = $a->[$indx];
$send->print(":$r->[0]");
for (my $args = 1; $args < @{$r}; $args++) {
(my $arg = $r->[$args]) =~ s!([\\"])!\\$1!g;
$send->print(',"', $arg, '"');
}
$send->print("\n");
}
if ($send->close()) {
my $msg = @{$a};
delete $self->{'queue'};
$self->{'queue'} = [ $sys ];
return "$msg steps submitted";
}
$self->error('error: psend failed');
return undef;

The problem is that psend turns out to possibly exit with a non-zero
exit code if it has problems, and this calling program doesn't appear
to notice the fact that psend has failed.

So I have been asked to update it to :

a. recognize that the program at the end of the IO:ipe has failed
and to exit with an error msg

You might be able to hack IO:ipe to do this, but it would be better
to use something else altogether.

b. pass any stderr messages back as part of the psend failed error
message, so that the user has some chance of fixing the problem.

Normally what psend prints to stderr will end up on Perl's STDERR, so
if you log that you will have psend's error in your log file. If you want
better granularity and control then that, you could use IPC::Open3 or
IPC::Run

In reading the IO:ipe and IO::Handle docs, I don't see a lot of
detail about handling the remote program's error "exits".

Does anyone have suggestions on what needs to happen? For instance,
the info in IO::Handle mentions $self->error, but says that it is a
boolean indicator; that doesn't seem like it is going to help me with
accessing the error messages theirselves. I suppose the specific exit
code won't be as big a deal as long as the error messages are unique.

IO:ipe and IO::Handle are very general tools. They can't be all
things to all people. You want something specifically from running
external commands.

Xho

 

[xhoster]

IO::Handle::close calls close() on the pipe and returns the value
returned by close(). From 'perldoc close':

"If the file handle came from a piped open, "close" will addi-
tionally return false if one of the other system calls involved
fails, or if the program exits with non-zero status.

But the handle here is *not* derived from a pipe open. It is derived
from IO:ipe->new() . So that part of the docs do not apply.

So you don't believe the documentation of IO::Handle that says:

"METHODS

See perlfunc for complete descriptions of each of the following sup-
ported "IO::Handle" methods, which are just front ends for the corre-
sponding built-in functions:

$io->close
..."[/QUOTE]

Yep, I believe it. And once you see the docs for close in perlfunc,
what you have still isn't from a piped open and thus the part of the doc
that says "If the file handle came from a piped open" still doesn't
apply.

But don't take me word for it, feel free to test it. (IO:ipe version
1.123)

$ perl -lwe 'use IO:ipe; my $pipe=IO:ipe->new(); \
$pipe->writer("cat -asdf >& cat"); print $pipe "lkj"; \
print close $pipe; warn "$?";'
1
0 at -e line 1.

So the one means the close succeeded, the $? is the exit code and
also allegedly indicates success. But if you look in "cat", you will
see it doesn't contain "lkj" but rather has complaints about bad options.


$ perl -lwe 'open my $pipe, "|cat -asdf >& cat"; print $pipe "lkj"; \
print close $pipe; warn "$?";'

256 at -e line 1.

So the empty line means the close failed, and $? was set appropriately.


Xho

 

[xhoster]

use strict;
use IO:ipe;

my $send = IO:ipe->new();
$send->writer("/tmp/displayerr.ksh", 'REQUEST');
$send->print("This is just some output\n");
print "error 1 returns " . $send->error() . "\n";
if ($send->error()) {
print "io error\n";
}
print "close returns " . $send->close() . "\n";

print "but \$? was $? (exit status " . ($?>>8). ")\n";

print "error 2 returns " . $send->error() . "\n";

Even thought the close is allegedly successful, it still
sets $?, as long as you do '$send->close()' rather than
"close $send". However, I would consider this behavior to
be quite fragile and would be reluctant to depend on it except
as a last resort.

Xho