weird behavior with open and pipe

[Moritz Karbach]

Hi,

I'm still tuning the run()-function of my Command class (see some older
postings of mine, if you like).

I'm using

open($fh_child,"-|");

to fork and to open a pipe to the child, which then exec's a system command.
The parent uses

my @output = <$fh_child>;
close($fh_child);

to receive the output. Here is the problem:

If the childs executes a certain system command and the system command gets
killed externally (by a watchdog), sometimes the parent doesn't receive any
output! Yes, I'm sure, that the system command has written something to std
out before it got killed.

The weird thing is, that if I execute something like

<test.sh>
#!/bin/bash
echo "this is a test"
sleep 10
</test.sh>

and if I kill the process after say 5 seconds, the parent indeed receives
the line "this is a test"!

Obviously it must be some difference in the binaries/system commands. Any
idea, what could be the reason for this difference? Is there a possible
workaround?

I already tried

while ( my $line = <$fh_child> )
{
push ( @{$this->{output}}, $line );
#print "X " . $line;
}

without success...

Cheers,

- Moritz

 

[Anno Siegel]

Hi,

I'm still tuning the run()-function of my Command class (see some older
postings of mine, if you like).

I'm using

open($fh_child,"-|");

to fork and to open a pipe to the child, which then exec's a system command.
The parent uses

my @output = <$fh_child>;
close($fh_child);

to receive the output. Here is the problem:

If the childs executes a certain system command and the system command gets
killed externally (by a watchdog), sometimes the parent doesn't receive any
output! Yes, I'm sure, that the system command has written something to std
out before it got killed.

Buffering. The output is still in a buffer when the program gets killed,
so the data is lost. At least, that's consistent with what you're
seeing.

The weird thing is, that if I execute something like

<test.sh>
#!/bin/bash
echo "this is a test"
sleep 10
</test.sh>

and if I kill the process after say 5 seconds, the parent indeed receives
the line "this is a test"!

A shell script is a bad approximation for a single process, it runs
as a series of sub-processes, held together by the shell. You probably
killed the shell, which is neither the process that echoed, nor the
one that sleeps. A perl program would be a better approximation (store
in /tmp/prog):

#!/usr/local/bin/perl
use strict; use warnings;
$| = 1; # auto-flush -- comment this out

print "this is a test\n";
sleep 5;

This runs all in a single process. The parent program

#!/usr/local/bin/perl
use strict; use warnings; $| = 1;

my $pid_child = open my $fh_child, '-|', '/tmp/prog' or
die "pipe open: $!";
sleep 2;
kill TERM => $pid_child or die;
print while <$fh_child>;

still sees the output, but if you comment-out "$| = 1" and run the kid
buffered it doesn't.

Anno

 

[Moritz Karbach]

Hi Anno,

Buffering. The output is still in a buffer when the program gets killed,
so the data is lost. At least, that's consistent with what you're
seeing.

Well the thing is, whem I'm running the executable directly on the shell, it
immediately prints out some lines, then starts "hard working" without any
output.

I'm killing it during the "hard working" phase (in fact the watchdog of my
Command class kills the whole process group, so any of the subprocesses
launched by my shell fake binary gets killed as well).

If it was buffering (steered from within the binary), I don't understand why
I see the output lines on the shell directly, but in the Perl program I
don't.

Cheers,

- Moritz

 

[Anno Siegel]

Hi Anno,


Well the thing is, whem I'm running the executable directly on the shell, it
immediately prints out some lines, then starts "hard working" without any
output.

I'm killing it during the "hard working" phase (in fact the watchdog of my
Command class kills the whole process group, so any of the subprocesses
launched by my shell fake binary gets killed as well).

If it was buffering (steered from within the binary), I don't understand why
I see the output lines on the shell directly, but in the Perl program I
don't.

You overall description certainly made it look like a buffering problem.
If you think it's something else, give us code to check.

Anno

 

[Moritz Karbach]

Hi Christian (and the others),

Your binary may behave differently when printing to a terminal
and to a pipe.

ok, I didn't know that something like that is possible... So it is unlikely,
that I see anything from the command if it gets killed.

Or are there some standard methods to turn output buffering off? Some
environment variable, for instance? Maybe I can simulate a terminal within
the child before it exec's the command?

Thanks for your help,

- Moritz

 

[Brian McCauley]

Hi Christian (and the others),


ok, I didn't know that something like that is possible... So it is unlikely,
that I see anything from the command if it gets killed.

Or are there some standard methods to turn output buffering off? Some
environment variable, for instance?

Not that I know of.

Maybe I can simulate a terminal within
the child before it exec's the command?

I _Expect_ there's something on CPAN.