Q on localizing *STDOUT and fork

[kj]

Let me preface this question by making it clear that there's no
particular problem I'm trying to solve, but rather I'm trying to
clarify my understanding of how Perl works, at least under Unix.

In the following snippet, the key fragment is the block labeled
LOOK_HERE. There I first "save" STDOUT by duplicating the handle;
then I redirect STDOUT; then I execute a (forked) command that has
the effect of sending some output to (the now redirected) STDOUT;
and finally I "restore" STDOUT with another duplication.

my $file = 'somefile';
open my $out, '>', $file or die $!;

LOOK_HERE:
{
open my $save, '>&', STDOUT or die $!;
open STDOUT, '>&', $out or die $!;

{
open my $pipe, '|-', '/usr/bin/sort', '-n' or die $!;
print $pipe int( rand( ~0 ) ), "\n" for 1..1_000_000;
}

open STDOUT, '>&', $save or die $!;
}

print "OK\n";
close $out;
my @sorted = map { chomp; $_ } File::Slurp::read_file( $file );

This all works: the output of /usr/bin/sort does end up in 'somefile',
and "OK\n" gets printed to the terminal, confirming that STDOUT
was properly restored.

So far so good.

My question is this: is there a way to avoid the bothersome saving
and restoring of STDOUT. I naively thought that one could do so
by localizing *STDOUT. IOW, replace the LOOK_HERE block with:

{
local *STDOUT;
open STDOUT, '>&', $out or die $!;

{
open my $pipe, '|-', '/usr/bin/sort', '-n' or die $!;
print $pipe int( rand( ~0 ) ), "\n" for 1..1_000_000;
}
}

Very nice, except it doesn't work. Now the output /usr/bin/sort
(which, incidentally, in this example happens to be pretty big)
goes to the terminal. BTW, this same thing happens if instead of
redirecting STDOUT by duplicating the write-handle $out, I simply
re-open STDOUT like this:

{
local *STDOUT;
open STDOUT, '>', 'somefile' or die $!;

{
open my $pipe, '|-', '/usr/bin/sort', '-n' or die $!;
print $pipe int( rand( ~0 ) ), "\n" for 1..1_000_000;
}
}

Ditto if instead I select $out before execution of the sort command.

I suspect that the problem with these failed solutions has to do
with the implicit fork triggered by the '|-' mode in the call to
open. I.e., I'm guessing that the child process uses the default
STDOUT irrespective of the parent's maneuvers. But if this is the
case, then my confusion simply shifts to wondering how the first
approach could have worked at all!

Anyway, BTAIM, is there anyway to avoid the save/restore rigmarole?

TIA!

kj

 

[Brian McCauley]

Let me preface this question by making it clear that there's no
particular problem I'm trying to solve, but rather I'm trying to
clarify my understanding of how Perl works, at least under Unix.

My question is this: is there a way to avoid the bothersome saving
and restoring of STDOUT. I naively thought that one could do so
by localizing *STDOUT. IOW, replace the LOOK_HERE block with:

{
local *STDOUT;
open STDOUT, '>&', $out or die $!;

{
open my $pipe, '|-', '/usr/bin/sort', '-n' or die $!;
print $pipe int( rand( ~0 ) ), "\n" for 1..1_000_000;
}
}

Very nice, except it doesn't work. Now the output /usr/bin/sort
(which, incidentally, in this example happens to be pretty big)
goes to the terminal. BTW, this same thing happens if instead of
redirecting STDOUT by duplicating the write-handle $out, I simply
re-open STDOUT like this:

{
local *STDOUT;
open STDOUT, '>', 'somefile' or die $!;

{
open my $pipe, '|-', '/usr/bin/sort', '-n' or die $!;
print $pipe int( rand( ~0 ) ), "\n" for 1..1_000_000;
}
}

I wrote a quite detailed explanation of this here...

http://groups.google.com/group/comp..._frm/thread/6d5c3d062c1e7608/fb1af6de388ee945

Anyway, BTAIM, is there anyway to avoid the save/restore rigmarole?

I'm fairly sure I've seen modules on CPAN to wrap it up a bit but
under the hood AFAIK they'd still do the same thing.

 

[xhoster]

My question is this: is there a way to avoid the bothersome saving
and restoring of STDOUT. I naively thought that one could do so
by localizing *STDOUT. IOW, replace the LOOK_HERE block with:

{
local *STDOUT;
open STDOUT, '>&', $out or die $!;

{
open my $pipe, '|-', '/usr/bin/sort', '-n' or die $!;
print $pipe int( rand( ~0 ) ), "\n" for 1..1_000_000;
}
}

Very nice, except it doesn't work. Now the output /usr/bin/sort
(which, incidentally, in this example happens to be pretty big)
goes to the terminal.

The piped "sort" doesn't inherit the Perl notion of STDOUT, it inherits the
C's notion of stdout. When Perl starts, Perl's STDOUT "points" to C's
stdout. When you localize STDOUT, you break that linkage, and so whatever
is done to Perl's STDOUT doesn't affect C's stdout.

I suspect that the problem with these failed solutions has to do
with the implicit fork triggered by the '|-' mode in the call to
open. I.e., I'm guessing that the child process uses the default
STDOUT irrespective of the parent's maneuvers. But if this is the
case, then my confusion simply shifts to wondering how the first
approach could have worked at all!

When you reopen STDOUT without localizing it, it does this by reopening
C's notion of stdout and leaving Perl's STDOUT pointing to C's stdout.
Forked commands inherit this C notion, complete with the change made to it.

Anyway, BTAIM, is there anyway to avoid the save/restore rigmarole?

You could do the redirect in the open command:

open my $pipe, '|-', "/usr/bin/sort -n > $filename", or die $!;

Alas, this requires you to use the shell-interpreted-version of the open
rather than the shell-less version.

Xho

 

[kj]

I wrote a quite detailed explanation of this here...

http://groups.google.com/group/comp..._frm/thread/6d5c3d062c1e7608/fb1af6de388ee945

That was certainly illuminating. Thank you!

But if you do local(*STDOUT) you stash away the current contents of
*STDOUT{IO} and make it empty. Now when you open(STDOUT,...) you get a
new IO-thingy associated with *STDOUT{IO} but this IO-thingy is not
associated with FD 1. As far as the current Perl process is concerned
this is now the "standard output" but any child process that's created
will still think of FD 1 as standard out.

Inspired by this I replaced

local *STDOUT;

with

local *STDOUT = *STDOUT;

The subsequent redirection of STDOUT now worked as desired, but
the original STDOUT was not restored after the end of the enclosing
block. My guess is that the "stashing away" that happens when one
uses local does not include the association of the file descriptor
with the file control block, which just gets lost after the
duplication with "open STDOUT, '>&', ...". If correct, this is a
shame, because it breaks the localization model. But I'm waaay
out of my depth here.

I'm fairly sure I've seen modules on CPAN to wrap it up a bit...

If anyone happens to know the CPAN module that Brian is referring
to here please let me know. I looked for it without any luck. I
searched for terms like "redirect" and "redirection". (FWIW, I
did my search with Google restricted to site:search.cpan.org.)

TIA!

kj

 

[Brian McCauley]

In <[email protected]> Brian McCauley

If anyone happens to know the CPAN module that Brian is referring
to here please let me know. I looked for it without any luck. I
searched for terms like "redirect" and "redirection". (FWIW, I
did my search with Google restricted to site:search.cpan.org.)

I think I was miss-remembering either Hook::Output::File or
SelectSaver - neither of which does what you want. Hook::Output::File
could be rewritten to do the right thing.

 

[kj]

I think I was miss-remembering either Hook::Output::File or
SelectSaver - neither of which does what you want. Hook::Output::File
could be rewritten to do the right thing.

Thanks! (That's one utterly mystifying chunk of code... And so
short too! I can't even begin to understand it; it sure is humbling.
Time for me to crawl back to my little world...)

kj

 

[Brian McCauley]

That was certainly illuminating. Thank you!


Inspired by this I replaced

local *STDOUT;

with

local *STDOUT = *STDOUT;

The subsequent redirection of STDOUT now worked as desired, but
the original STDOUT was not restored after the end of the enclosing
block. My guess is that the "stashing away" that happens when one
uses local does not include the association of the file descriptor
with the file control block, which just gets lost after the
duplication with "open STDOUT, '>&', ...".

With local *STDOUT = *STDOUT all you are doing is manipulating the
very top layers of the stuff described in my other post.

You are not creating a new IO-thingy nor are you changing anything in
the IO-thingy, filedescriptor, FCB chain. There's still only one
filedescriptor (1) associated with the original FCB and as soon as you
reopen STDOUT your process looses all connection with that FCB.

 

[Brian McCauley]

I think I was miss-remembering either Hook::Output::File or
SelectSaver - neither of which does what you want. Hook::Output::File
could be rewritten to do the right thing.

On the other hand simply abstracting the code from my previous post
into a subroutine is not too hard:

use AtExit;
sub save_fd {
my $std = shift;
open ( my $saved,'>&', $std) or die $!;
my $mode = shift;
open $std, $mode, @_ and new AtExit sub {
# Actually the or die is of dubious utility here
open ($std,'>&', $saved) or die $!;
};
}

print "To terminal\n";
{
my $save_STDOUT = save_fd \*STDOUT, '>', 'x.log' or die $!;
print "To log\n";
}
print "To terminal\n";

 

[kj]

On Aug 3, 8:02 pm, Brian McCauley <[email protected]> wrote:

On the other hand simply abstracting the code from my previous post
into a subroutine is not too hard:

use AtExit;
sub save_fd {
my $std = shift;
open ( my $saved,'>&', $std) or die $!;
my $mode = shift;
open $std, $mode, @_ and new AtExit sub {
# Actually the or die is of dubious utility here
open ($std,'>&', $saved) or die $!;
};
}

Thanks once more. AtExit is a handy module to know.

kj

 

[Dr.Ruud]

(e-mail address removed) schreef:

open my $pipe, '|-', "/usr/bin/sort -n > $filename", or die $!;

Alas, this requires you to use the shell-interpreted-version of the
open rather than the shell-less version.

Isn't there "concept" in Perl to change "SHELLMETAS"? (name as is used
in `man procmailrc`)

I often undefine SHELLMETAS in a .procmailrc, to prevent the shell being
used.

 

[xhoster]

(e-mail address removed) schreef:


Isn't there "concept" in Perl to change "SHELLMETAS"? (name as is used
in `man procmailrc`)

Well, there is the 4 or more form of open. That suppresses the shell.

I often undefine SHELLMETAS in a .procmailrc, to prevent the shell being
used.

But that would defeat the purpose. The ">" redirection has to be
interpreted by the shell, otherwise there is no point to doing it that way.

You could always pass $filename through quotemeta, although for all I know
that might backwhack some things which are ordinary to the shell but become
special when backwhacked.

Xho

 

[Brian McCauley]

I think I was miss-remembering either Hook::Output::File or
SelectSaver - neither of which does what you want.

For the sake of the archives, I'd like to add that various modules
with "Capture" in the name do indeed do what the OP was seeking and
maybe this is what I was remembering.