Fast pipe communication

[Andrei Alexandrescu (See Website For Email)]

Hello everybody,


I've written a small utility that allows Perl to communicate with TeX
(knowledge of TeX is not necessary to understand my question). Basically
I run perl to communicate with two pipes: perlin.tex and perlout.tex.
Perl reads a line off perlin.tex, evaluates the string, and writes it to
perlout.tex. Here's what I came up with:

===========================================
#!/bin/env perl
use strict;
use warnings;

system("rm -f perlin.tex perlout.tex && mkfifo perlin.tex perlout.tex") == 0
or die "Couldn't create pipes";

while () {
print "Opening perlin.tex...\n";
open(my $perlin, "perlin.tex") or die "Can't open: $!";
print "Opened perlin.tex\n";
print "Opening perlout.tex...\n";
open(my $perlout, ">perlout.tex") or die "Can't open: $!";
print "Opened perlout.tex\n";
$| = 1;
while (<$perlin>) {
chomp;
print "Read a line: `$_'\n";
next if $_ eq "";
my $r = eval($_);
die $@ if $@;
chomp $r;
print $perlout "$r\n";
}
print "Input pipe finished.\n";
}
===========================================

Test code:

$ perl perlengine.pl &
$ echo "1+1" > perlin.tex
$ cat perlout.tex
2
$ _

(Ignoring, of course, all of the debug messages.)

The question is, how can I make the code more efficient? I noticed that
every evaluation is really a pass through the outer loop. In other
words, I was unable to convince perl to block in <$perlin> once the
process writing to perlin.tex has finished. I need to reopen perlin.tex
for that; the call to open will block until someone wrote to perlin.tex.

Is there any chance of avoiding the expensive trip of opening and
closing the whole thing over and over again? Can I tell perl, "read to
the EOF of this pipe, and then rewind and read again, blocking if nobody
wrote to it"?


Thanks,

Andrei

 

[xhoster]

"Andrei Alexandrescu (See Website For Email)"

Hello everybody,

I've written a small utility that allows Perl to communicate with TeX
(knowledge of TeX is not necessary to understand my question). Basically
I run perl to communicate with two pipes: perlin.tex and perlout.tex.
Perl reads a line off perlin.tex, evaluates the string, and writes it to
perlout.tex. Here's what I came up with:

===========================================
#!/bin/env perl
use strict;
use warnings;

system("rm -f perlin.tex perlout.tex && mkfifo perlin.tex perlout.tex")
== 0
or die "Couldn't create pipes";

while () {
print "Opening perlin.tex...\n";
open(my $perlin, "perlin.tex") or die "Can't open: $!";
print "Opened perlin.tex\n";
print "Opening perlout.tex...\n";
open(my $perlout, ">perlout.tex") or die "Can't open: $!";
print "Opened perlout.tex\n";
$| = 1;

This turns on autoflush only for STDOUT. You probably need to turn it on
for $perlout as well (although that is not the cause of the current
problem.)

use IO::Handle; # This should go near the top of the script, not here
$perlout->autoflush(1);

while (<$perlin>) {
chomp;
print "Read a line: `$_'\n";
next if $_ eq "";
my $r = eval($_);
die $@ if $@;
chomp $r;
print $perlout "$r\n";
}
print "Input pipe finished.\n";
}
===========================================

Test code:

$ perl perlengine.pl &
$ echo "1+1" > perlin.tex
$ cat perlout.tex
2
$ _

(Ignoring, of course, all of the debug messages.)

The question is, how can I make the code more efficient? I noticed that
every evaluation is really a pass through the outer loop. In other
words, I was unable to convince perl to block in <$perlin> once the
process writing to perlin.tex has finished.

That's correct. Perl will not block on handles which are closed on the
other end.

I need to reopen perlin.tex
for that; the call to open will block until someone wrote to perlin.tex.

So it blocks at a slightly different place. Why is that a problem?

Is there any chance of avoiding the expensive trip of opening and
closing the whole thing over and over again?

You could move the opening of the perlout handle out of the loop, I see no
reason it needs to be repeated each time.

Anyway, assuming there were a way to hook a new process up to an existing
but closed pipe on one end, why should that be significantly more efficient
than re-opening the pipe? Both operations seem to be of about the same
complexity. The way to improve efficiency is to make the writing process
stay on the line rather than hanging up every time.

Can I tell perl, "read to
the EOF of this pipe, and then rewind and read again, blocking if nobody
wrote to it"?

I know of no way to do it. It seems like that more a matter for the OS
than for Perl.

Xho

 

[Andrei Alexandrescu (See Website For Email)]

So it blocks at a slightly different place. Why is that a problem?

You're right that the complexity of operation is the same, but generally
it's good to get away with as few system calls as it gets.

You could move the opening of the perlout handle out of the loop, I see no
reason it needs to be repeated each time.

I tried, but then TeX becomes unhappy (hangs), and I have exceedingly
little control over the way TeX manipulates files...

Anyway, assuming there were a way to hook a new process up to an existing
but closed pipe on one end, why should that be significantly more efficient
than re-opening the pipe? Both operations seem to be of about the same
complexity. The way to improve efficiency is to make the writing process
stay on the line rather than hanging up every time.


I know of no way to do it. It seems like that more a matter for the OS
than for Perl.

Thanks for your input, Xho. Made me feel better ). After all, the
system runs very smooth as is.


Andrei