Fast pipe communication

Discussion in 'Perl Misc' started by Andrei Alexandrescu (See Website For Email), Feb 3, 2006.

  1. Hello everybody,


    I've written a small utility that allows Perl to communicate with TeX
    (knowledge of TeX is not necessary to understand my question). Basically
    I run perl to communicate with two pipes: perlin.tex and perlout.tex.
    Perl reads a line off perlin.tex, evaluates the string, and writes it to
    perlout.tex. Here's what I came up with:

    ===========================================
    #!/bin/env perl
    use strict;
    use warnings;

    system("rm -f perlin.tex perlout.tex && mkfifo perlin.tex perlout.tex") == 0
    or die "Couldn't create pipes";

    while () {
    print "Opening perlin.tex...\n";
    open(my $perlin, "perlin.tex") or die "Can't open: $!";
    print "Opened perlin.tex\n";
    print "Opening perlout.tex...\n";
    open(my $perlout, ">perlout.tex") or die "Can't open: $!";
    print "Opened perlout.tex\n";
    $| = 1;
    while (<$perlin>) {
    chomp;
    print "Read a line: `$_'\n";
    next if $_ eq "";
    my $r = eval($_);
    die $@ if $@;
    chomp $r;
    print $perlout "$r\n";
    }
    print "Input pipe finished.\n";
    }
    ===========================================

    Test code:

    $ perl perlengine.pl &
    $ echo "1+1" > perlin.tex
    $ cat perlout.tex
    2
    $ _

    (Ignoring, of course, all of the debug messages.)

    The question is, how can I make the code more efficient? I noticed that
    every evaluation is really a pass through the outer loop. In other
    words, I was unable to convince perl to block in <$perlin> once the
    process writing to perlin.tex has finished. I need to reopen perlin.tex
    for that; the call to open will block until someone wrote to perlin.tex.

    Is there any chance of avoiding the expensive trip of opening and
    closing the whole thing over and over again? Can I tell perl, "read to
    the EOF of this pipe, and then rewind and read again, blocking if nobody
    wrote to it"?


    Thanks,

    Andrei
    Andrei Alexandrescu (See Website For Email), Feb 3, 2006
    #1
    1. Advertising

  2. Andrei Alexandrescu (See Website For Email)

    Guest

    "Andrei Alexandrescu (See Website For Email)"
    <> wrote:
    > Hello everybody,
    >
    > I've written a small utility that allows Perl to communicate with TeX
    > (knowledge of TeX is not necessary to understand my question). Basically
    > I run perl to communicate with two pipes: perlin.tex and perlout.tex.
    > Perl reads a line off perlin.tex, evaluates the string, and writes it to
    > perlout.tex. Here's what I came up with:
    >
    > ===========================================
    > #!/bin/env perl
    > use strict;
    > use warnings;
    >
    > system("rm -f perlin.tex perlout.tex && mkfifo perlin.tex perlout.tex")
    > == 0
    > or die "Couldn't create pipes";
    >
    > while () {
    > print "Opening perlin.tex...\n";
    > open(my $perlin, "perlin.tex") or die "Can't open: $!";
    > print "Opened perlin.tex\n";
    > print "Opening perlout.tex...\n";
    > open(my $perlout, ">perlout.tex") or die "Can't open: $!";
    > print "Opened perlout.tex\n";
    > $| = 1;


    This turns on autoflush only for STDOUT. You probably need to turn it on
    for $perlout as well (although that is not the cause of the current
    problem.)

    use IO::Handle; # This should go near the top of the script, not here
    $perlout->autoflush(1);

    > while (<$perlin>) {
    > chomp;
    > print "Read a line: `$_'\n";
    > next if $_ eq "";
    > my $r = eval($_);
    > die $@ if $@;
    > chomp $r;
    > print $perlout "$r\n";
    > }
    > print "Input pipe finished.\n";
    > }
    > ===========================================
    >
    > Test code:
    >
    > $ perl perlengine.pl &
    > $ echo "1+1" > perlin.tex
    > $ cat perlout.tex
    > 2
    > $ _
    >
    > (Ignoring, of course, all of the debug messages.)
    >
    > The question is, how can I make the code more efficient? I noticed that
    > every evaluation is really a pass through the outer loop. In other
    > words, I was unable to convince perl to block in <$perlin> once the
    > process writing to perlin.tex has finished.


    That's correct. Perl will not block on handles which are closed on the
    other end.

    > I need to reopen perlin.tex
    > for that; the call to open will block until someone wrote to perlin.tex.


    So it blocks at a slightly different place. Why is that a problem?

    > Is there any chance of avoiding the expensive trip of opening and
    > closing the whole thing over and over again?


    You could move the opening of the perlout handle out of the loop, I see no
    reason it needs to be repeated each time.

    Anyway, assuming there were a way to hook a new process up to an existing
    but closed pipe on one end, why should that be significantly more efficient
    than re-opening the pipe? Both operations seem to be of about the same
    complexity. The way to improve efficiency is to make the writing process
    stay on the line rather than hanging up every time.

    > Can I tell perl, "read to
    > the EOF of this pipe, and then rewind and read again, blocking if nobody
    > wrote to it"?


    I know of no way to do it. It seems like that more a matter for the OS
    than for Perl.

    Xho

    --
    -------------------- http://NewsReader.Com/ --------------------
    Usenet Newsgroup Service $9.95/Month 30GB
    , Feb 3, 2006
    #2
    1. Advertising

  3. wrote:
    >> I need to reopen perlin.tex
    >> for that; the call to open will block until someone wrote to perlin.tex.

    >
    > So it blocks at a slightly different place. Why is that a problem?


    You're right that the complexity of operation is the same, but generally
    it's good to get away with as few system calls as it gets.

    >> Is there any chance of avoiding the expensive trip of opening and
    >> closing the whole thing over and over again?

    >
    > You could move the opening of the perlout handle out of the loop, I see no
    > reason it needs to be repeated each time.


    I tried, but then TeX becomes unhappy (hangs), and I have exceedingly
    little control over the way TeX manipulates files...

    > Anyway, assuming there were a way to hook a new process up to an existing
    > but closed pipe on one end, why should that be significantly more efficient
    > than re-opening the pipe? Both operations seem to be of about the same
    > complexity. The way to improve efficiency is to make the writing process
    > stay on the line rather than hanging up every time.
    >
    >> Can I tell perl, "read to
    >> the EOF of this pipe, and then rewind and read again, blocking if nobody
    >> wrote to it"?

    >
    > I know of no way to do it. It seems like that more a matter for the OS
    > than for Perl.


    Thanks for your input, Xho. Made me feel better :eek:). After all, the
    system runs very smooth as is.


    Andrei
    Andrei Alexandrescu (See Website For Email), Feb 4, 2006
    #3
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. H Brett Bolen

    fast communication with c++

    H Brett Bolen, Oct 15, 2003, in forum: Java
    Replies:
    9
    Views:
    525
    Gordon Beaton
    Oct 18, 2003
  2. lee, wonsun
    Replies:
    1
    Views:
    484
    Jack Klein
    Nov 2, 2004
  3. Replies:
    0
    Views:
    661
  4. J Rice
    Replies:
    6
    Views:
    717
    J Rice
    Mar 11, 2006
  5. Replies:
    1
    Views:
    222
    Ben Morrow
    Jun 2, 2004
Loading...

Share This Page