synchronistaion of STDIN with redirected STDOUT

Discussion in 'Perl Misc' started by Dilbert, Jan 12, 2010.

  1. Dilbert

    Dilbert Guest

    I have the following program:

    use strict;
    use warnings;

    # redirect STDOUT to a file, but keep output to
    # screen alive (via the "tee" command)
    open STDOUT, "| tee logfile.txt" or die "Error: $!";

    print "abc", "z" x 30, "enter a number >";
    my $num = <STDIN>;
    print "\nYou entered: $num\n";

    When I run this program, I see

    abczzzzzzzzzzzzzzzzzz

    but I can see no prompt "enter a number >"

    the cursor waits after the "...zzzz"

    I can blindly type in a number (e.g. 123), when I do this, I see the
    following on my screen:

    abczzzzzzzzzzzzzzzzzz123
    zzzzzzzzzzzzenter a number >
    You entered: 123

    The problem here is that STDIN and STDOUT are not synchronized anymore
    (probably due to the previous open STDOUT, "| tee logfile.txt"). --
    and by synchronized I mean that the STDOUT buffer is flushed before
    every read from STDIN.

    How can I teach perl that STDOUT buffer is to be flushed before every
    read from STDIN, even after an open STDOUT, "| tee logfile.txt".

    I am on Ubuntu Linux:

    This is perl, v5.10.1 built for i686-linux-thread-multi
    (with 2 registered patches, see perl -V for more detail)
    Copyright 1987-2009, Larry Wall
    Binary build 1006 [291086] provided by ActiveState http://www.ActiveState.com
    Built Aug 24 2009 13:45:03
    Dilbert, Jan 12, 2010
    #1
    1. Advertising

  2. Dilbert <> wrote:

    >How can I teach perl that STDOUT buffer is to be flushed before every
    >read from STDIN, even after an open STDOUT, "| tee logfile.txt".


    What happened when you asked the FAQ?

    perldoc -q flush
    perldoc -q buffer

    Of course this only tells perl to not buffer the output, your OS might
    still do its own buffering.

    jue
    Jürgen Exner, Jan 12, 2010
    #2
    1. Advertising

  3. Dilbert

    Dilbert Guest

    On 12 jan, 17:15, Jürgen Exner <> wrote:
    > Dilbert <> wrote:
    > >How can I teach perl that STDOUT buffer is to be flushed before every
    > >read from STDIN, even after an open STDOUT, "| tee logfile.txt".

    >
    > What happened when you asked the FAQ?
    >
    >         perldoc -q flush
    >         perldoc -q buffer
    >
    > Of course this only tells perl to not buffer the output, your OS might
    > still do its own buffering.


    The FAQ gave me an example with setting autoflush(1), ...
    use IO::Handle;
    open my( $io_fh ), ">", "output.txt";
    $io_fh->autoflush(1);
    .... but this disables buffering for the whole program, which is not
    what I want, I just want to flush automatically before every read from
    STDIN.

    The other example from the FAQ is better...
    $io_fh->flush;
    ....but still not satisfactory. In fact I would have to intervene in
    the programs and add an STDOUT->flush before each read from STDIN.
    It's a good solution, but I want something better.

    From what I understand, Perl puts STDIN and STDOUT in a "special
    relationship", such that STDOUT buffered under normal circumstances,
    but is *automatically* flushed before each read from STDIN, no need to
    manually flush using STDOUT->flush. The reason for that is that STDIN
    and STDOUT are both connected to the same terminal.

    Now, if I re-open STDOUT '| tee logfile.txt', Perl sees that STDOUT is
    not connected anymore to the terminal and therefore it cancels the
    "special relationship" between STDIN and STDOUT, the re-opened STDOUT
    is *not* flushed automatically before each read from STDIN.

    The point I make is that, yes, I re-open STDOUT to write to a flat
    file, but I use a backdoor and re-connect it via the "tee" command
    back to the terminal.

    What I want in this case (where STDOUT is re-opened to a flat file and
    then re-connected back to the terminal) is to establish the original
    relationship between STDIN and STDOUT where STDOUT is *automatically*
    flushed before each read from STDIN, no need to manually flush using
    STDOUT->flush.
    Dilbert, Jan 12, 2010
    #3
  4. Dilbert

    Jim Gibson Guest

    In article
    <>,
    Dilbert <> wrote:

    > On 12 jan, 17:15, Jürgen Exner <> wrote:
    > > Dilbert <> wrote:
    > > >How can I teach perl that STDOUT buffer is to be flushed before every
    > > >read from STDIN, even after an open STDOUT, "| tee logfile.txt".

    > >
    > > What happened when you asked the FAQ?
    > >
    > >         perldoc -q flush
    > >         perldoc -q buffer
    > >
    > > Of course this only tells perl to not buffer the output, your OS might
    > > still do its own buffering.

    >


    [problem with flushing output to screen before reading input snipped]

    It is likely that it is the tee program that is doing the buffering and
    needs to be flushed before input is read. Since this is not possible,
    you should consider some alternative approach that does not use an
    external program to duplicate output to the screen and a file.

    How about writing a function that takes input and writes it to the
    screen and a file?

    This has been done before, so search at http://search.cpan.org for
    "tee" and find modules such as File::Tee, IO::Tee, and PerlIO::tee. I
    have not used any of these, so I don't know if they will work for you.

    --
    Jim Gibson
    Jim Gibson, Jan 12, 2010
    #4
  5. On 2010-01-12 22:48, Jim Gibson <> wrote:
    > [problem with flushing output to screen before reading input snipped]
    >
    > It is likely that it is the tee program that is doing the buffering and
    > needs to be flushed before input is read.


    No. tee doesn't buffer (at least not the implementations I'm familiar
    with). Dilbert correctly (except that he erroneously calls a pipe a
    "flat file", but that makes no difference in this context) the behaviour
    required for stdio by the C standard. Apparently perlio emulates this
    behaviour.

    hp
    Peter J. Holzer, Jan 12, 2010
    #5
  6. Dilbert

    Dilbert Guest

    On 13 jan, 00:09, "Peter J. Holzer" <> wrote:
    > On 2010-01-12 22:48, Jim Gibson <> wrote:
    >
    > > [problem with flushing output to screen before reading input snipped]

    >
    > > It is likely that it is the tee program that is doing the buffering and
    > > needs to be flushed before input is read.

    >
    > No. tee doesn't buffer (at least not the implementations I'm familiar
    > with).


    I confirm, if I flush STDOUT in my Perl program, then the output is
    ok.

    > Dilbert correctly (except that he erroneously calls a pipe a
    > "flat file", but that makes no difference in this context) the behaviour
    > required for stdio by the C standard. Apparently perlio emulates this
    > behaviour.


    Under normal circumstances, perlio emulates the behaviour required for
    stdio by the C standard correctly. And I understand that this
    behaviour is dropped if I re-open STDOUT to another file.

    What I would like to achieve is to teach perlio *not* to forget this
    behaviour in this particular case when STDOUT is re-opened to a pipe
    that tee's back to the terminal.
    Dilbert, Jan 13, 2010
    #6
  7. On 2010-01-13 09:40, Dilbert <> wrote:
    > On 13 jan, 00:09, "Peter J. Holzer" <> wrote:
    >> Dilbert correctly (except that he erroneously calls a pipe a
    >> "flat file", but that makes no difference in this context) the behaviour
    >> required for stdio by the C standard. Apparently perlio emulates this
    >> behaviour.

    >
    > Under normal circumstances, perlio emulates the behaviour required for
    > stdio by the C standard correctly.


    It also emulates the behaviour correctly in this case.

    > And I understand that this behaviour is dropped if I re-open STDOUT to
    > another file.


    AIUI, you didn't reopen STDOUT. Filedescriptor 1 already referred to a
    non-terminal when your program was called.


    > What I would like to achieve is to teach perlio *not* to forget this
    > behaviour in this particular case when STDOUT is re-opened to a pipe
    > that tee's back to the terminal.


    Yes, I understood that. But the behaviour you want is *not* allowed by
    the C standard. Of course the C standard is only marginally relevant to
    perl, so perl could implement a more intelligent flushing scheme.
    However, it cannot detect what the program on the other end of the pipe
    (tee in this case) does, so it either needs to be a better heuristic
    (which may be difficult because "better" includes "doesn't break
    existing programs") or it needs to be user/programmer-controllable.
    Maybe a function "flush that stream before reading from this stream". Or
    - even more generic - just a hook which is called for certain ops. So
    you could do something like

    $in->add_hook(pre_read => sub { $out->flush });

    hp
    Peter J. Holzer, Jan 13, 2010
    #7
  8. Dilbert

    Dilbert Guest

    On 13 jan, 13:29, "Peter J. Holzer" <> wrote:
    > Yes, I understood that. But the behaviour you want is *not* allowed by
    > the C standard. Of course the C standard is only marginally relevant to
    > perl, so perl could implement a more intelligent flushing scheme.
    > However, it cannot detect what the program on the other end of the pipe
    > (tee in this case) does, so it either needs to be a better heuristic
    > (which may be difficult because "better" includes "doesn't break
    > existing programs")


    I agree, detecting what the program on the other end of the pipe does
    is difficult.

    > or it needs to be user/programmer-controllable.
    > Maybe a function "flush that stream before reading from this stream". Or
    > - even more generic - just a hook which is called for certain ops.


    Adding a hook which is called for certain ops is the way I want to go.

    > So you could do something like
    >
    > $in->add_hook(pre_read => sub { $out->flush });


    Can this hook also be added to a simple
    my $answer = <STDIN>;

    Maybe the following ?
    use IO::Handle;
    STDIN->add_hook(pre_read => sub { STDOUT->flush });

    ....but that gave me an error:
    Can't locate object method "add_hook" via package "IO::Handle"
    Dilbert, Jan 13, 2010
    #8
  9. On 2010-01-13 13:41, Dilbert <> wrote:
    > On 13 jan, 13:29, "Peter J. Holzer" <> wrote:
    >> Maybe a function "flush that stream before reading from this stream". Or
    >> - even more generic - just a hook which is called for certain ops.

    >
    > Adding a hook which is called for certain ops is the way I want to go.
    >
    >> So you could do something like
    >>
    >> $in->add_hook(pre_read => sub { $out->flush });

    >
    > Can this hook also be added to a simple
    > my $answer = <STDIN>;
    >
    > Maybe the following ?
    > use IO::Handle;
    > STDIN->add_hook(pre_read => sub { STDOUT->flush });
    >
    > ...but that gave me an error:
    > Can't locate object method "add_hook" via package "IO::Handle"


    This was an idea how such a feature could look like in a future version
    of perl. It doesn't currently exist (as far as I know).

    I think the simplest solution for you is to just set $|. Since you
    you send the output to the screen anyway, this is unlikely to degrade
    performance noticeably.

    hp
    Peter J. Holzer, Jan 13, 2010
    #9
  10. On 2010-01-14 02:19, Ben Morrow <> wrote:
    > Quoth "Peter J. Holzer" <>:
    >> Maybe a function "flush that stream before reading from this stream". Or
    >> - even more generic - just a hook which is called for certain ops. So
    >> you could do something like
    >>
    >> $in->add_hook(pre_read => sub { $out->flush });

    >
    > This is called a 'PerlIO layer'.


    Duh! You are right of course. I haven't thought of that. (I did think of
    subclassing IO::Handle, but I couldn't figure out how to make STDOUT use
    the new class).

    hp
    Peter J. Holzer, Jan 14, 2010
    #10
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Ben_
    Replies:
    0
    Views:
    322
  2. Replies:
    8
    Views:
    5,578
  3. Pappy
    Replies:
    3
    Views:
    370
  4. Dirk Loss
    Replies:
    3
    Views:
    314
    Bjoern Schliessmann
    Dec 19, 2007
  5. Replies:
    2
    Views:
    314
    A. Sinan Unur
    Dec 7, 2005
Loading...

Share This Page