Really unbuffered reads...

Discussion in 'Perl Misc' started by Matthew Braid, Feb 20, 2004.

  1. Hi all,

    I'm using GnuPG::Interface to verify some PGP signed data, and for the
    most part it all seems to be working nicely.

    I have come across a small problem though. Sometimes (not always, which
    makes this nicely annoying) the function I use to read through the
    output blocks indefinitely.

    The segment of code that causes the problem is:

    # $fh is a filehandle passed in to the function -
    # its an IO::Handle object that's been used as the
    # stderr handle to GnuPG::Interface::verify
    my ($buffer, $ok) = ('', undef);
    while (1) {
    if (not sysread($fh, $buffer, 1024, length($buffer)) {
    last if not length $buffer;
    $ok = [1, $1] if $buffer =~ PGP_SIGOK;
    last;
    }
    # Do some stuff with buffer to look for individual lines...
    }

    This _almost_ always works fine. Rarely, that sysread hangs. If i'm
    looking at the process table at the time and I kill off the gpg process,
    everything trucks along as it should.

    Is there a way to fix this. Maybe a different method for reading in the
    data? I'm guessing that the problem is that sometimes gpg doesn't spit
    out a newline at the end of its stderr messages, but I can't find any
    obvious differences between messages that work and those that block.

    MB
     
    Matthew Braid, Feb 20, 2004
    #1
    1. Advertising

  2. Matthew Braid

    Anno Siegel Guest

    Matthew Braid <> wrote in comp.lang.perl.misc:
    > Hi all,
    >
    > I'm using GnuPG::Interface to verify some PGP signed data, and for the
    > most part it all seems to be working nicely.
    >
    > I have come across a small problem though. Sometimes (not always, which
    > makes this nicely annoying) the function I use to read through the
    > output blocks indefinitely.
    >
    > The segment of code that causes the problem is:
    >
    > # $fh is a filehandle passed in to the function -
    > # its an IO::Handle object that's been used as the
    > # stderr handle to GnuPG::Interface::verify
    > my ($buffer, $ok) = ('', undef);
    > while (1) {
    > if (not sysread($fh, $buffer, 1024, length($buffer)) {
    > last if not length $buffer;


    "last unless ..." reads better here.

    > $ok = [1, $1] if $buffer =~ PGP_SIGOK;

    ^^^^^^^^^
    Uh... I suppose that's a constant that returns a regex.

    > last;
    > }
    > # Do some stuff with buffer to look for individual lines...
    > }
    >
    > This _almost_ always works fine. Rarely, that sysread hangs. If i'm


    Use select() or IO::Select to see if the filehandle is ready for reading.
    You could also make it non-blocking, but I don't know how well that works
    with sysread().

    In both cases you'll have to decide what to do if the handle never
    becomes ready, and how long "never" is.

    Anno
     
    Anno Siegel, Feb 20, 2004
    #2
    1. Advertising

  3. Matthew Braid

    Uri Guttman Guest

    >>>>> "AS" == Anno Siegel <-berlin.de> writes:

    AS> Use select() or IO::Select to see if the filehandle is ready for reading.
    AS> You could also make it non-blocking, but I don't know how well that works
    AS> with sysread().

    sysread/write work fine with non-blocking as long as you handle the buffering
    yourself. in fact they usually required for non-blocking sockets as you
    could do a large print to such a socket and not all of the data will be
    written and the rest will sit in the buffer forever until you flush
    it. and you have no control then over how much gets written each time.

    uri

    --
    Uri Guttman ------ -------- http://www.stemsystems.com
    --Perl Consulting, Stem Development, Systems Architecture, Design and Coding-
    Search or Offer Perl Jobs ---------------------------- http://jobs.perl.org
     
    Uri Guttman, Feb 20, 2004
    #3
  4. Anno Siegel wrote:

    > Matthew Braid <> wrote in comp.lang.perl.misc:

    <snip>
    >> $ok = [1, $1] if $buffer =~ PGP_SIGOK;

    >
    > ^^^^^^^^^
    > Uh... I suppose that's a constant that returns a regex.


    Whoops - sorry, yeah it does.

    > Use select() or IO::Select to see if the filehandle is ready for reading.
    > You could also make it non-blocking, but I don't know how well that works
    > with sysread().
    >
    > In both cases you'll have to decide what to do if the handle never
    > becomes ready, and how long "never" is.


    As a quick fix I used a couple of surrounding alarm statements, which
    worked for a while until another test case came along that led to the
    solution.

    My first two test cases were very similar, the third was much larger. It
    seems the first two were just on either side of a boundary problem (they
    differed in size by 23 bytes, so it was a close thing :) )

    The problem was the output handle (not the error handle). The function
    that called the problem code was trying to handle verification,
    encryption and decryption. When it came to verification it sometimes
    decrypted the message instead of just verifying it so that the
    signatures on encrypted messages could be checked. Unfortunately, this
    meant that it 'threw away' the decrypted output but supplying an
    IO::Handle object that was never read from. It was also not closed until
    _after_ the error handle was scanned.

    This meant that for large enough messages whatever underlying buffer
    IO::Handle has was filling up. What I was seeing in top was gpg sitting
    in pipe-write state, and my script sitting in pipe-read - but gpg was
    trying to write to the output handle, while my script was trying to read
    from its error handle. I had thought that the two were stuck on the same
    handle.

    Once I changed the code so that if decryption is required for
    verification the output handle is undef (and thus the output disappears
    into the void) everything suddenly worked perfectly. It also explains
    why the problem only came up with verification - on encryption and
    decryption the output filehandle was opened to a temp file (I needed to
    keep it, and I needed it seekable), so the data could pretty much flow
    straight through unhindered.

    Phew :)

    Thanks for mentioning select though - looks a little cleaner than alarms...

    MB
     
    Matthew Braid, Feb 23, 2004
    #4
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Starbase Commander

    Unbuffered keyboard input???

    Starbase Commander, Sep 10, 2004, in forum: Perl
    Replies:
    1
    Views:
    700
    J├╝rgen Exner
    Sep 11, 2004
  2. Rich

    unbuffered output file

    Rich, Apr 15, 2004, in forum: C++
    Replies:
    5
    Views:
    2,640
    David Harmon
    Apr 16, 2004
  3. Guest
    Replies:
    0
    Views:
    394
    Guest
    Aug 28, 2004
  4. Guest
    Replies:
    1
    Views:
    558
    Jonathan Turkanis
    Aug 29, 2004
  5. michael young

    unbuffered input

    michael young, Feb 5, 2004, in forum: Python
    Replies:
    1
    Views:
    398
    Diez B. Roggisch
    Feb 5, 2004
Loading...

Share This Page