sysread & eof

Discussion in 'Perl Misc' started by lucas, Feb 28, 2007.

  1. lucas

    lucas Guest

    I'm use IO::Socket on linux to read from a webpage. I don't want to use
    LWP::Simple. How would I find the EOF when the web server is finished
    sending me the webpage? Here's what I'm using right now:

    while (my $res = sysread($sock_in,$buf,$bufsize)) {
    syswrite($sock_out,$buf,length($buf));
    last if ($res < $bufsize);
    }

    It works, but obviously, if a webpage has a size that is a multiple of
    $bufsize, this will loop forever.

    Regards,
    --
    lucas
    -------------------------
    Perl Coder since 2001
    shift || die;
    -------------------------
     
    lucas, Feb 28, 2007
    #1
    1. Advertising

  2. lucas

    Ben Morrow Guest

    Quoth :
    > I'm use IO::Socket on linux to read from a webpage. I don't want to use
    > LWP::Simple. How would I find the EOF when the web server is finished
    > sending me the webpage? Here's what I'm using right now:
    >
    > while (my $res = sysread($sock_in,$buf,$bufsize)) {
    > syswrite($sock_out,$buf,length($buf));
    > last if ($res < $bufsize);
    > }
    >
    > It works, but obviously, if a webpage has a size that is a multiple of
    > $bufsize, this will loop forever.


    I don't know what makes you think that. Did you try it? sysread returns
    0 on EOF.

    Note that you should also be checking defined($res), as sysread returns
    undef on errors. Depending on how you've opened the socket and your
    system, some errors (EINTR and EAGAIN, specifically) are not in fact
    errors, but indications that you need to retry the read.

    I think you need to read

    perldoc -f sysread

    and

    man 2 read

    before you go much further (the latter assumes you are on Unix; this is
    the documentation for the C read function which is what Perl's sysread
    calls. If you are on some other system you will need to find the
    appropriate documentation for yourself).

    Ben

    --
    I must not fear. Fear is the mind-killer. I will face my fear and
    I will let it pass through me. When the fear is gone there will be
    nothing. Only I will remain.
    Frank Herbert, 'Dune'
     
    Ben Morrow, Feb 28, 2007
    #2
    1. Advertising

  3. lucas

    lucas Guest

    Ben Morrow wrote:

    > Quoth :
    > I don't know what makes you think that. Did you try it? sysread returns
    > 0 on EOF.
    >
    > Note that you should also be checking defined($res), as sysread returns
    > undef on errors. Depending on how you've opened the socket and your
    > system, some errors (EINTR and EAGAIN, specifically) are not in fact
    > errors, but indications that you need to retry the read.
    >
    > I think you need to read
    >
    > perldoc -f sysread
    >
    > and
    >
    > man 2 read
    >
    > before you go much further (the latter assumes you are on Unix; this is
    > the documentation for the C read function which is what Perl's sysread
    > calls. If you are on some other system you will need to find the
    > appropriate documentation for yourself).


    In my original post, I mentioned I am using Linux; Slackware in particular.
    I have read the perldoc for sysread. The problem is sysread blocks. So,
    when the last of the data is transmitted, $res has a value, the data is
    processed, and when the loop goes back to read more data, there is none,
    and sysread just sits, waiting for more.

    So, I figure, the only way to get sysread to do this is with non-blocking.
    Using IO::Select, I came up with this:

    my $select = IO::Select->new($socket);

    while (1) {
    if (my $ready = $select->can_read(1)) {
    my $res = sysread($socket,$_,4096);
    last if(!defined($res)); #never used
    print "RES:$res\nLEN:",length($_),"\n$_\n";
    }
    else { print "!READY\n"; last; }
    }

    I tried using "can_read(0)", so that it wouldn't wait, but it was too fast
    and the "else { print "!READY\n"; last; }" statement would catch. Without
    the else statement, the loop never exists. The "last if(!defined($res));"
    is never used because the can_read line doesn't enter the loop if there
    isn't any data.

    I guess this is the best way to get this to work. It just seems a bit
    clunky.

    --
    lucas
    -------------------------
    Perl Coder since 2001
    shift || die;
    -------------------------
     
    lucas, Mar 1, 2007
    #3
  4. lucas

    Guest

    lucas <> wrote:
    > Ben Morrow wrote:
    >
    > In my original post, I mentioned I am using Linux; Slackware in
    > particular. I have read the perldoc for sysread. The problem is sysread
    > blocks.


    It blocks waiting on more data. It doesn't block on eof, as there is no
    more data to wait on.

    > So, when the last of the data is transmitted, $res has a value,
    > the data is processed, and when the loop goes back to read more data,
    > there is none, and sysread just sits, waiting for more.


    If that is actually the case, then the server is not sending an eof when it
    should (or maybe it isn't supposed to send an eof and you are
    misunderstanding the protocol).

    .....
    >
    > I guess this is the best way to get this to work. It just seems a bit
    > clunky.


    That often happens when you decline to use the existing modules and insist
    on rolling your own.

    Xho

    --
    -------------------- http://NewsReader.Com/ --------------------
    Usenet Newsgroup Service $9.95/Month 30GB
     
    , Mar 1, 2007
    #4
  5. lucas

    J. Gleixner Guest

    lucas wrote:
    > I'm use IO::Socket on linux to read from a webpage. I don't want to use
    > LWP::Simple. [...]


    Of course not, otherwise you'd be finished and working on something else
    by now.
     
    J. Gleixner, Mar 1, 2007
    #5
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Kobu
    Replies:
    10
    Views:
    640
    Keith Thompson
    Mar 4, 2005
  2. SpreadTooThin

    ifstream eof not reporting eof?

    SpreadTooThin, Jun 13, 2007, in forum: C++
    Replies:
    10
    Views:
    707
    James Kanze
    Jun 15, 2007
  3. Hal Fulton

    sysread and buffered I/O

    Hal Fulton, Jul 21, 2004, in forum: Ruby
    Replies:
    39
    Views:
    666
    Tanaka Akira
    Jul 24, 2004
  4. William E. Rubin
    Replies:
    3
    Views:
    127
    William E. Rubin
    Dec 8, 2005
  5. Jan Burse
    Replies:
    67
    Views:
    1,082
    Jan Burse
    Mar 14, 2012
Loading...

Share This Page