sysread & eof

L

lucas

I'm use IO::Socket on linux to read from a webpage. I don't want to use
LWP::Simple. How would I find the EOF when the web server is finished
sending me the webpage? Here's what I'm using right now:

while (my $res = sysread($sock_in,$buf,$bufsize)) {
syswrite($sock_out,$buf,length($buf));
last if ($res < $bufsize);
}

It works, but obviously, if a webpage has a size that is a multiple of
$bufsize, this will loop forever.

Regards,
 
B

Ben Morrow

Quoth (e-mail address removed):
I'm use IO::Socket on linux to read from a webpage. I don't want to use
LWP::Simple. How would I find the EOF when the web server is finished
sending me the webpage? Here's what I'm using right now:

while (my $res = sysread($sock_in,$buf,$bufsize)) {
syswrite($sock_out,$buf,length($buf));
last if ($res < $bufsize);
}

It works, but obviously, if a webpage has a size that is a multiple of
$bufsize, this will loop forever.

I don't know what makes you think that. Did you try it? sysread returns
0 on EOF.

Note that you should also be checking defined($res), as sysread returns
undef on errors. Depending on how you've opened the socket and your
system, some errors (EINTR and EAGAIN, specifically) are not in fact
errors, but indications that you need to retry the read.

I think you need to read

perldoc -f sysread

and

man 2 read

before you go much further (the latter assumes you are on Unix; this is
the documentation for the C read function which is what Perl's sysread
calls. If you are on some other system you will need to find the
appropriate documentation for yourself).

Ben
 
L

lucas

Ben said:
Quoth (e-mail address removed):
I don't know what makes you think that. Did you try it? sysread returns
0 on EOF.

Note that you should also be checking defined($res), as sysread returns
undef on errors. Depending on how you've opened the socket and your
system, some errors (EINTR and EAGAIN, specifically) are not in fact
errors, but indications that you need to retry the read.

I think you need to read

perldoc -f sysread

and

man 2 read

before you go much further (the latter assumes you are on Unix; this is
the documentation for the C read function which is what Perl's sysread
calls. If you are on some other system you will need to find the
appropriate documentation for yourself).

In my original post, I mentioned I am using Linux; Slackware in particular.
I have read the perldoc for sysread. The problem is sysread blocks. So,
when the last of the data is transmitted, $res has a value, the data is
processed, and when the loop goes back to read more data, there is none,
and sysread just sits, waiting for more.

So, I figure, the only way to get sysread to do this is with non-blocking.
Using IO::Select, I came up with this:

my $select = IO::Select->new($socket);

while (1) {
if (my $ready = $select->can_read(1)) {
my $res = sysread($socket,$_,4096);
last if(!defined($res)); #never used
print "RES:$res\nLEN:",length($_),"\n$_\n";
}
else { print "!READY\n"; last; }
}

I tried using "can_read(0)", so that it wouldn't wait, but it was too fast
and the "else { print "!READY\n"; last; }" statement would catch. Without
the else statement, the loop never exists. The "last if(!defined($res));"
is never used because the can_read line doesn't enter the loop if there
isn't any data.

I guess this is the best way to get this to work. It just seems a bit
clunky.
 
X

xhoster

lucas said:
Ben Morrow wrote:

In my original post, I mentioned I am using Linux; Slackware in
particular. I have read the perldoc for sysread. The problem is sysread
blocks.

It blocks waiting on more data. It doesn't block on eof, as there is no
more data to wait on.
So, when the last of the data is transmitted, $res has a value,
the data is processed, and when the loop goes back to read more data,
there is none, and sysread just sits, waiting for more.

If that is actually the case, then the server is not sending an eof when it
should (or maybe it isn't supposed to send an eof and you are
misunderstanding the protocol).

.....
I guess this is the best way to get this to work. It just seems a bit
clunky.

That often happens when you decline to use the existing modules and insist
on rolling your own.

Xho
 
J

J. Gleixner

lucas said:
I'm use IO::Socket on linux to read from a webpage. I don't want to use
LWP::Simple. [...]

Of course not, otherwise you'd be finished and working on something else
by now.
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

No members online now.

Forum statistics

Threads
473,744
Messages
2,569,482
Members
44,900
Latest member
Nell636132

Latest Threads

Top