[ANN] EventLoop 0.0.20050825.1600

[Bill Kelly]

^^^^^^^^ hang

I guess it must be something very basic I'm not aware of
about #eof? ... Because even:

hangs.


Is it not possible to check for EOF without blocking?


Regards,

Bill

 

[Ara.T.Howard]

I guess it must be something very basic I'm not aware of
about #eof? ... Because even:


hangs.


Is it not possible to check for EOF without blocking?

sometimes yes, sometimes no, or so it seems:

[ahoward@localhost ~]$ cat a.rb
require 'socket'
s = TCPSocket::new '127.0.0.1', 80
s.eof?

[ahoward@localhost ~]$ strace ruby a.rb 2>&1
...
...
...
read(3


and, from io.c:

/*
* call-seq:
* ios.eof => true or false
* ios.eof? => true or false
*
* Returns true if <em>ios</em> is at end of file. The stream must be
* opened for reading or an <code>IOError</code> will be raised.
*
* f = File.new("testfile")
* dummy = f.readlines
* f.eof #=> true
*/

VALUE
rb_io_eof(io)
VALUE io;
{
OpenFile *fptr;
int ch;

GetOpenFile(io, fptr);
rb_io_check_readable(fptr);

if (feof(fptr->f)) return Qtrue;
if (READ_DATA_PENDING(fptr->f)) return Qfalse;
READ_CHECK(fptr->f);
clearerr(fptr->f);
TRAP_BEG;
ch = getc(fptr->f); // look here !!!
TRAP_END;

if (ch != EOF) {
ungetc(ch, fptr->f);
return Qfalse;
}
rb_io_check_closed(fptr);
clearerr(fptr->f);
return Qtrue;
}


i'm no systems/network guy - but it seems like maybe one shouldn't try to read
a char from anything like a socket, pipe, etc. that might hang for a read in
this case... i dunno how to check for this kind of thing though... maybe:

if(fseek(fptr->f) == EBADF) {
return Qnil; // we can't tell for this stream!
}
else {
TRAP_BEG;
ch = getc(fptr->f);
TRAP_END;
}


or maybe an error could be thrown up front for this case (Errno::EWOULDBLOCK
for example)... but this doesn't seem right since you may want to check eof
for a socket at times... nil could mean "don't know."

anyhow... it sure makes sense that it'd block though...

hth.

-a
--
===============================================================================
| email :: ara [dot] t [dot] howard [at] noaa [dot] gov
| phone :: 303.497.6469
| Your life dwells amoung the causes of death
| Like a lamp standing in a strong breeze. --Nagarjuna
===============================================================================

 

[Bill Kelly]

[ahoward@localhost ~]$ strace ruby a.rb 2>&1
...
...
...
read(3


and, from io.c: [...]
ch = getc(fptr->f); // look here !!! [...]
anyhow... it sure makes sense that it'd block though...

Thanks, Ara !

Guess I would have expected it just to do the

if (feof(fptr->f)) return Qtrue;

part...


I've changed my program to infer EOF when gets() returns
nil. I no longer call #eof? because although it admittedly
looks pretty darned thorough, it's no good for me to block.


Thanks again,

Regards,

Bill

 

[Tanaka Akira]

Is it not possible to check for EOF without blocking?

I documented the blocking behavior of IO#eof?. I hope it reduces
confusion about IO#eof?.

Index: io.c
===================================================================
RCS file: /src/ruby/io.c,v
retrieving revision 1.376
diff -u -p -r1.376 io.c
--- io.c 30 Aug 2005 14:49:51 -0000 1.376
+++ io.c 5 Sep 2005 14:52:58 -0000
@@ -908,12 +908,31 @@ io_getc(OpenFile *fptr)
* ios.eof => true or false
* ios.eof? => true or false
*
- * Returns true if <em>ios</em> is at end of file. The stream must be
- * opened for reading or an <code>IOError</code> will be raised.
+ * Returns true if <em>ios</em> is at end of file that means
+ * there are no more data to read.
+ * The stream must be opened for reading or an <code>IOError</code> will be
+ * raised.
*
* f = File.new("testfile")
* dummy = f.readlines
* f.eof #=> true
+ *
+ * If <em>ios</em> is a stream such as pipe or socket, <code>IO#eof?</code>
+ * blocks until the other end sends some data or closes it.
+ *
+ * r, w = IO.pipe
+ * Thread.new { sleep 1; w.close }
+ * r.eof? #=> true after 1 second blocking
+ *
+ * r, w = IO.pipe
+ * Thread.new { sleep 1; w.puts "a" }
+ * r.eof? #=> false after 1 second blocking
+ *
+ * r, w = IO.pipe
+ * r.eof? # blocks forever
+ *
+ * Note that <code>IO#eof?</code> reads data to a input buffer.
+ * So <code>IO#sysread</code> doesn't work with <code>IO#eof?</code>.
*/

VALUE

 

[Morgan]

I documented the blocking behavior of IO#eof?. I hope it reduces
confusion about IO#eof?.

Hmmmm...

Correct me if I'm wrong, but using eof? on a socket would be rather
useless, wouldn't it?

I'd *really* like to find something that will just tell me if there's data
waiting on a socket. Something I can use like

someSocket.incoming_data_waiting?

And if there's something I can read, I'll get true, and otherwise I'll get
false. But nothing I've tried seems to work that way.

-Morgan, wonders why such a simple seeming thing is so difficult?

 

[Joel VanderWerf]

Hmmmm...

Correct me if I'm wrong, but using eof? on a socket would be rather
useless, wouldn't it?

I'd *really* like to find something that will just tell me if there's data
waiting on a socket. Something I can use like

someSocket.incoming_data_waiting?

And if there's something I can read, I'll get true, and otherwise I'll get
false. But nothing I've tried seems to work that way.

Doesn't select work, with a timeout of 0? I haven't tried that, but it
seems like what you are asking for.

 

[Hal Fulton]

This whole issue always confuses me. But isn't that the purpose
of readpartial? Or am I mistaken?


Hal

 

[Tanaka Akira]

Correct me if I'm wrong, but using eof? on a socket would be rather
useless, wouldn't it?

I'd *really* like to find something that will just tell me if there's data
waiting on a socket. Something I can use like

someSocket.incoming_data_waiting?

I'm not sure why people want a such method.

What is a situation you need the method?

And if there's something I can read, I'll get true, and otherwise I'll get
false. But nothing I've tried seems to work that way.

If it returns false, your program cannot have data from the socket.
So your program cannot do about the data. If your program has
something to do in such case, there should be some work which doesn't
depend the data.

If the work is done by other threads, the blocking behavior is
appropriate because the other threads is run at the blocking time.

If the program use a event driven framework, the readability test
should be done by the main event loop in the framework. So the
program don't need the method.

So I guess your program has another situation I don't imagine.

However it is possible to implement the method by IO.select.

 

[Morgan]

I'm not sure why people want a such method.

What is a situation you need the method?

I suppose because it's part of the only way I've found to get around the
lack of
a someSocket.readallavailable - something that will give me *all* the data
that can be read from the socket at the time I call it. (I could use recv with
a number larger than anything I should ever have waiting for an argument,
but that's crude. Admittedly, the workaround is also crude.)

Sometimes the only way I have of being certain that I've gotten all of an
incoming line is that I haven't been sent anything more. So, in place of
a readallavailable function, I have this procedure (which I have yet to code,
but I think should work):

1. Get a chunk of data with recv. At this point, I don't care if it blocks.
2. Add received data to a buffer.
3. Check to see if there's any data waiting.
4. If there is, go back to step 1. If there isn't, go on.
5. Now being sure that the buffer contains all data currently
available, the program can process it.

I'd also love to see a method that would tell me how many bytes are
waiting to be read at the moment. But I wouldn't be surprised if that's
not feasible for some reason or another.

If it returns false, your program cannot have data from the socket.
So your program cannot do about the data. If your program has
something to do in such case, there should be some work which doesn't
depend the data.

If the work is done by other threads, the blocking behavior is
appropriate because the other threads is run at the blocking time.

If the program use a event driven framework, the readability test
should be done by the main event loop in the framework. So the
program don't need the method.

I tried doing something like this (or at least what I think you're
referring to) in the program I'm working on now. (Using Fox toolkit
via FXRuby.) However, I couldn't get it to work right on my own, and
my message here asking for help got no replies...

In any case, the method called by the framework to handle the
event will still have to deal with the "do I have all the data?" issue.

Threads, though powerful, are rather annoying to deal with sometimes.
In the program I'm working on now, they're probably the best way
of handling things anyway, but I'd like to have a way of working
with sockets that doesn't force me to use them when they don't
suit my needs.

So I guess your program has another situation I don't imagine.

However it is possible to implement the method by IO.select.

It can do the job, yes. But it also seems like overkill for someone
who's only interested in one object. And it'd be a lot easier to read
something that simply calls a method on the socket of concern.

-Morgan

 

[Bill Kelly]

(I could use recv with
a number larger than anything I should ever have waiting for an argument,
but that's crude.

Just curious why that would seem crude. It's my understanding
that's why recv() behaves as it does.

The following is untested, but is similar to what I've written
many times:

def TCPSocket.read_all_available
buf = ""
while select([self], nil, nil, 0)
dat = self.recv(65536) rescue nil
break unless dat && ! dat.empty?
buf << dat
end
buf
end

Usually instead of just a "break", I'll set an "eof" flag
when I get a nil (or an exception) back from recv().
(Note that select() will indicate that a socket is
'readable' if it's at EOF, but then recv() returns nothing
and/or an exception... as I recall... Thus the above
'rescue nil' followed by checking for no data returned.)

But anyway . . .

Hope this helps,

Regards,

Bill

 

[Bill Kelly]

def TCPSocket.read_all_available

Oops, I meant

class TCPSocket
def read_all_available

...
end
end


Regards,

Bill

 

[Tanaka Akira]

I suppose because it's part of the only way I've found to get around the
lack of
a someSocket.readallavailable - something that will give me *all* the data
that can be read from the socket at the time I call it. (I could use recv with
a number larger than anything I should ever have waiting for an argument,
but that's crude. Admittedly, the workaround is also crude.)

Why you needs the all available data?

Even if you got the all data at the time, some data may be arrived
just after that.

Sometimes the only way I have of being certain that I've gotten all of an
incoming line is that I haven't been sent anything more. So, in place of
a readallavailable function, I have this procedure (which I have yet to code,
but I think should work):

1. Get a chunk of data with recv. At this point, I don't care if it blocks.
2. Add received data to a buffer.
3. Check to see if there's any data waiting.
4. If there is, go back to step 1. If there isn't, go on.
5. Now being sure that the buffer contains all data currently
available, the program can process it.

You can't assume that the all data is in the buffer because
some data may be arrived between 4 and 5.

Is it acceptable?

If it is acceptable, the program can process the data incrementaly;
for each 4K bytes or a line or another.

If it is not acceptable, I think it's a problem of the protocol.

I tried doing something like this (or at least what I think you're
referring to) in the program I'm working on now. (Using Fox toolkit
via FXRuby.) However, I couldn't get it to work right on my own, and
my message here asking for help got no replies...

Hm. GUI library. I understand why your program can't block.

In any case, the method called by the framework to handle the
event will still have to deal with the "do I have all the data?" issue.

Yes. The framework notify that some data (or EOF) available.
It doesn't notify how much data available.

However the framework should notify readability until kernel has no
data. So it is no problem that the callback doesn't process the all
data. If the callback process 1byte or more at once, all data will be
processed.

It can do the job, yes. But it also seems like overkill for someone
who's only interested in one object. And it'd be a lot easier to read
something that simply calls a method on the socket of concern.

I'm not sure that Ruby should encourage to read all data available.