oddities with select

B

Ben Giddings

I'm trying to use Ruby to talk to an network application, and noticed
something odd. Using ethereal, I can see that the server is returning a
newline to the client, however my Ruby code, calling select([socket],
[], [], 0.1) repeatedly for 5 seconds never seems to know that there's
data to be read.

I've tried to make sure the socket is non-blocking with:

@sock.fcntl(Fcntl::F_SETFL, Fcntl::O_NONBLOCK)

But that seems to have no effect.

Is there something I'm missing?

I haven't tried the same code in another language to see if the problem
may be there. At the moment though, this seems broken.

Any ideas?

Ben
 
A

Ara.T.Howard

I'm trying to use Ruby to talk to an network application, and noticed
something odd. Using ethereal, I can see that the server is returning a
newline to the client, however my Ruby code, calling select([socket],
[], [], 0.1) repeatedly for 5 seconds never seems to know that there's
data to be read.

I've tried to make sure the socket is non-blocking with:

@sock.fcntl(Fcntl::F_SETFL, Fcntl::O_NONBLOCK)

But that seems to have no effect.

Is there something I'm missing?

don't you want to block? i mean, isn't that why you are using select? to do
nothing until data arrives? if it IS blocking your code should sleep untill
data becomes available.

also, i'm not sure what the buffering semantics of select are - but i'm sure
there are some - might want to check that out too...

I haven't tried the same code in another language to see if the problem
may be there. At the moment though, this seems broken.

Any ideas?

Ben

-a
--
===============================================================================
| EMAIL :: Ara [dot] T [dot] Howard [at] noaa [dot] gov
| PHONE :: 303.497.6469
| A flower falls, even though we love it; and a weed grows, even though we do
| not love it. --Dogen
===============================================================================
 
B

Ben Giddings

Ara.T.Howard said:
don't you want to block? i mean, isn't that why you are using select?
to do
nothing until data arrives? if it IS blocking your code should sleep
untill
data becomes available.

Actually, no. What I want to do is know that a command was accepted
(indicated by a blank line), unfortunately the code that I'm talking to
doesn't always do the right thing, so I want to bail out eventually if
it isn't going to happen.

What I want to do is: "Try to read until you get a blank line, for a
maximum of N seconds"

I know that some commands will go quickly, giving me a blank line in
under 200ms, and others will take a few seconds to complete, but if
they're taking 500ms and say 4.5s respectively, I want to stop waiting.
also, i'm not sure what the buffering semantics of select are - but i'm
sure
there are some - might want to check that out too...

Hmm, ok... I thought select just told you when there was data available,
and either blocked or didn't, based on the timeout parameter you give
it. That's why I thought that I could essentially poll it with a low
timeout value, and when data became availble, I could use 'recv' to read it.

Ben
 
A

Ara.T.Howard

Actually, no. What I want to do is know that a command was accepted
(indicated by a blank line), unfortunately the code that I'm talking to
doesn't always do the right thing, so I want to bail out eventually if
it isn't going to happen.

What I want to do is: "Try to read until you get a blank line, for a
maximum of N seconds"


why not something along the lines of this

buf = ''
t = Thread.new{ loop{ buf << io.getc } }
sleep n_seconds
t.kill

case buf
when %r/^$/o
raise "command not accepted"
when %r/^\s+$/o
#
# handle command
#
else
raise "bad response <#{ buf.inspect }>"
end


??

I know that some commands will go quickly, giving me a blank line in
under 200ms, and others will take a few seconds to complete, but if
they're taking 500ms and say 4.5s respectively, I want to stop waiting.


Hmm, ok... I thought select just told you when there was data available, and
either blocked or didn't, based on the timeout parameter you give it.
That's why I thought that I could essentially poll it with a low timeout
value, and when data became availble, I could use 'recv' to read it.

Ben

no - you are correct. ignore me! it's just that you don't know how MUCH data
is available - only that it is, or that it's eof.

-a
--
===============================================================================
| EMAIL :: Ara [dot] T [dot] Howard [at] noaa [dot] gov
| PHONE :: 303.497.6469
| A flower falls, even though we love it; and a weed grows, even though we do
| not love it. --Dogen
===============================================================================
 
B

Ben Giddings

Ara.T.Howard said:
why not something along the lines of this

buf = ''
t = Thread.new{ loop{ buf << io.getc } }
sleep n_seconds
t.kill

case buf
when %r/^$/o
raise "command not accepted"
when %r/^\s+$/o
#
# handle command
#
else
raise "bad response <#{ buf.inspect }>"
end


What I'm actually using is pretty close to that:

def internal_read(timeout=DEFAULT_TIMEOUT, stop_pattern=nil)
retval = ""

timed_out = false

begin
timeout(timeout) do
while true
if select([@sock], [], [], 0.1)
retval += @sock.recv(256)
if stop_pattern
if retval =~ stop_pattern
break
end
end
end
end
end
rescue Timeout::Error
timed_out = true
end


# if timed_out
# puts "Timed out"
# raise
# end

retval
end

The problem is that although I've seen a newline go over the network to
my PC, it is never spotted by the Ruby app, which sits there doing the
'select' forever.

Your code looks slightly different, but I think it would do the exact
same thing.

For some reason, the newline goes over the network, but doesn't make it
far enough (in 5s) that the 'select()' call sees it.

Ben
 
A

Ara.T.Howard

What I'm actually using is pretty close to that:

def internal_read(timeout=DEFAULT_TIMEOUT, stop_pattern=nil)
retval = ""

timed_out = false

begin
timeout(timeout) do
while true
if select([@sock], [], [], 0.1)
retval += @sock.recv(256)
if stop_pattern
if retval =~ stop_pattern
break
end
end
end
end
end
rescue Timeout::Error
timed_out = true
end


# if timed_out
# puts "Timed out"
# raise
# end

retval
end

The problem is that although I've seen a newline go over the network to
my PC, it is never spotted by the Ruby app, which sits there doing the
'select' forever.

Your code looks slightly different, but I think it would do the exact
same thing.

For some reason, the newline goes over the network, but doesn't make it
far enough (in 5s) that the 'select()' call sees it.

Ben

didn't you say something about your socket being in non-blocking mode? if so,
isn't this just a busy loop since select will keep returning 'true - a read
will not block':

man 2 select

...
three independent sets of descriptors are watched. those listed in
readfds will be watched to see if characters become available for read-
ing (more precisely, to see if a read will not block - in particular, a
file descriptor is also ready on end-of-file)
...

if not in non-blocking mode, perhaps your pattern is bad? what happens if you
do this:

while true
if select([@sock], [], [], 0.1)
retval += @sock.recv(256)
p retval
</snip>

??

do you see retval growing?


-a
--
===============================================================================
| EMAIL :: Ara [dot] T [dot] Howard [at] noaa [dot] gov
| PHONE :: 303.497.6469
| A flower falls, even though we love it; and a weed grows, even though we do
| not love it. --Dogen
===============================================================================
 
C

Chris Reay

Ben Giddings said:
def internal_read(timeout=DEFAULT_TIMEOUT, stop_pattern=nil)
retval = ""

timed_out = false

begin
timeout(timeout) do
while true
if select([@sock], [], [], 0.1)
retval += @sock.recv(256)
if stop_pattern
if retval =~ stop_pattern
break
end
end
end
end
end
rescue Timeout::Error
timed_out = true
end
<snip>

A reference TFM shows that select(rdArr, <wrArr>, <errArr>, <timeOut>)
returns anArray or nil. [[mySock], [], []] == true returns false (I
suppose).

Hth

Chris
 
C

Chris Reay

Ben Giddings said:
def internal_read(timeout=DEFAULT_TIMEOUT, stop_pattern=nil)
retval = ""

timed_out = false

begin
timeout(timeout) do
while true
if select([@sock], [], [], 0.1)
retval += @sock.recv(256)
if stop_pattern
if retval =~ stop_pattern
break
end
end
end
end
end
rescue Timeout::Error
timed_out = true
end
<snip>

A reference TFM shows that select(rdArr, <wrArr>, <errArr>, <timeOut>)
returns anArray or nil. [[mySock], [], []] == true returns false (I
suppose).

Hth

Chris
 
B

Ben Giddings

On Wed, 9 Jun 2004, Ben Giddings wrote:
didn't you say something about your socket being in non-blocking mode?
if so,
isn't this just a busy loop since select will keep returning 'true - a
read
will not block':

I'm pretty sure that the timeout parameter is what tells select whether
or not to block. I think the nonblocking flag only affects reading
type operations, like 'recv' or 'read'. If you're using blocking
sockets, they wait until there's input, if you're using nonblocking IO,
they return immediately, either with data or without. Select is used
(I think generally with blocking IO) to test to see if there's any
point in attempting a read, to avoid blocking when no data will be
read.

If there's no serial traffic, it should return my socket if there's
input to be read, otherwise it should return nil.
if not in non-blocking mode, perhaps your pattern is bad? what
happens if you
do this:

while true
if select([@sock], [], [], 0.1)
retval += @sock.recv(256)
p retval
</snip>

??

do you see retval growing?

No, in those cases when there's some data there but 'select' doesn't
see it, I don't see anything being appended to 'retval' and I get an
empty string back.

Thanks for the ideas though,

Ben
 
B

Ben Giddings

begin
timeout(timeout) do
while true
if select([@sock], [], [], 0.1)
retval += @sock.recv(256)
if stop_pattern
if retval =~ stop_pattern
break
end
end
end
end
end
rescue Timeout::Error
timed_out = true
end
A reference TFM shows that select(rdArr, <wrArr>, <errArr>, <timeOut>)
returns anArray or nil. [[mySock], [], []] == true returns false (I
suppose).

Well since select() returns anArray or nil, if it returns 'nil', nil
evaluates to false in conditionals, but anything else, other than
'false', evaluates to true. So if it returns an array of arrays, or an
array, or even an empty array, it will evaluate to true:

irb(main):001:0> var = nil
=> nil
irb(main):002:0> if var then puts "#{var.inspect} is true(ish)" end
=> nil
irb(main):003:0> var = false
=> false
irb(main):004:0> if var then puts "#{var.inspect} is true(ish)" end
=> nil
irb(main):005:0> var = true
=> true
irb(main):006:0> if var then puts "#{var.inspect} is true(ish)" end
true is true(ish)
=> nil
irb(main):007:0> var = []
=> []
irb(main):008:0> if var then puts "#{var.inspect} is true(ish)" end
[] is true(ish)
=> nil
irb(main):009:0> require 'socket'
=> true
irb(main):010:0> sock = TCPSocket.new('localhost', 80)
=> #<TCPSocket:0x23d4dc>
irb(main):011:0> var = sock
=> #<TCPSocket:0x23d4dc>
irb(main):012:0> if var then puts "#{var.inspect} is true(ish)" end
#<TCPSocket:0x23d4dc> is true(ish)
=> nil
irb(main):013:0> var = [[sock], [], []]
=> [[#<TCPSocket:0x23d4dc>], [], []]
irb(main):014:0> if var then puts "#{var.inspect} is true(ish)" end
[[#<TCPSocket:0x23d4dc>], [], []] is true(ish)
=> nil
irb(main):015:0> [[sock], [], []] == true
=> false

Ben
 
C

Chris Reay

Ben Giddings said:
Chris said:
irb(main):015:0> [[sock], [], []] == true
=> false

Wow. This *is* hard yakka.

if select([@sock], [], [], 0.1)
@sock.recv()
etc, etc.

Your program is asking the "recv" block to be called if select()
returns true.

We have established the following:

(a) select() returns nil or anArray (from a RTFM).
(b) nil != true (axiom).
(c) anArray != true. (See your irb snippet above).

Thus you are requesting the block never to be called. Your program is
working perfectly, and the socket is brimming with bytes :)

Try this (and pardon my ruby, I've forgotten most of the syntax):

while true
sockArr = select([@sock], [], [], 0.1)
if !sockArr.nil? and sockArr.at(0).include?(@sock)
@sock.recv(256)
etc, etc.

Pardon my didactic tone too. I hope this helps.

Yours

Chris
 
M

Martin DeMello

Chris Reay said:
(c) anArray != true. (See your irb snippet above).

$ ruby -e "p [[nil], [], []] == true"
false

$ ruby -e "p 'true' if [[nil], [], []]"
"true"

martin
 
B

Ben Giddings

I'm going to assume you really misunderstand and aren't trolling,
because even if you are trolling, maybe this explanation will come in
useful for someone else.

Chris said:
if select([@sock], [], [], 0.1)
@sock.recv()
etc, etc.

Your program is asking the "recv" block to be called if select()
returns true.

Actually no. In Ruby, as I showed in my previous message, "if _____"
doesn't test to see if "_____" is true or false, it check to see if it
is true, false, nil, or another object.

If _____ is false, then the conditional is not executed. If _____ is
nil, then the conditional is executed. Otherwise, whether _____ is the
boolean 'true' or whether it is any other non-nil, non-false object, the
conditional is executed.

Conditional execution truth table:

parameter | conditional executed
------------------------------------
false | no
true | yes
nil | no
anything else | yes

And note, that 'anything else' includes some things that may surprise
people used to programming in C, including the number 0, an empty
string, an empty array, etc.

What I'm asking Ruby to do, is to execute that bit of code if the result
of calling select() is not nil.
We have established the following:

(a) select() returns nil or anArray (from a RTFM).
(b) nil != true (axiom).
Right

(c) anArray != true. (See your irb snippet above).

Right, however

anArray = []

if anArray
puts "This is printed because anArray is non-nil and not false"
else
puts "This is not printed."
end
Thus you are requesting the block never to be called. Your program is
working perfectly, and the socket is brimming with bytes :)

I'm confused by your talking about the block. There is a block in that
bit of code, but it isn't inside the conditional.

In this section of code:

if select([@sock], [], [], 0.1)
retval += @sock.recv(256)
if stop_pattern
if retval =~ stop_pattern
break
end
end
end

This bit:

retval += @sock.recv(256)
if stop_pattern
if retval =~ stop_pattern
break
end
end

Will get executed if "select([@sock], [], [], 0.1)" returns something
which isn't nil, and isn't the boolean 'false'. The documentation of
'select' says that it returns nil, or an array. If it returns an array,
that
Try this (and pardon my ruby, I've forgotten most of the syntax):

while true
sockArr = select([@sock], [], [], 0.1)
if !sockArr.nil? and sockArr.at(0).include?(@sock)
@sock.recv(256)
etc, etc.

That code will work, but, as I explained above, it isn't really
necessary. Since the only possible non-nil return value of
"select([@sock], [], [], 0.1)" is a set of arrays including "@sock". I
don't really need to look at the return value. I know that if select()
returns non-nil, my socket will not block if I try to read it.

I hope that convinces you. If not, try playing around with if, and
select() and see if you can figure out what I'm talking about.



I think this does point out a need in Ruby for a 'to_bool' operator.
Obviously, this kind of test already happens behind the scenes when it
comes to conditionals.

All you'd need is the equivalent of:

class Object
def to_bool
true
end
end

class NilClass
def to_bool
false
end
end

class TrueClass
def to_bool
true
end
end

class FalseClass
def to_bool
false
end
end

If nothing else, this would help explain to people why "if 0" takes the
'true' branch. Just point out that 0.to_bool is true

Any opinions? Should I make an RCR?

Ben
 
G

Gennady

Ben said:
I'm going to assume you really misunderstand and aren't trolling,
because even if you are trolling, maybe this explanation will come in
useful for someone else.

Chris said:
if select([@sock], [], [], 0.1)
@sock.recv()
etc, etc.

Your program is asking the "recv" block to be called if select()
returns true.


Actually no. In Ruby, as I showed in my previous message, "if _____"
doesn't test to see if "_____" is true or false, it check to see if it
is true, false, nil, or another object.

If _____ is false, then the conditional is not executed. If _____ is
nil, then the conditional is executed. Otherwise, whether _____ is the
You mistyped here --------> is not executed. ...
 
B

Ben Giddings

Gennady said:
Ben said:
I'm going to assume you really misunderstand and aren't trolling,
because even if you are trolling, maybe this explanation will come in
useful for someone else.

Chris said:
if select([@sock], [], [], 0.1)
@sock.recv()
etc, etc.

Your program is asking the "recv" block to be called if select()
returns true.



Actually no. In Ruby, as I showed in my previous message, "if _____"
doesn't test to see if "_____" is true or false, it check to see if it
is true, false, nil, or another object.

If _____ is false, then the conditional is not executed. If _____ is
nil, then the conditional is executed. Otherwise, whether _____ is the

You mistyped here --------> is not executed. ...

Yup. :)

Good thing I overexplained elsewhere.

Ben
 
C

Chris Reay

Ben Giddings said:
I'm going to assume you really misunderstand and aren't trolling,
because even if you are trolling, maybe this explanation will come in
useful for someone else.

Of course I'm not trolling; don't be so touchy.
Conditional execution truth table:

parameter | conditional executed
------------------------------------
false | no
true | yes
nil | no
anything else | yes

And note, that 'anything else' includes some things that may surprise
people used to programming in C, including the number 0, an empty
string, an empty array, etc.

Well, that does surprise me! But I'm often enough wrong to be used to
it. Now I don't understand why your program doesn't work, but mine
does. I'll think about it.

Remember The PP's tip #26 :)
 
C

Chris Reay

I replied to this a moment ago, and then did the ad hoc experiment
that I should have done before opening my virtual mouth earlier -
sorry 'bout that.

I tested this:

require "socket"

svr = TCPServer.new("localhost", 4567)
sock = svr.accept
printf "Accepted conn\n"
while true
if select([sock], [], [], 3.0)
printf("%s\n", sock.recv(256))
end
end

I sent some "hello world" messages to it from a TCPSocket. It works
perfectly.

Sorry to appear tardy, tedious or trollish; but what gives?
 
B

Ben Giddings

I replied to this a moment ago, and then did the ad hoc experiment
that I should have done before opening my virtual mouth earlier -
sorry 'bout that.

No worries, sorry for implying you might be trolling. I thought that
my earlier example made it clear how the 'if' statement in Ruby works,
but I guess the truth table was even more clear.
I tested this:

require "socket"

svr = TCPServer.new("localhost", 4567)
sock = svr.accept
printf "Accepted conn\n"
while true
if select([sock], [], [], 3.0)
printf("%s\n", sock.recv(256))
end
end

I sent some "hello world" messages to it from a TCPSocket. It works
perfectly.

Sorry to appear tardy, tedious or trollish; but what gives?

My question exactly. For me, it works -- *most* of the time. But
sometimes, especially when I send it only a single newline, that
newline just isn't seen. Select returns nil, but ethereal shows that a
packet has arrived. Hence my original question -- why isn't select()
seeing that newline.

I guess the correct thing to do is rewrite the thing in C and see if
it's a problem in the OS or if it's a problem in Ruby.

Ben
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

No members online now.

Forum statistics

Threads
473,776
Messages
2,569,603
Members
45,197
Latest member
ScottChare

Latest Threads

Top