Windows - Socket - Connect - Nonblocking

[Jean-Francois Nadeau]

Hi,

Ruby1.8.1, native ruby (from the pragmatic guys), WinXP.

I am building a very nice Windows RSS aggregator which some of it is written
in Ruby. I really like Ruby because it makes me very productive. Nice work.

In my software, I need to pull the various RSS feeds as fast as I can.
Therefore, I create a thread pool to minimize the network latency of each
feed.

It seemed at first that the IO was not multi-thread enabled because the ruby
process unexpectedly became idle sometimes and after a few seconds, it
started back running.

I decided to investigate the problem by looking at the ruby source code. I
saw that because the thread scheduler works with "select()", there should
not be any blocking in multi-threaded socket IO.

I put some traces in the ruby C code and discovered that the socket connect
call is not nonblocking in Windows.

Here is a ruby snippet to try it:

require 'net/http'

t= Thread.new do
while 1
puts "in thread"
end
end


h = Net::HTTP.new('192.168.0.80', 80)

t.join

Replace 192.168.0.80 with your bogus IP and you will see that "in thread"
messages do not appear while ruby enters Net::HTTP.new.

So, I was wondering if this was a known problem. I have modified the ruby
code to fix that problem but I don't know if my patch is 100% safe. Where
should I send my patch and how?

My patch definitely improves the performance of my code.

Regards,

Jean-Francois Nadeau
http://www.jfnadeau.com

 

[gabriele renzi]

I decided to investigate the problem by looking at the ruby source code. I
saw that because the thread scheduler works with "select()", there should
not be any blocking in multi-threaded socket IO.

select() on windows is just able to handle FD representing sockets,
baed on things I heard here.. But for thos it should work.. :/
Bad things happen when you mix standard IO and socket stuff, which I
believe is, well, everytime.

Pathc would be much appreciated, I think.

So, I was wondering if this was a known problem. I have modified the ruby
code to fix that problem but I don't know if my patch is 100% safe. Where
should I send my patch and how?

My patch definitely improves the performance of my code.

try here or on the ruby-core mailing list.

OTOH, ou may just try your patched interpreter against the standard
test suite, as a first try.

 

[Jean-Francois Nadeau]

So, I was wondering if this was a known problem. I have modified the ruby

try here or on the ruby-core mailing list.

OTOH, ou may just try your patched interpreter against the standard
test suite, as a first try.

I ran nmake test and it is successful.

However, in dir "test", I don't see a socket unit test. I will clean my code
and send a patch shortly.

One thing that would help me is to know the location in the ruby source code
where a socket is put in nonblocking mode. I did not see it in the socket
creation part...

In ruby_connect (socket.c I think), fcntl is called to make it non-blocking
but that does not compile in Win32. (fcntl does not exist.)

My quick patch is to put the Win32 socket in non-blocking mode with
ioctlsocket() after the ::socket call. Then, I have to trap WSAEWOULDBLOCK
in Win32 connect (win32.c) and return instead EINPROGRESS. Then everything
seems to work very well. (ruby_connect gives the FD to the thread
scheduler...)

Does a socket is in non-blocking mode by default in Ruby?

Jean-Francois Nadeau
http://www.jfnadeau.com

 

[Lothar Scholz]

Hello gabriele,

gr> il Tue, 6 Jul 2004 21:50:40 +0900, "Jean-Francois Nadeau"

Yes i hope that some persons of the ruby core team will fix this.
Having such a huge bug and call ruby platform independent is something
that does not fit together.

Technically the code must be factored out in an abstract event loop,
that also allows to play nicely with GUI interfaces. Only TCL and Perl
have a good working event loop at the moment.

 

[Jean-Francois Nadeau]

Hi,

For those interested by this thread, I have just sent my patch to the
ruby-core mailing list.

Regards,

Jean-Francois Nadeau
http://www.jfnadeau.com