Windows - Socket - Connect - Nonblocking

Discussion in 'Ruby' started by Jean-Francois Nadeau, Jul 6, 2004.

  1. Hi,

    Ruby1.8.1, native ruby (from the pragmatic guys), WinXP.

    I am building a very nice Windows RSS aggregator which some of it is written
    in Ruby. I really like Ruby because it makes me very productive. Nice work.

    In my software, I need to pull the various RSS feeds as fast as I can.
    Therefore, I create a thread pool to minimize the network latency of each
    feed.

    It seemed at first that the IO was not multi-thread enabled because the ruby
    process unexpectedly became idle sometimes and after a few seconds, it
    started back running.

    I decided to investigate the problem by looking at the ruby source code. I
    saw that because the thread scheduler works with "select()", there should
    not be any blocking in multi-threaded socket IO.

    I put some traces in the ruby C code and discovered that the socket connect
    call is not nonblocking in Windows.

    Here is a ruby snippet to try it:

    require 'net/http'

    t= Thread.new do
    while 1
    puts "in thread"
    end
    end


    h = Net::HTTP.new('192.168.0.80', 80)

    t.join

    Replace 192.168.0.80 with your bogus IP and you will see that "in thread"
    messages do not appear while ruby enters Net::HTTP.new.

    So, I was wondering if this was a known problem. I have modified the ruby
    code to fix that problem but I don't know if my patch is 100% safe. Where
    should I send my patch and how?

    My patch definitely improves the performance of my code.

    Regards,

    Jean-Francois Nadeau
    http://www.jfnadeau.com
     
    Jean-Francois Nadeau, Jul 6, 2004
    #1
    1. Advertising

  2. il Tue, 6 Jul 2004 21:50:40 +0900, "Jean-Francois Nadeau"
    <> ha scritto::


    >I decided to investigate the problem by looking at the ruby source code. I
    >saw that because the thread scheduler works with "select()", there should
    >not be any blocking in multi-threaded socket IO.


    select() on windows is just able to handle FD representing sockets,
    baed on things I heard here.. But for thos it should work.. :/
    Bad things happen when you mix standard IO and socket stuff, which I
    believe is, well, everytime.

    Pathc would be much appreciated, I think.

    >So, I was wondering if this was a known problem. I have modified the ruby
    >code to fix that problem but I don't know if my patch is 100% safe. Where
    >should I send my patch and how?
    >
    >My patch definitely improves the performance of my code.


    try here or on the ruby-core mailing list.

    OTOH, ou may just try your patched interpreter against the standard
    test suite, as a first try.
     
    gabriele renzi, Jul 6, 2004
    #2
    1. Advertising

  3. > >So, I was wondering if this was a known problem. I have modified the ruby
    > >code to fix that problem but I don't know if my patch is 100% safe. Where
    > >should I send my patch and how?
    > >
    > >My patch definitely improves the performance of my code.

    >
    > try here or on the ruby-core mailing list.
    >
    > OTOH, ou may just try your patched interpreter against the standard
    > test suite, as a first try.


    I ran nmake test and it is successful.

    However, in dir "test", I don't see a socket unit test. I will clean my code
    and send a patch shortly.

    One thing that would help me is to know the location in the ruby source code
    where a socket is put in nonblocking mode. I did not see it in the socket
    creation part...

    In ruby_connect (socket.c I think), fcntl is called to make it non-blocking
    but that does not compile in Win32. (fcntl does not exist.)

    My quick patch is to put the Win32 socket in non-blocking mode with
    ioctlsocket() after the ::socket call. Then, I have to trap WSAEWOULDBLOCK
    in Win32 connect (win32.c) and return instead EINPROGRESS. Then everything
    seems to work very well. (ruby_connect gives the FD to the thread
    scheduler...)

    Does a socket is in non-blocking mode by default in Ruby?

    Jean-Francois Nadeau
    http://www.jfnadeau.com
     
    Jean-Francois Nadeau, Jul 6, 2004
    #3
  4. Hello gabriele,

    gr> il Tue, 6 Jul 2004 21:50:40 +0900, "Jean-Francois Nadeau"
    gr> <> ha scritto::


    >>I decided to investigate the problem by looking at the ruby source code. I
    >>saw that because the thread scheduler works with "select()", there should
    >>not be any blocking in multi-threaded socket IO.


    Yes i hope that some persons of the ruby core team will fix this.
    Having such a huge bug and call ruby platform independent is something
    that does not fit together.

    Technically the code must be factored out in an abstract event loop,
    that also allows to play nicely with GUI interfaces. Only TCL and Perl
    have a good working event loop at the moment.


    --
    Best regards, emailto: scholz at scriptolutions dot com
    Lothar Scholz http://www.ruby-ide.com
    CTO Scriptolutions Ruby, PHP, Python IDE 's
     
    Lothar Scholz, Jul 6, 2004
    #4
  5. Hi,

    For those interested by this thread, I have just sent my patch to the
    ruby-core mailing list.

    Regards,

    Jean-Francois Nadeau
    http://www.jfnadeau.com
     
    Jean-Francois Nadeau, Jul 8, 2004
    #5
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Laszlo Nagy
    Replies:
    1
    Views:
    4,914
    Mark Wooding
    Jan 27, 2009
  2. Jean-Paul Calderone
    Replies:
    0
    Views:
    990
    Jean-Paul Calderone
    Jan 27, 2009
  3. Laszlo Nagy
    Replies:
    0
    Views:
    564
    Laszlo Nagy
    Feb 1, 2009
  4. Steve Holden
    Replies:
    0
    Views:
    681
    Steve Holden
    Feb 1, 2009
  5. Angus
    Replies:
    3
    Views:
    136
    Angus
    Aug 29, 2003
Loading...

Share This Page