Nonblocking Sockets

Discussion in 'Ruby' started by James Edward Gray II, Jul 16, 2005.

  1. Is this the "standard" way to make a nonblocking Socket in Ruby?

    require "socket"
    require "fcntl"

    server = TCPServer.new(...)
    server.fcntl(Fcntl::F_SETFL, Fcntl::O_NONBLOCK)

    __END__

    Will that work on Unix and Windows?

    If I accept() a connection from that, will the connecting socket be
    nonblocking as well?

    Thanks for the information.

    James Edward Gray II
     
    James Edward Gray II, Jul 16, 2005
    #1
    1. Advertising

  2. James Edward Gray II

    Bill Kelly Guest

    From: "James Edward Gray II" <>
    >
    > Is this the "standard" way to make a nonblocking Socket in Ruby?
    >
    > require "socket"
    > require "fcntl"
    >
    > server = TCPServer.new(...)
    > server.fcntl(Fcntl::F_SETFL, Fcntl::O_NONBLOCK)
    >
    > __END__
    >
    > Will that work on Unix and Windows?


    As far as I know it's the "standard" way (but that
    just means it's the only way I know of. :)

    It doesn't work on Windows. :(

    I've ended up tacking on: ... if defined? Fcntl::O_NONBLOCK
    onto the fcntl() line so that my scripts still run on
    windows, even though they may block inappropriately.

    Blocking I/O issues have been a thorny issue for me in
    trying to write cross-platform network applications in
    Ruby.

    > If I accept() a connection from that, will the connecting socket be
    > nonblocking as well?


    I seem to recall discussion of a patch to fix something
    related to nonblocking I/O and accept() within the last
    year or so... but I could be wrong... (I couldn't find
    it with Google.)

    I tend to re-issue the nonblock fcntl() call before
    every socket operation I perform.

    Even so, I *still* hit this timeout occasionally:

    if select([sock], nil, nil, UDP_RECV_TIMEOUT)
    begin
    timeout(UDP_RECV_TIMEOUT) {
    sock.fcntl(Fcntl::F_SETFL, Fcntl::O_NONBLOCK) if defined? Fcntl::O_NONBLOCK
    resp = sock.recvfrom(65536)
    }
    rescue Timeout::Error
    $stderr.puts "Timeout::Error in sock.recvfrom !"
    end
    end

    ...Since select() said data was ready, AND since I'm
    requesting a nonblocking operation... I have no idea
    why #recvfrom sometimes hangs. It used to totally hang
    my program (on linux), indefinitely, about once a day,
    until I added the timeout().


    Regards,

    Bill
     
    Bill Kelly, Jul 16, 2005
    #2
    1. Advertising

  3. James Edward Gray II

    Guest

    Hi,

    At Sun, 17 Jul 2005 05:52:34 +0900,
    James Edward Gray II wrote in [ruby-talk:148402]:
    > Is this the "standard" way to make a nonblocking Socket in Ruby?


    Though not a "standard" way, there is io/nonblock module in
    "rough".
    <http://www.ruby-lang.org/cgi-bin/cvsweb.cgi/rough/ext/io/nonblock/>

    > Will that work on Unix and Windows?


    Still not on Windows.

    > If I accept() a connection from that, will the connecting socket be
    > nonblocking as well?


    It may be changed so in the near future.

    --
    Nobu Nakada
     
    , Jul 17, 2005
    #3
  4. James Edward Gray II

    Tanaka Akira Guest

    In article <01a001c58a52$13828780$6442a8c0@musicbox>,
    "Bill Kelly" <> writes:

    > I tend to re-issue the nonblock fcntl() call before
    > every socket operation I perform.
    >
    > Even so, I *still* hit this timeout occasionally:
    >
    > if select([sock], nil, nil, UDP_RECV_TIMEOUT)
    > begin
    > timeout(UDP_RECV_TIMEOUT) {
    > sock.fcntl(Fcntl::F_SETFL, Fcntl::O_NONBLOCK) if defined? Fcntl::O_NONBLOCK
    > resp = sock.recvfrom(65536)
    > }
    > rescue Timeout::Error
    > $stderr.puts "Timeout::Error in sock.recvfrom !"
    > end
    > end
    >
    > ...Since select() said data was ready, AND since I'm
    > requesting a nonblocking operation... I have no idea
    > why #recvfrom sometimes hangs.


    Hmm. Linux, UDP, readable by select, not readable by recvfrom.

    It may be caused by wrong UDP checksum.

    Linux-Kernel Archive: UDP recvmsg blocks after select(), 2.6 bug?
    http://www.ussg.iu.edu/hypermail/linux/kernel/0410.0/1372.html

    Debian Bug report logs - #275585 - /usr/sbin/inetd: UDP builtins can be used to hang inetd
    http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=275585&archive=yes

    select(2):
    Under Linux, select may report a socket file descriptor as "ready for
    reading", while nevertheless a subsequent read blocks. This could for
    example happen when data has arrived but upon examination has wrong
    checksum and is discarded. There may be other circumstances. Thus it
    may be safer to use O_NONBLOCK on sockets that should not block.

    You can test UDP with wrong checksum by hping2.
    See the debian bug report #275585.

    > It used to totally hang
    > my program (on linux), indefinitely, about once a day,
    > until I added the timeout().


    It seems that Ruby process doesn't hang because timeout works.
    timeout is implemented by Ruby thread.

    So your problem is IPSocket#recvfrom retry when EAGAIN.

    You may need lower level method which makes EAGAIN user visible.
    --
    Tanaka Akira
     
    Tanaka Akira, Jul 17, 2005
    #4
  5. James Edward Gray II <> writes:

    > server.fcntl(Fcntl::F_SETFL, Fcntl::O_NONBLOCK)


    that should work in most cases, but perhaps you should have preserved
    any existing values as well, something like:

    server.fcntl(Fcntl::F_SETFL, server.fcntl(Fcntl::F_GETFL) |
    Fcntl::O_NONBLOCK)

    YS.
     
    Yohanes Santoso, Jul 17, 2005
    #5
  6. On Jul 16, 2005, at 9:16 PM, wrote:

    >> If I accept() a connection from that, will the connecting socket be
    >> nonblocking as well?
    >>

    >
    > It may be changed so in the near future.


    So do I need to call it once on each Socket accept() spawns? Will
    once be enough?

    What is the nonblocking behavior of accept? Will it throw an
    Exception, return nil, or what?

    Thanks.

    James Edward Gray II
     
    James Edward Gray II, Jul 17, 2005
    #6
  7. James Edward Gray II

    Guest

    Hi,

    At Sun, 17 Jul 2005 13:10:03 +0900,
    James Edward Gray II wrote in [ruby-talk:148440]:
    > So do I need to call it once on each Socket accept() spawns? Will
    > once be enough?


    Once on each accepted sockets.

    > What is the nonblocking behavior of accept? Will it throw an
    > Exception, return nil, or what?


    Errno::EWOULDBLOCK will be raised.

    --
    Nobu Nakada
     
    , Jul 17, 2005
    #7
  8. James Edward Gray II

    Bill Kelly Guest

    From: "Tanaka Akira" <>
    > In article <01a001c58a52$13828780$6442a8c0@musicbox>,
    > "Bill Kelly" <> writes:
    >
    > > ...Since select() said data was ready, AND since I'm
    > > requesting a nonblocking operation... I have no idea
    > > why #recvfrom sometimes hangs.

    >
    > Hmm. Linux, UDP, readable by select, not readable by recvfrom.
    >
    > It may be caused by wrong UDP checksum.
    >
    > Linux-Kernel Archive: UDP recvmsg blocks after select(), 2.6 bug?
    > http://www.ussg.iu.edu/hypermail/linux/kernel/0410.0/1372.html
    >
    > Debian Bug report logs - #275585 - /usr/sbin/inetd: UDP builtins can be used to hang inetd
    > http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=275585&archive=yes


    Thank you! A most interesting read.

    > > It used to totally hang
    > > my program (on linux), indefinitely, about once a day,
    > > until I added the timeout().

    >
    > It seems that Ruby process doesn't hang because timeout works.
    > timeout is implemented by Ruby thread.
    >
    > So your problem is IPSocket#recvfrom retry when EAGAIN.
    >
    > You may need lower level method which makes EAGAIN user visible.


    Starting with
    http://www.ussg.iu.edu/hypermail/linux/kernel/0410.0/1837.html
    there were some number of messages in the thread saying
    they can't return EAGAIN in this situation because
    POSIX forbids it. (But that's in the nonblocking case.)

    Since I'm requesting O_NONBLOCK prior to recvfrom(),
    shouldn't it ............ ahh "your problem is
    IPSocket#recvfrom retry when EAGAIN." Hmm.....

    Interesting, doesn't that imply IPSocket#recvfrom is
    essentially broken for NONBLOCK semantics? What I mean
    is, if the socket is in nonblocking mode, and recvfrom()
    gets an EAGAIN, shouldn't it just return nil to the
    caller? (Or whatever the TCP methods do in a NONBLOCK
    situation?) Why should it ever "block" in a NONBLOCK
    situation?


    Thanks,

    Regards,

    Bill
     
    Bill Kelly, Jul 17, 2005
    #8
  9. On Sun, Jul 17, 2005 at 07:01:54AM +0900, Bill Kelly wrote:
    > From: "James Edward Gray II" <>
    > >
    > > Is this the "standard" way to make a nonblocking Socket in Ruby?
    > >
    > > require "socket"
    > > require "fcntl"
    > >
    > > server = TCPServer.new(...)
    > > server.fcntl(Fcntl::F_SETFL, Fcntl::O_NONBLOCK)
    > >
    > > __END__
    > >
    > > Will that work on Unix and Windows?

    >
    > As far as I know it's the "standard" way (but that
    > just means it's the only way I know of. :)
    >
    > It doesn't work on Windows. :(
    >
    > I've ended up tacking on: ... if defined? Fcntl::O_NONBLOCK
    > onto the fcntl() line so that my scripts still run on
    > windows, even though they may block inappropriately.
    >
    > Blocking I/O issues have been a thorny issue for me in
    > trying to write cross-platform network applications in
    > Ruby.


    Aside: would it be appropriate to use Ruby threads in your application? It
    can hide a lot of socket blocking nastiness. And I presume the interpreter
    copes with Windows.

    Regards,

    Brian.
     
    Brian Candler, Jul 17, 2005
    #9
  10. On Jul 17, 2005, at 5:41 AM, Brian Candler wrote:

    > Aside: would it be appropriate to use Ruby threads in your
    > application? It
    > can hide a lot of socket blocking nastiness. And I presume the
    > interpreter
    > copes with Windows.


    This works for *some* applications. Unfortunately, I large write can
    still stall the process. I wish it wasn't so, but I believe you
    still need to go with nonblocking IO for a robust setup.

    James Edward Gray II
     
    James Edward Gray II, Jul 17, 2005
    #10
  11. On Jul 17, 2005, at 12:00 AM, wrote:

    > Hi,
    >
    > At Sun, 17 Jul 2005 13:10:03 +0900,
    > James Edward Gray II wrote in [ruby-talk:148440]:
    >
    >> So do I need to call it once on each Socket accept() spawns? Will
    >> once be enough?
    >>

    >
    > Once on each accepted sockets.
    >
    >
    >> What is the nonblocking behavior of accept? Will it throw an
    >> Exception, return nil, or what?
    >>

    >
    > Errno::EWOULDBLOCK will be raised.


    Thanks so much for all this information. You've been very helpful
    and I appreciate it.

    A couple more questions and then I promise I'm done:

    1. Do I then use send() and recvfrom() to write and read data?
    (Other messages in this thread have me nervous about recvfrom().)
    2. Will these also throw Errno::EWOULDBLOCK?

    Thanks again.

    James Edward Gray II
     
    James Edward Gray II, Jul 17, 2005
    #11
  12. James Edward Gray II

    Zed A. Shaw Guest

    James,

    Looks like this has already been explained from the Ruby perspective,
    but thought I'd give you what I know of the "official" information. All
    of this comes from Effective TCP/IP Programming by Jon C. Snader, which
    is probably the absolute best book on TCP/IP programming ever written.
    Go get it or I will hunt you down and paint you up like a clown. :)

    * You should always get the original fcntl settings and reuse them in
    the set operation. Many operating systems use fcntl settings in weird
    ways.

    * You can set O_NONBLOCK on a socket during connect when you want a
    client to attempt a connect, and then go back into your select loop to
    wait for the response. Also useful as a connect timeout. Add a timeout
    to your select and then it will return if the connect..well..times out.
    This has the following requirements though:
    - If the connection is not established during the connect call, then it
    will return an EINPROGRESS error. Not sure what Ruby does with this.
    - If the connect is so fast that it happens immediately then you'll get
    a 0 (success) return code. You need to check for this after the connect
    or else your select will not work. Skip the select if this happens as
    you're already connected.
    - After the connect comes back you'll need to use your previously saved
    fcntl flags to reset it if you don't want nonblocking anymore.

    * Nonblocking accept is usually not needed if you're doing a select/poll
    style loop. The exception to this is if you have so much processing
    that you don't get around to the accept in time to avoid dropping
    clients. In this case either re-write your client handlers to not eat
    up so much time, or set the accept socket to nonblock so that you can do
    multiple accepts in a loop.

    * The practical rule on nonblocking accept is:
    - An OS could return EWOULDBLOCK, ECONNABORTED, EINTR as a non-fatal
    error which just means try again later.
    - All other errno are considered bad and you should deal with them as
    an error.
    - The semantics are different for different platforms, and I believe
    Windows is really weird. You'll need to test on as many as you can, but
    I think if you handle those you'll be good.

    * Select does have a problem with getting the read response right, which
    means you almost always need to set the socket to non-blocking, but also
    select only indicates that you *can* read/write, not how much. So, if
    you need to read/write 1024 bytes, but the OS buffers only allow for 512
    at that moment, then your call will block. Finding out how much you can
    write is only possible on some OS with non-standard fcntl. It's just
    easier to use nonblocking IO. I'm not sure if poll, /dev/poll, epoll,
    or kqueue have a similar problem as select.

    * The major problem with Windows vs. BSD sockets is that Windows uses a
    special SOCKET type rather than an integer file descriptor. Snader's
    book has a nice little wrapper around this, but I believe Ruby handles
    this problem as well. Where you'll run into problems (as I am with the
    libevent extension I'm working on) is when you pass these SOCKETs to an
    external library expecting an integer. Still haven't figured that one
    out.

    Anyway, enjoy.

    Zed A. Shaw

    On Sun, 2005-07-17 at 05:52 +0900, James Edward Gray II wrote:
    <snip>
    > If I accept() a connection from that, will the connecting socket be
    > nonblocking as well?
     
    Zed A. Shaw, Jul 17, 2005
    #12
  13. James Edward Gray II

    Tanaka Akira Guest

    In article <022c01c58a8c$e2e462c0$6442a8c0@musicbox>,
    "Bill Kelly" <> writes:

    > Interesting, doesn't that imply IPSocket#recvfrom is
    > essentially broken for NONBLOCK semantics? What I mean
    > is, if the socket is in nonblocking mode, and recvfrom()
    > gets an EAGAIN, shouldn't it just return nil to the
    > caller? (Or whatever the TCP methods do in a NONBLOCK
    > situation?) Why should it ever "block" in a NONBLOCK
    > situation?


    Ruby's nonblocking behaviour is not well designed.
    So it should be redesigned anyway.

    However nonblocking I/O is used for several reasons.

    There is a case that it is fine that an I/O method hides
    nonblockingness. For example, nonblocking I/O which is used to avoid
    process blocking by write(2) is such case.

    Assume a threaded (non-forking) network server program which use
    IO#read to read a request and IO#write to write a response and it
    doesn't use nonblocking I/O until someone reports DoS problem because
    IO#write may block the server process instead of the thread which
    invokes IO#write. If some I/O methods behaves differently between
    blocking mode and nonblocking mode, it makes hard to fix the DoS
    problem: simply set an socket to nonblocking mode by fcntl may not
    work well because nonblocking method behavior may not appropriate for
    the server program. So all I/O invocation must be examined or
    nonblocking mode must be set only around IO#write method. Note that
    IO#read in Ruby 1.8 behaves differently between the modes.

    So I think it is reasonable that I/O methods hides nonblocking
    behavior: retrying when EAGAIN and partial result. Current Ruby
    doesn't do it well, though.

    I guess there is a case which needs nonblocking behaviour visible from
    script. I'm not sure good design to proviede nonblocking behaviour
    for such script, though.
    --
    Tanaka Akira
     
    Tanaka Akira, Jul 17, 2005
    #13
  14. James Edward Gray II

    Tanaka Akira Guest

    In article <>,
    James Edward Gray II <> writes:

    > This works for *some* applications. Unfortunately, I large write can
    > still stall the process. I wish it wasn't so, but I believe you
    > still need to go with nonblocking IO for a robust setup.


    Ruby threads and nonblocking I/O are not exclusive.

    How about using the threads with nonblocking I/O?
    It should avoid process wide blocking.
    --
    Tanaka Akira
     
    Tanaka Akira, Jul 18, 2005
    #14
  15. James Edward Gray II

    Tanaka Akira Guest

    In article <>,
    "Zed A. Shaw" <> writes:

    > * Select does have a problem with getting the read response right, which
    > means you almost always need to set the socket to non-blocking, but also
    > select only indicates that you *can* read/write, not how much. So, if
    > you need to read/write 1024 bytes, but the OS buffers only allow for 512
    > at that moment, then your call will block. Finding out how much you can


    read 1024bytes doesn't block when OS has 512bytes.
    --
    Tanaka Akira
     
    Tanaka Akira, Jul 18, 2005
    #15
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. KimTaehwan

    setSoTimeout in nonblocking mode?

    KimTaehwan, Nov 16, 2003, in forum: Java
    Replies:
    0
    Views:
    1,145
    KimTaehwan
    Nov 16, 2003
  2. nonblocking sockets

    , Apr 7, 2006, in forum: Java
    Replies:
    2
    Views:
    1,103
    Gordon Beaton
    Apr 8, 2006
  3. Peter Ammon

    nonblocking read()

    Peter Ammon, Nov 16, 2004, in forum: Python
    Replies:
    3
    Views:
    567
    Donn Cave
    Nov 17, 2004
  4. anki
    Replies:
    3
    Views:
    315
  5. rakaur
    Replies:
    4
    Views:
    619
    Bill Kelly
    Feb 14, 2006
Loading...

Share This Page