Announcing Rev: a high performance event library for Ruby 1.9

Discussion in 'Ruby' started by Tony Arcieri, Dec 26, 2007.

  1. Tony Arcieri

    Tony Arcieri Guest

    [Note: parts of this message were removed to make it a legal post.]

    Hello everyone,

    I've just made my initial public release of Rev, a high performance event
    library for Ruby 1.9 built on top of libev. Events are monitored with
    advanced system calls including epoll on Linux, kqueues on BSD, and event
    ports on Solaris. You can read more about libev here:
    http://software.schmorp.de/pkg/libev.html

    The release has not yet reached the gem mirrors, so you can grab it here for
    the time being:

    http://rev.rubyforge.org/releases/gems/rev-0.1.0.gem

    Rev gives you the ability to monitor Ruby IO objects for readability and/or
    writability. It also lets you configure either one-shot or periodic
    timers. Future plans include wrappers for monitoring signals as well as
    filesystem events.

    Rev makes use of a number of new Ruby 1.9 features, namely the ability to do
    blocking calls inside of native threads using the
    rb_thread_blocking_region() function. The entire event loop is written with
    scalability in mind, with nearly constant time overhead compared to the
    number of events being monitored, at least on Linux, FreeBSD, or Solaris.

    In addition to a C extension which provides Ruby bindings to libev, Rev also
    provides asynchronous observer-like wrappers for Ruby's core socket classes
    using the new Ruby 1.9 *_nonblock methods. It also provides asynchronous
    DNS resolution for the purpose of making outgoing connections.

    You can view the RDoc here:

    http://rev.rubyforge.org/

    Enjoy!

    --
    Tony Arcieri
    ClickCaster, Inc.
     
    Tony Arcieri, Dec 26, 2007
    #1
    1. Advertising

  2. Tony Arcieri

    Thomas Hurst Guest

    * Tony Arcieri () wrote:

    > I've just made my initial public release of Rev, a high performance
    > event library for Ruby 1.9 built on top of libev. Events are monitored
    > with advanced system calls including epoll on Linux, kqueues on BSD,
    > and event ports on Solaris.


    Yum, thanks :)

    Buffered writing seems to be broken; the buffer is sliced incorrectly,
    so it spins writing the last few KB, and the Watcher doesn't have an
    enabled? method so that fails if it ever reaches it. Patch:

    http://voi.aagh.net/rev-0.1.0_lib-rev-buffered_io.rb.diff

    Few other issues: spurious closed connections - if I do one big write
    I'm ok, but splitting a 500k buffer into 4k writes almost always results
    in a closed stream IOError on the second write, and a ECONNRESET on the
    reading side.

    Servers don't serve much; they serve one client and exit even in the
    exception-free case, including the example echo server. Multiple
    clients will be served if they manage to connect before the rest close,
    but once the listen queue's empty Rev::Loop#run returns.

    This is tested with FreeBSD 6.2, using kqueue, select and poll.

    --
    Thomas 'Freaky' Hurst
    http://hur.st/
     
    Thomas Hurst, Dec 28, 2007
    #2
    1. Advertising

  3. Tony Arcieri

    Tony Arcieri Guest

    [Note: parts of this message were removed to make it a legal post.]

    On Dec 28, 2007 2:05 PM, Thomas Hurst <> wrote:

    > Buffered writing seems to be broken; the buffer is sliced incorrectly,
    > so it spins writing the last few KB



    Heh, I caught that one earlier today and a fix is checked in. Thanks for
    pointing it out.


    > and the Watcher doesn't have an enabled? method so that fails if it ever
    > reaches it. Patch:
    >
    > http://voi.aagh.net/rev-0.1.0_lib-rev-buffered_io.rb.diff
    >


    Thanks, I'll get that addressed as well. You might notice the painful lack
    of tests... I'd like to use RSpec but that's unfortunately broken by
    1.9changes at the moment. Perhaps I could use Rubinius's minispec if
    that's
    working on 1.9, but my guess is I'll have to end up using Test::Unit for the
    time being.


    > Few other issues: spurious closed connections - if I do one big write
    > I'm ok, but splitting a 500k buffer into 4k writes almost always results
    > in a closed stream IOError on the second write, and a ECONNRESET on the
    > reading side.
    >


    I may have a fix for that shortly... I'll ping you off list if I do


    > Servers don't serve much; they serve one client and exit even in the
    > exception-free case, including the example echo server. Multiple
    > clients will be served if they manage to connect before the rest close,
    > but once the listen queue's empty Rev::Loop#run returns.



    > This is tested with FreeBSD 6.2, using kqueue, select and poll.
    >


    Bizzare. I don't presently have a FreeBSD system, although I plan on
    installing FreeBSD later tonight actually, so I can test there.

    I'm using kqueue on MacOS X and have not experienced that problem, nor have
    I experienced the problem with Linux/epoll.

    I'm curious how that's even happening as the listen socket should be open
    and the Rev::Listener object should still be attached to the loop.

    --
    Tony Arcieri
    ClickCaster, Inc.
     
    Tony Arcieri, Dec 28, 2007
    #3
  4. Tony Arcieri

    Thomas Hurst Guest

    * Tony Arcieri () wrote:

    > Thanks, I'll get that addressed as well. You might notice the painful
    > lack of tests... I'd like to use RSpec but that's unfortunately broken
    > by 1.9changes at the moment. Perhaps I could use Rubinius's minispec
    > if that's working on 1.9, but my guess is I'll have to end up using
    > Test::Unit for the time being.


    mspec seems to work. After changing target in mspec/scripts/mspec.rb
    =66rom shotgun/rubinius to ruby19, I can do:

    -% ruby19 bin/mspec spec/core/fixnum/*
    Started
    .......................F....................... (etc)

    > > Few other issues: spurious closed connections [..]

    > I may have a fix for that shortly... I'll ping you off list if I do


    Thanks.

    > > Servers don't serve much; they serve one client and exit even in the
    > > exception-free case, including the example echo server. Multiple
    > > clients will be served if they manage to connect before the rest
    > > close, but once the listen queue's empty Rev::Loop#run returns.

    >=20
    > Bizzare. I don't presently have a FreeBSD system, although I plan on
    > installing FreeBSD later tonight actually, so I can test there.


    I just tried on my laptop which is running Ubuntu; behavior's identical
    to FreeBSD with epoll and select.

    Also, :backend =3D> :kqueue makes it crash horribly there ;)

    > I'm using kqueue on MacOS X and have not experienced that problem, nor
    > have I experienced the problem with Linux/epoll.
    >=20
    > I'm curious how that's even happening as the listen socket should be
    > open and the Rev::Listener object should still be attached to the
    > loop.


    Further testing shows this doesn't happen if I use small writes, so it
    seems to be related to buffering. I can't see any immediately obvious
    reason why, though.

    --=20
    Thomas 'Freaky' Hurst
    http://hur.st/
     
    Thomas Hurst, Dec 29, 2007
    #4
  5. Tony Arcieri

    Tony Arcieri Guest

    [Note: parts of this message were removed to make it a legal post.]

    On Dec 28, 2007 6:21 PM, Thomas Hurst <> wrote:

    >
    > mspec seems to work. After changing target in mspec/scripts/mspec.rb
    > from shotgun/rubinius to ruby19, I can do:
    >
    > -% ruby19 bin/mspec spec/core/fixnum/*
    > Started
    > .......................F....................... (etc)
    >


    Nice, I'll get that installed...

    I just tried on my laptop which is running Ubuntu; behavior's identical
    > to FreeBSD with epoll and select.
    > Further testing shows this doesn't happen if I use small writes, so it
    > seems to be related to buffering. I can't see any immediately obvious
    > reason why, though.
    >


    Running off the svn trunk version of Rev I get:

    rev/lib/rev/buffered_io.rb:44:in `write_nonblock': Resource temporarily
    unavailable (Errno::EAGAIN)
    rev/lib/rev/buffered_io.rb:44:in `write'

    Which should be relatively trivial to fix. That exception can probably be
    safely swallowed, although that may not be the greatest thing from a
    performance perspective.

    If you try the version of Rev in trunk do you get the same thing?

    --
    Tony Arcieri
    ClickCaster, Inc.
     
    Tony Arcieri, Dec 29, 2007
    #5
  6. Tony Arcieri

    Tony Arcieri Guest

    [Note: parts of this message were removed to make it a legal post.]

    On Dec 29, 2007 11:13 AM, Tony Arcieri <> wrote:

    > Running off the svn trunk version of Rev I get:
    >
    > rev/lib/rev/buffered_io.rb:44:in `write_nonblock': Resource temporarily
    > unavailable (Errno::EAGAIN)
    > rev/lib/rev/buffered_io.rb:44:in `write'
    >
    > Which should be relatively trivial to fix. That exception can probably be
    > safely swallowed, although that may not be the greatest thing from a
    > performance perspective.
    >


    Yeah, that's a bit of a snag. Do you know how to do a nonblocking write
    that swallows that exception and gets the return value from write(2)?

    --
    Tony Arcieri
    ClickCaster, Inc.
     
    Tony Arcieri, Dec 29, 2007
    #6
  7. Tony Arcieri

    Tony Arcieri Guest

    [Note: parts of this message were removed to make it a legal post.]

    On Dec 29, 2007 12:57 PM, Tony Arcieri <> wrote:

    > Yeah, that's a bit of a snag. Do you know how to do a nonblocking write
    > that swallows that exception and gets the return value from write(2)?
    >


    I released Rev 0.1.1 (should be available through the gem servers now) which
    resolves these problems, and also bundles an asynchronous HTTP client.

    Give it a try and let me know if it fixes your problems.

    --
    Tony Arcieri
    ClickCaster, Inc.
     
    Tony Arcieri, Jan 1, 2008
    #7
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Mark Hahn
    Replies:
    22
    Views:
    672
    Carlos Ribeiro
    Sep 9, 2004
  2. Tony Arcieri
    Replies:
    1
    Views:
    157
    Roger Pack
    Feb 20, 2008
  3. Tony Arcieri
    Replies:
    1
    Views:
    138
    Roger Pack
    Feb 2, 2009
  4. Tony Arcieri
    Replies:
    14
    Views:
    455
    Tony Arcieri
    Dec 9, 2010
  5. Tony Arcieri
    Replies:
    0
    Views:
    410
    Tony Arcieri
    Dec 14, 2010
Loading...

Share This Page