Announcing Rev: a high performance event library for Ruby 1.9

[Tony Arcieri]

[Note: parts of this message were removed to make it a legal post.]

Hello everyone,

I've just made my initial public release of Rev, a high performance event
library for Ruby 1.9 built on top of libev. Events are monitored with
advanced system calls including epoll on Linux, kqueues on BSD, and event
ports on Solaris. You can read more about libev here:
http://software.schmorp.de/pkg/libev.html

The release has not yet reached the gem mirrors, so you can grab it here for
the time being:

http://rev.rubyforge.org/releases/gems/rev-0.1.0.gem

Rev gives you the ability to monitor Ruby IO objects for readability and/or
writability. It also lets you configure either one-shot or periodic
timers. Future plans include wrappers for monitoring signals as well as
filesystem events.

Rev makes use of a number of new Ruby 1.9 features, namely the ability to do
blocking calls inside of native threads using the
rb_thread_blocking_region() function. The entire event loop is written with
scalability in mind, with nearly constant time overhead compared to the
number of events being monitored, at least on Linux, FreeBSD, or Solaris.

In addition to a C extension which provides Ruby bindings to libev, Rev also
provides asynchronous observer-like wrappers for Ruby's core socket classes
using the new Ruby 1.9 *_nonblock methods. It also provides asynchronous
DNS resolution for the purpose of making outgoing connections.

You can view the RDoc here:

http://rev.rubyforge.org/

Enjoy!

 

[Thomas Hurst]

I've just made my initial public release of Rev, a high performance
event library for Ruby 1.9 built on top of libev. Events are monitored
with advanced system calls including epoll on Linux, kqueues on BSD,
and event ports on Solaris.

Yum, thanks

Buffered writing seems to be broken; the buffer is sliced incorrectly,
so it spins writing the last few KB, and the Watcher doesn't have an
enabled? method so that fails if it ever reaches it. Patch:

http://voi.aagh.net/rev-0.1.0_lib-rev-buffered_io.rb.diff

Few other issues: spurious closed connections - if I do one big write
I'm ok, but splitting a 500k buffer into 4k writes almost always results
in a closed stream IOError on the second write, and a ECONNRESET on the
reading side.

Servers don't serve much; they serve one client and exit even in the
exception-free case, including the example echo server. Multiple
clients will be served if they manage to connect before the rest close,
but once the listen queue's empty Rev::Loop#run returns.

This is tested with FreeBSD 6.2, using kqueue, select and poll.

 

[Tony Arcieri]

[Note: parts of this message were removed to make it a legal post.]

Buffered writing seems to be broken; the buffer is sliced incorrectly,
so it spins writing the last few KB

Heh, I caught that one earlier today and a fix is checked in. Thanks for
pointing it out.

and the Watcher doesn't have an enabled? method so that fails if it ever
reaches it. Patch:

http://voi.aagh.net/rev-0.1.0_lib-rev-buffered_io.rb.diff

Thanks, I'll get that addressed as well. You might notice the painful lack
of tests... I'd like to use RSpec but that's unfortunately broken by
1.9changes at the moment. Perhaps I could use Rubinius's minispec if
that's
working on 1.9, but my guess is I'll have to end up using Test::Unit for the
time being.

Few other issues: spurious closed connections - if I do one big write
I'm ok, but splitting a 500k buffer into 4k writes almost always results
in a closed stream IOError on the second write, and a ECONNRESET on the
reading side.

I may have a fix for that shortly... I'll ping you off list if I do

Servers don't serve much; they serve one client and exit even in the
exception-free case, including the example echo server. Multiple
clients will be served if they manage to connect before the rest close,
but once the listen queue's empty Rev::Loop#run returns.

This is tested with FreeBSD 6.2, using kqueue, select and poll.

Bizzare. I don't presently have a FreeBSD system, although I plan on
installing FreeBSD later tonight actually, so I can test there.

I'm using kqueue on MacOS X and have not experienced that problem, nor have
I experienced the problem with Linux/epoll.

I'm curious how that's even happening as the listen socket should be open
and the Rev::Listener object should still be attached to the loop.

 

[Thomas Hurst]

Thanks, I'll get that addressed as well. You might notice the painful
lack of tests... I'd like to use RSpec but that's unfortunately broken
by 1.9changes at the moment. Perhaps I could use Rubinius's minispec
if that's working on 1.9, but my guess is I'll have to end up using
Test::Unit for the time being.

mspec seems to work. After changing target in mspec/scripts/mspec.rb
=66rom shotgun/rubinius to ruby19, I can do:

-% ruby19 bin/mspec spec/core/fixnum/*
Started
.......................F....................... (etc)

Few other issues: spurious closed connections [..]

I may have a fix for that shortly... I'll ping you off list if I do
Thanks.

Servers don't serve much; they serve one client and exit even in the
exception-free case, including the example echo server. Multiple
clients will be served if they manage to connect before the rest
close, but once the listen queue's empty Rev::Loop#run returns.

=20
Bizzare. I don't presently have a FreeBSD system, although I plan on
installing FreeBSD later tonight actually, so I can test there.

I just tried on my laptop which is running Ubuntu; behavior's identical
to FreeBSD with epoll and select.

Also, :backend =3D> :kqueue makes it crash horribly there

I'm using kqueue on MacOS X and have not experienced that problem, nor
have I experienced the problem with Linux/epoll.
=20
I'm curious how that's even happening as the listen socket should be
open and the Rev::Listener object should still be attached to the
loop.

Further testing shows this doesn't happen if I use small writes, so it
seems to be related to buffering. I can't see any immediately obvious
reason why, though.

--=20
Thomas 'Freaky' Hurst
http://hur.st/

 

[Tony Arcieri]

[Note: parts of this message were removed to make it a legal post.]

mspec seems to work. After changing target in mspec/scripts/mspec.rb
from shotgun/rubinius to ruby19, I can do:

-% ruby19 bin/mspec spec/core/fixnum/*
Started
.......................F....................... (etc)

Nice, I'll get that installed...

I just tried on my laptop which is running Ubuntu; behavior's identical

to FreeBSD with epoll and select.
Further testing shows this doesn't happen if I use small writes, so it
seems to be related to buffering. I can't see any immediately obvious
reason why, though.

Running off the svn trunk version of Rev I get:

rev/lib/rev/buffered_io.rb:44:in `write_nonblock': Resource temporarily
unavailable (Errno::EAGAIN)
rev/lib/rev/buffered_io.rb:44:in `write'

Which should be relatively trivial to fix. That exception can probably be
safely swallowed, although that may not be the greatest thing from a
performance perspective.

If you try the version of Rev in trunk do you get the same thing?

 

[Tony Arcieri]

[Note: parts of this message were removed to make it a legal post.]

Running off the svn trunk version of Rev I get:

rev/lib/rev/buffered_io.rb:44:in `write_nonblock': Resource temporarily
unavailable (Errno::EAGAIN)
rev/lib/rev/buffered_io.rb:44:in `write'

Which should be relatively trivial to fix. That exception can probably be
safely swallowed, although that may not be the greatest thing from a
performance perspective.

Yeah, that's a bit of a snag. Do you know how to do a nonblocking write
that swallows that exception and gets the return value from write(2)?

 

[Tony Arcieri]

[Note: parts of this message were removed to make it a legal post.]

Yeah, that's a bit of a snag. Do you know how to do a nonblocking write
that swallows that exception and gets the return value from write(2)?

I released Rev 0.1.1 (should be available through the gem servers now) which
resolves these problems, and also bundles an asynchronous HTTP client.

Give it a try and let me know if it fixes your problems.