D
Daniel Brockman
Hi list,
Due to the somewhat popular demand of an event loop for Ruby,
I've recently been working on packaging the one I've written
for a network application of mine (Refusde, an NMDC client).
With the help of Tilman Sauerbeck, I've now managed to put
together some documentation and a gem/tarball of what I've
decided is going to be the first publicly announced version.
Here's the canonical short package overview:
EventLoop is a simple IO::select-based main event loop
featuring IO event notification and timeout callbacks.
It comes with a signal system inspired by that of GLib.
The code is licensed under the GPL and can be found at
<http://www.brockman.se/software/ruby-event-loop/>.
At this point, some of you will probably want to see an
example of how it works --- a kind of screenshot.
For this purpose, I chose to implement a simple asynchronous
buffered IO reader:
require "event-loop"
class BufferedReader
include SignalEmitter
define_signals :line, :done
def initialize(io, eol=3D"\n")
yield self if block_given?
io =3D File.new(io) if io.kind_of? String
buffer =3D String.new
io.on_readable do
begin
buffer << io.readpartial(1024)
while i =3D buffer.index(eol)
signal :line, buffer.slice!(0, i)
buffer.slice!(0, eol.size)
end
rescue EOFError
signal :done, buffer
io.close
end
end
end
end
reader =3D BufferedReader.new("/etc/passwd") do |r|
r.on_line { |content| puts "Line: #{content}" }
r.on_done { |leftover| puts "Done: #{leftover}" }
r.on_done { EventLoop.quit }
end
EventLoop.run
See how easy the event loop is to use, and how nicely it
blends into the rest of Ruby?
For good measure, maybe I should also attach a section of
the manual (i.e., the README file) that describes how event
loops fit into the rest of the world:
The Event Loop
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
This section explains how IO multiplexing works in general
(albeit briefly and not very in-depth), and specifically the
issues relevant for Ruby applications. You may safely skip
it if you (a) already know this subject, or (b) don't care.
Plain ol' blocking IO works well when you're reading from
just a single file descriptor. But when you're interested
in a whole bunch of FDs, you can't wait for any single one
of them to become readable or writable, because then you'll
inevitably miss that happening to the other ones. Instead,
you need a multiplexer that can wait for them *all at once*.
There are a handful of low-level multiplexing primitives:
=E2=80=98select=E2=80=99, =E2=80=98poll=E2=80=99, =E2=80=98epoll=E2=80=99=
, =E2=80=98/dev/poll=E2=80=99, and =E2=80=98kqueue=E2=80=99.
In addition, there are portable low-level wrapper libraries
such as libevent, which can use any of those primitives.
The event loop in this package uses the standard =E2=80=98select=E2=80=99
wrapper shipped with Ruby, =E2=80=98IO::select=E2=80=99. But in the futu=
re,
I'd like to use libevent instead, because that'd be cooler.
Most applications use a higher-level abstraction built on
top of the low-level multiplexer, usually called a =E2=80=98main
loop=E2=80=99, an =E2=80=98event loop=E2=80=99, or an =E2=80=98event sour=
ce=E2=80=99. There are
also libraries such as liboop, which generalizes the event
source and event sink concepts, so that components (event
sinks) written against liboop become event-source-agnostic.
Actually, the combination of blocking IO and Ruby's green
threads works well in most cases where you would normally
use an event loop. When you call =E2=80=98IO#read=E2=80=99 on an empty f=
ile
descriptor, for instance, Ruby suspends that thread until
its internal event loop, known as the scheduler (currently
based on =E2=80=98select=E2=80=99), determines that the file descriptor h=
as
become readable. In particular, Ruby never calls the
low-level =E2=80=98read=E2=80=99 function unless it knows that it will no=
t
block (because =E2=80=98select=E2=80=99 said it wouldn't, but see below).
There are several reasons why you would use an event loop
such as the one implemented by this library instead of
not-so-plain ol' blocking IO with Ruby's green threads.
First of all, you may consider the event loop API more
pleasant than Ruby's threads and not-quite-blocking IO.
Otherwise, don't listen to me; go on using the latter.
Blocking IO can occasionally cause unexpected problems.
For example, in some cases a blocking read *can* block even
though select said that the file descriptor was readable.
This problem may be rare (it can happen, for instance, when
the checksum of a piece of data fails to match the payload),
but the bottom line is that non-blocking IO is safer.
Perhaps most importantly, while Ruby's threads are green,
they are still effectively preemptively scheduled, with all
the implications thereof =E2=80=94 in a word, synchronization hell.
By contrast, event handlers are executed in a strictly
sequential manner; an event loop will never run two event
handlers simultaneously. (Though, of course, all bets are
off if you run multiple event loops in separate threads.)
--=20
Daniel Brockman <[email protected]>
Due to the somewhat popular demand of an event loop for Ruby,
I've recently been working on packaging the one I've written
for a network application of mine (Refusde, an NMDC client).
With the help of Tilman Sauerbeck, I've now managed to put
together some documentation and a gem/tarball of what I've
decided is going to be the first publicly announced version.
Here's the canonical short package overview:
EventLoop is a simple IO::select-based main event loop
featuring IO event notification and timeout callbacks.
It comes with a signal system inspired by that of GLib.
The code is licensed under the GPL and can be found at
<http://www.brockman.se/software/ruby-event-loop/>.
At this point, some of you will probably want to see an
example of how it works --- a kind of screenshot.
For this purpose, I chose to implement a simple asynchronous
buffered IO reader:
require "event-loop"
class BufferedReader
include SignalEmitter
define_signals :line, :done
def initialize(io, eol=3D"\n")
yield self if block_given?
io =3D File.new(io) if io.kind_of? String
buffer =3D String.new
io.on_readable do
begin
buffer << io.readpartial(1024)
while i =3D buffer.index(eol)
signal :line, buffer.slice!(0, i)
buffer.slice!(0, eol.size)
end
rescue EOFError
signal :done, buffer
io.close
end
end
end
end
reader =3D BufferedReader.new("/etc/passwd") do |r|
r.on_line { |content| puts "Line: #{content}" }
r.on_done { |leftover| puts "Done: #{leftover}" }
r.on_done { EventLoop.quit }
end
EventLoop.run
See how easy the event loop is to use, and how nicely it
blends into the rest of Ruby?
For good measure, maybe I should also attach a section of
the manual (i.e., the README file) that describes how event
loops fit into the rest of the world:
The Event Loop
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
This section explains how IO multiplexing works in general
(albeit briefly and not very in-depth), and specifically the
issues relevant for Ruby applications. You may safely skip
it if you (a) already know this subject, or (b) don't care.
Plain ol' blocking IO works well when you're reading from
just a single file descriptor. But when you're interested
in a whole bunch of FDs, you can't wait for any single one
of them to become readable or writable, because then you'll
inevitably miss that happening to the other ones. Instead,
you need a multiplexer that can wait for them *all at once*.
There are a handful of low-level multiplexing primitives:
=E2=80=98select=E2=80=99, =E2=80=98poll=E2=80=99, =E2=80=98epoll=E2=80=99=
, =E2=80=98/dev/poll=E2=80=99, and =E2=80=98kqueue=E2=80=99.
In addition, there are portable low-level wrapper libraries
such as libevent, which can use any of those primitives.
The event loop in this package uses the standard =E2=80=98select=E2=80=99
wrapper shipped with Ruby, =E2=80=98IO::select=E2=80=99. But in the futu=
re,
I'd like to use libevent instead, because that'd be cooler.
Most applications use a higher-level abstraction built on
top of the low-level multiplexer, usually called a =E2=80=98main
loop=E2=80=99, an =E2=80=98event loop=E2=80=99, or an =E2=80=98event sour=
ce=E2=80=99. There are
also libraries such as liboop, which generalizes the event
source and event sink concepts, so that components (event
sinks) written against liboop become event-source-agnostic.
Actually, the combination of blocking IO and Ruby's green
threads works well in most cases where you would normally
use an event loop. When you call =E2=80=98IO#read=E2=80=99 on an empty f=
ile
descriptor, for instance, Ruby suspends that thread until
its internal event loop, known as the scheduler (currently
based on =E2=80=98select=E2=80=99), determines that the file descriptor h=
as
become readable. In particular, Ruby never calls the
low-level =E2=80=98read=E2=80=99 function unless it knows that it will no=
t
block (because =E2=80=98select=E2=80=99 said it wouldn't, but see below).
There are several reasons why you would use an event loop
such as the one implemented by this library instead of
not-so-plain ol' blocking IO with Ruby's green threads.
First of all, you may consider the event loop API more
pleasant than Ruby's threads and not-quite-blocking IO.
Otherwise, don't listen to me; go on using the latter.
Blocking IO can occasionally cause unexpected problems.
For example, in some cases a blocking read *can* block even
though select said that the file descriptor was readable.
This problem may be rare (it can happen, for instance, when
the checksum of a piece of data fails to match the payload),
but the bottom line is that non-blocking IO is safer.
Perhaps most importantly, while Ruby's threads are green,
they are still effectively preemptively scheduled, with all
the implications thereof =E2=80=94 in a word, synchronization hell.
By contrast, event handlers are executed in a strictly
sequential manner; an event loop will never run two event
handlers simultaneously. (Though, of course, all bets are
off if you run multiple event loops in separate threads.)
--=20
Daniel Brockman <[email protected]>