Announcing Revactor: an Actor model implementation for Ruby 1.9

[Tony Arcieri]

[Note: parts of this message were removed to make it a legal post.]

I'm pleased to announce the initial public availability of Revactor, an
Actor framework for Ruby 1.9. The Actor model has seen increasing popularity
in languages like Erlang and Scala. Revactor adds asynchronous message
passing to the Fibers mechanism, easing the development of concurrent
programs using them. You can read more about the Actor model here:

http://revactor.org/philosophy/

Revactor also includes a high performance sockets API, largely compatible
with Ruby's own, but which also seamlessly interoperate with the Actor API.
Using this approach means you can simultaneously process messages from both
sockets and other Actors. Furthermore, with a small amount of monkeypatching
it's able to run Mongrel, using Actors instead of threads as the underlying
concurrency mechanism. Initial tests of the performance of Mongrel on top of
Revactor surpass the threaded version.

Revactor is available as a gem and can be installed with:

*gem install revactor*

For additional information please see the web site:

http://revactor.org

 

[Chauk-Mean P.]

Hi,

I'm pleased to announce the initial public availability of Revactor, an
Actor framework for Ruby 1.9. The Actor model has seen increasing
popularity
in languages like Erlang and Scala. Revactor adds asynchronous message
passing to the Fibers mechanism, easing the development of concurrent
programs using them. You can read more about the Actor model here:

http://revactor.org/philosophy/

It sounds very interesting. I will take a look at it.

Chauk-Mean.

 

[Chauk-Mean P.]

It sounds very interesting. I will take a look at it.

Chauk-Mean.

Oops. Revactor requires Rev which requires libev.
I'm on Windows and compiling Rev and libev is not an obvious/easy task.
I will try to have a look on a Linux machine.

Chauk-Mean.

 

[Eivind Eklund]

I'm pleased to announce the initial public availability of Revactor, an
Actor framework for Ruby 1.9. The Actor model has seen increasing popularity
in languages like Erlang and Scala. Revactor adds asynchronous message
passing to the Fibers mechanism, easing the development of concurrent
programs using them. You can read more about the Actor model here:

http://revactor.org/philosophy/

I noticed this use the Case gem to match patterns. Compared to what
you detail, you should be able to get even more flexibility for at
least some cases by using the matching framework that's in types.rb -
http://people.freebsd.org/~eivind/ruby/types/

Eivind.

 

[NewtonApple]

This is very nice. Given Ruby's flexibility, I know sooner or later
there's bound to be a ruby implementation of Erlang's message passing
system (more elegant and pragmatic, hopefully). Hell, even Matz wants
one. May I suggest a few more examples in the RDoc to highlight the
cooperations of actors? They would probably help deepen imaginations
for a beginner like me. Meanwhile I'll keep poking at it until I find
my ha-ha moment (hopefully soon). Thanks for your contribution. Keep
up the good work!

p.s. I know Ruby1.9 is fast, but I didn't think it'd catch on this
fast!

David

 

[Joel VanderWerf]

I'm pleased to announce the initial public availability of Revactor

Any plan to support UDP sockets? (A quick scan of the source seems to
indicate that it doesn't yet.)

 

[Tony Arcieri]

[Note: parts of this message were removed to make it a legal post.]

I noticed this use the Case gem to match patterns. Compared to what
you detail, you should be able to get even more flexibility for at
least some cases by using the matching framework that's in types.rb -
http://people.freebsd.org/~eivind/ruby/types/<http://people.freebsd.org/~eivind/ruby/types/>

Interesting. Any plans on releasing this as a gem?

 

[Tony Arcieri]

[Note: parts of this message were removed to make it a legal post.]

Any plan to support UDP sockets? (A quick scan of the source seems to
indicate that it doesn't yet.)

Yes, there will eventually be a Revactor::UDP module as a counterpart to the
TCP one. This isn't a particularly high priority at the present time,
though.

 

[Eivind Eklund]

Interesting. Any plans on releasing this as a gem?

I can do a gem release if you're interested in using it.

Eivind.

 

[Tony Arcieri]

[Note: parts of this message were removed to make it a legal post.]

I can do a gem release if you're interested in using it.

I think it might be overkill for the purposes of inter-Actor messaging. For
the most part all that's needed for matching expected messages are the
expected tokens and wildcards.

I'm trying to port some more complex applications to Revactor and will
hopefully discover shortly if there's pattern matching corner cases the Case
gem can't provide.

 

[aemadrid]

I'm still trying to wrap my head around this whole concept. What is
the difference in approach between Revactor and EventMachine [1]? I
can see that one works with 1.8 and the other starts with 1.9 but they
seem to me, the uneducated reader, very similar. Am I missing
something?

Thanks in advance,


Adrian Madrid

 

[Tony Arcieri]

[Note: parts of this message were removed to make it a legal post.]

I'm still trying to wrap my head around this whole concept. What is
the difference in approach between Revactor and EventMachine [1]? I
can see that one works with 1.8 and the other starts with 1.9 but they
seem to me, the uneducated reader, very similar. Am I missing
something?

Revactor implements the Actor model and EventMachine implements the Reactor
pattern (hmm, Reactor, Revactor, a little confusing I'm sure. My bad). The
similarities between Actor and Reactor do go much deeper than their names.
Both provide ways of programming performant and highly concurrent network
applications.

The main drawback of Reactor I've encountered is that it's fully
asynchronous and relies on inversion of control. That's not to say this is
a bad thing: it's great for anything that does event processing,
particularly of incoming messages. However, Actors also support this style
of programming. For more information see the Revactor::TCP documentation on
active mode.

The main advantage of Actor over Reactor in this regard involves APIs which
require synchronous interfaces and just won't work without them. Perhaps
the best example of this I can think of is ActiveRecord. With a simple
Foo.findfirst).bar.baz you've traversed across two associations which each
made their own query and blocked until they had the results, because the
next method invocation needed it. It's not the most efficient approach in
the world, but it's there when you need it. Unfortunately, it's just not
possible on top of a Reactor API, but works fine on top of Actors.

Using Reactor means you must abandon any code which is structured around
making synchronous blocking calls to interact with the network. This
includes pretty much everything built on the traditional imperative sockets
API. Everything must use an asynchronous API. There are ways around this,
such as spinning off any synchronous blocking calls in a separate thread,
but then you need a thread for each blocking call you wish to make, and
there are performance issues with threads and I/O in Ruby, not to mention
the traditional pitfalls of threaded programming.

With Actors, you can pull in any code that uses the existing Sockets API,
(monkey)patch in Revactor::TCP::Sockets in their place, and you're good to
go. You can mix and match synchronous and asynchronous programming without
ever having to involve threads.

Perhaps the biggest argument for Actors is how many network applications end
up being structured internally. Time and time again the optimal
architecture seems to be discrete components which communicate with message
passing. For an idea of this, have a look at a digram of qmail:

http://www.nrg4u.com/qmail/the-big-qmail-picture-103-p1.gif
http://www.nrg4u.com/qmail/the-big-qmail-picture-103-p2.gif

Qmail is implemented as a number of C programs which communicate using
pipes. Reactor also works well with this approach: implement each component
as a process, and have them communicate using pipes, sockets, etc.

The Actor model is built around the idea of discrete components which
communicate using message passing, but gets rid of the headaches involved
with process invocation, setting up IPC channels, etc. Rather than being a
heavyweight OS process, each discrete component is a lightweight Ruby
Fiber. This means building systems which rely on independent components
which communicate with message passing is both lightweight and performant.

 

[Kevin Williams]

I could go on and on about Fibers and lightweight concurrency
(Erlang-esque) programming, but suffice it to say that I'm really,
really excited to check this out. I think this is something you should
blog about and explain until you're blue in the face, until we all get
comfortable in concurrent programming. We can't just let all those CPU
cores go to waste, can we? Thanks for injecting this into our
consciousness.

 

[NewtonApple]

Using Reactor means you must abandon any code which is structured around

making synchronous blocking calls to interact with the network. This
includes pretty much everything built on the traditional imperative sockets
API. Everything must use an asynchronous API. There are ways around this,
such as spinning off any synchronous blocking calls in a separate thread,
but then you need a thread for each blocking call you wish to make, and
there are performance issues with threads and I/O in Ruby, not to mention
the traditional pitfalls of threaded programming.

How does Revactor do in terms of memory? I know for things like
Stackless Python and Erlang, they can spawn thousands of threads
without even blinking... How does Revactor fare in this regard?

 

[Tony Arcieri]

[Note: parts of this message were removed to make it a legal post.]

How does Revactor do in terms of memory? I know for things like
Stackless Python and Erlang, they can spawn thousands of threads
without even blinking... How does Revactor fare in this regard?

As best as I can calculate, the baseline overhead for an individual Actor
appears to be in the realm of 9.3kB. Or at least, I noticed a ~93MB memory
increase after spawning 10,000 Actors.

It certainly can't hold a candle to Erlang's ~300 bytes, but it's also
nowhere close to the overhead of a thread, especially in Ruby 1.9.

 

[aemadrid]

Thanks so much for the explanation. I feel a little bit dizzy trying
to figure this all out but I can see a bit clearer now.

Thanks again,


Adrian Madrid

[Note: parts of this message were removed to make it a legal post.]

I'm still trying to wrap my head around this whole concept. What is
the difference in approach between Revactor and EventMachine [1]? I
can see that one works with 1.8 and the other starts with 1.9 but they
seem to me, the uneducated reader, very similar. Am I missing
something?

Revactor implements the Actor model and EventMachine implements the Reactor
pattern (hmm, Reactor, Revactor, a little confusing I'm sure. My bad). The
similarities between Actor and Reactor do go much deeper than their names.
Both provide ways of programming performant and highly concurrent network
applications.

The main drawback of Reactor I've encountered is that it's fully
asynchronous and relies on inversion of control. That's not to say this is
a bad thing: it's great for anything that does event processing,
particularly of incoming messages. However, Actors also support this style
of programming. For more information see the Revactor::TCP documentation on
active mode.

The main advantage of Actor over Reactor in this regard involves APIs which
require synchronous interfaces and just won't work without them. Perhaps
the best example of this I can think of is ActiveRecord. With a simple
Foo.findfirst).bar.baz you've traversed across two associations which each
made their own query and blocked until they had the results, because the
next method invocation needed it. It's not the most efficient approach in
the world, but it's there when you need it. Unfortunately, it's just not
possible on top of a Reactor API, but works fine on top of Actors.

Using Reactor means you must abandon any code which is structured around
making synchronous blocking calls to interact with the network. This
includes pretty much everything built on the traditional imperative sockets
API. Everything must use an asynchronous API. There are ways around this,
such as spinning off any synchronous blocking calls in a separate thread,
but then you need a thread for each blocking call you wish to make, and
there are performance issues with threads and I/O in Ruby, not to mention
the traditional pitfalls of threaded programming.

With Actors, you can pull in any code that uses the existing Sockets API,
(monkey)patch in Revactor::TCP::Sockets in their place, and you're good to
go. You can mix and match synchronous and asynchronous programming without
ever having to involve threads.

Perhaps the biggest argument for Actors is how many network applications end
up being structured internally. Time and time again the optimal
architecture seems to be discrete components which communicate with message
passing. For an idea of this, have a look at a digram of qmail:

http://www.nrg4u.com/qmail/the-big-...4u.com/qmail/the-big-qmail-picture-103-p2.gif

Qmail is implemented as a number of C programs which communicate using
pipes. Reactor also works well with this approach: implement each component
as a process, and have them communicate using pipes, sockets, etc.

The Actor model is built around the idea of discrete components which
communicate using message passing, but gets rid of the headaches involved
with process invocation, setting up IPC channels, etc. Rather than being a
heavyweight OS process, each discrete component is a lightweight Ruby
Fiber. This means building systems which rely on independent components
which communicate with message passing is both lightweight and performant.

 

[Tony Arcieri]

[Note: parts of this message were removed to make it a legal post.]

I could go on and on about Fibers and lightweight concurrency
(Erlang-esque) programming, but suffice it to say that I'm really,
really excited to check this out. I think this is something you should
blog about and explain until you're blue in the face, until we all get
comfortable in concurrent programming. We can't just let all those CPU
cores go to waste, can we? Thanks for injecting this into our
consciousness.

There's some really great opportunities for Ruby in this regard,
specifically with Rubinius. They recently added multi-VM support and
already support Actors. I've hacked together an inter-VM bridge which
allows Actors to communicate across VMs (almost) as if they're in the same
VM. With that sort of thing you can begin to achieve the dream of programs
that run N-times faster on N CPUs.

There is a blog for Revactor, BTW, although right now it only has the
release announcement. I'll certainly start blogging about the future of
Actors on Ruby soon:

http://revactor.org/rss

 

[fedzor]

I've hacked together an inter-VM bridge which
allows Actors to communicate across VMs (almost) as if they're in
the same
VM. With that sort of thing you can begin to achieve the dream of
programs
that run N-times faster on N CPUs.

Would you mind sharing the code for this? I was looking for a way to
accomplish this by running multiple instances of the Ruby VM (by
using fork { exec } ), and having them communicate through sockets.

 

[Matthias Wächter]

As best as I can calculate, the baseline overhead for an individual Actor
appears to be in the realm of 9.3kB. Or at least, I noticed a ~93MB memory
increase after spawning 10,000 Actors.

It certainly can't hold a candle to Erlang's ~300 bytes, but it's also
nowhere close to the overhead of a thread, especially in Ruby 1.9.

If Actor would be(come) a first-class citizen of Ruby, what would you expect the footprint for each Actor to be?

- Mat

 

[Paul Brannan]

This is very cool. I've played with implementing something like this
and, even though it's conceptually simple, it's not trivial at all to
get the details right.

It's interestnig that you chose the make Actor inherit from Fiber. I'm
not sure but I think one side-effect of this is that it makes it
impossible to run an actor in another process transparently (which, if
you're using a fully functional programming style, it should be possible
to do).

Have you done any performance measurements to compare against a threaded
model?

Paul