Non-blocking communication between Ruby processes

[Iñaki Baz Castillo]

El Jueves, 7 de Enero de 2010, Eric Wong escribi=F3:

=20
These queues are completely atomic at the message level and descriptors
can be safely shared between processes/threads. SysV message queues
weren't thread-safe, but POSIX ones are.

Great!


=2D-=20
I=F1aki Baz Castillo <[email protected]>

 

[Iñaki Baz Castillo]

El Jueves, 7 de Enero de 2010, I=F1aki Baz Castillo escribi=F3:

El Jueves, 7 de Enero de 2010, Eric Wong escribi=F3:
=20
Great!

I've already tested it

I've also realized that in case two processes perform "receive" for the sam=
e=20
mq then the messages received are distributed at 50% (one message for each=
=20
receiver).


=2D-=20
I=F1aki Baz Castillo <[email protected]>

 

[Iñaki Baz Castillo]

El Jueves, 7 de Enero de 2010, Robert Klemme escribi=F3:

I'd personally prefer to use the DRb approach because then you can
actually send typed messages, i.e. whatever information you need. Also,
it was fun to play around with those small test programs. ;-) And you
can have the reader run on any machine in the network.

Hi Robert, I'd like to thank you the help you gave me in this and other=20
threads. Finally I've decided to use posix message queue [*] under Ruby.

The reason is that it allows safely multiple processes or threads using the=
=20
same mqueue to write message (atomic strings) and also having multiple=20
processes reading from the same mqueue which means load-balancing out of th=
e=20
box

The queue size is configurable and the writer/reader can write/read in the=
=20
mqueue in a blocking or non blocking way.

Also, mqueues allow setting a priority to the messages so those messages wi=
th=20
higher priority are fetched first when reading the mqueue.

Posix message queues are just 20-40% slower than pipes in my benchmarks (bu=
t=20
pipes are no multiprocess/thread safe).


I would like to share a working example:


=2D--- posix_mq_reader.rb ------------------------------
require "posix_mq"

# Parameters:
# - queue name (must start by "/")
# - flags:
# - IO::RDONLY =3D> Just to read from the queue
# - IO::CREAT =3D> Create if it doesn't exist
MQ =3D POSIX_MQ.new "/my_mq", IO::RDONLY | IO::CREAT

loop do
# Blocking waiting:
msg =3D MQ.receive.first # It returns an array [message, priority]
puts "messsage received: #{msg}"
end
=2D-----------------------------------------------------


=2D--- posix_mq_writer.rb ------------------------------
require "posix_mq"

# Open with these options:
# - IO::WRONLY =3D> Just to write into the queue.
# - IO::CREAT =3D> Create if it doesn't exist.
# - IO::NONBLOCK =3D> Don't block when writting (instead raise Errno::EAGA=
IN)
MQ =3D POSIX_MQ.new("/my_mq", IO::WRONLY | IO::CREAT | IO::NONBLOCK)

def send(msg)
begin
MQ << msg
rescue Errno::EAGAIN
puts "Errno::EAGAIN received, the queue is full!"
end
end
=2D-----------------------------------------------------


Now the reader and writer can be open multiple times sharing the same mqueu=
e=20



I also tested your suggested solution with DRb with is really nice, but I=20
don't need all the features DRb provides (I just need to pass a simple stri=
ng=20
to other process(es) from multiple workers).


Again thanks a lot to all the people who contributed in this thread, I've=20
learnt a lot.

Best regards.



[*] http://bogomips.org/ruby_posix_mq/README.html

=2D-=20
I=F1aki Baz Castillo <[email protected]>

 

[Robert Klemme]

El Jueves, 7 de Enero de 2010, Robert Klemme escribi=F3:

I'd personally prefer to use the DRb approach because then you can
actually send typed messages, i.e. whatever information you need. =A0Als= o,
it was fun to play around with those small test programs. ;-) =A0And you
can have the reader run on any machine in the network.

Hi Robert, I'd like to thank you the help you gave me in this and other
threads. Finally I've decided to use posix message queue [*] under Ruby.

You're welcome!

The reason is that it allows safely multiple processes or threads using t= he
same mqueue to write message (atomic strings) and also having multiple
processes reading from the same mqueue which means load-balancing out of = the
box

The queue size is configurable and the writer/reader can write/read in th= e
mqueue in a blocking or non blocking way.

Also, mqueues allow setting a priority to the messages so those messages = with
higher priority are fetched first when reading the mqueue.

Posix message queues are just 20-40% slower than pipes in my benchmarks (= but
pipes are no multiprocess/thread safe).

That sounds good! I have never worked with POSIX MQ so I definitively
learned something new as well.

I would like to share a working example:

Thank you for the summary and the code! That way other readers will
benefit as well.

I also tested your suggested solution with DRb with is really nice, but I
don't need all the features DRb provides (I just need to pass a simple st= ring
to other process(es) from multiple workers).

Well, you don't have to use them. But POSIX MQ looks equally simple to=
use.

Kind regards

robert

--=20
remember.guy do |as, often| as.you_can - without end
http://blog.rubybestpractices.com/

 

[Gary Wright]

Posix message queues are just 20-40% slower than pipes in my = benchmarks (but=20
pipes are no multiprocess/thread safe).

I believe pipes can be used concurrently if reads and writes are less =
than
or equal to PIPE_BUF bytes. Is the size limitation the problem you were
hinting at or something else?

Gary Wright

 

[Iñaki Baz Castillo]

El Domingo, 10 de Enero de 2010, Gary Wright escribi=F3:

=20
I believe pipes can be used concurrently if reads and writes are less than
or equal to PIPE_BUF bytes. Is the size limitation the problem you were
hinting at or something else?

Yes, pipes allow atomic operation but just in case the message is less than=
4=20
or 8 bytes, no more. In C it can be useful if you pass a pointer via pipe.

I tested it (in ruby) by sending strings of 10 bytes size from various=20
processes to a shared pipe. The reader gets the strings mixed
However this doesn't occur with posix message queues.

Regards.

=2D-=20
I=F1aki Baz Castillo <[email protected]>

 

[Gary Wright]

=20
I tested it (in ruby) by sending strings of 10 bytes size from various=20=

processes to a shared pipe. The reader gets the strings mixed
However this doesn't occur with posix message queues.

I just tried this on Mac OS X using threads and using forks to write to
a shared pipe. Each write was 256 bytes done using syswrite. All the
reads on the other end were: sysread(256). I tried using forked writers
and simple Ruby threads. Each writer wrote 10000 messages and I didn't
get any mixed up data in any of my tests. I also tried using plain
old write (vs. syswrite) and still didn't get any mixed up data.

The behavior you described just doesn't match my experience or
understanding of pipes.

Gary Wright

 

[Iñaki Baz Castillo]

El Lunes, 11 de Enero de 2010, Gary Wright escribi=F3:

=20
I just tried this on Mac OS X using threads and using forks to write to
a shared pipe. Each write was 256 bytes done using syswrite. All the
reads on the other end were: sysread(256). I tried using forked writers
and simple Ruby threads. Each writer wrote 10000 messages and I didn't
get any mixed up data in any of my tests. I also tried using plain
old write (vs. syswrite) and still didn't get any mixed up data.
=20
The behavior you described just doesn't match my experience or
understanding of pipes.

I got the mixed strings using a name pipe (created with "mkfifo" command) a=
nd=20
shared by two different Ruby programs (a writer and a reader).

However I don't know if I've modified something since that test but the fac=
t=20
is that I don't get the mixed strings...


=2D-=20
I=F1aki Baz Castillo <[email protected]>

 

[Gary Wright]

=20
I got the mixed strings using a name pipe (created with "mkfifo" = command) and=20
shared by two different Ruby programs (a writer and a reader).

A named pipe and a pipe should behave the same, the only difference =
being
that one has a name in the filesystem.

 

[Iñaki Baz Castillo]

El Lunes, 11 de Enero de 2010, Gary Wright escribi=F3:

=20
A named pipe and a pipe should behave the same, the only difference being
that one has a name in the filesystem.

Yes, I know, I just told it because I tested with named piped.
However it's true that I don't get the issue occuring when writting in a na=
med=20
pipe from different processes....=20


=2D-=20
I=F1aki Baz Castillo <[email protected]>