asynchronous select from queue

[Boris Mojo-jojo]

Please advise on how to implement multiple select from queues in Ruby. I
have seen only selcet which operates on IO streams. I need to pass
objects not symbols. I have 100 queues and I must wait when message
appears on one of them.

 

[Eric Hodel]

Please advise on how to implement multiple select from queues in
Ruby. I
have seen only selcet which operates on IO streams. I need to pass
objects not symbols. I have 100 queues and I must wait when message
appears on one of them.

ri Queue ?

 

[Boris Mojo-jojo]

ri Queue ?

Sorry but no info on select in ri. Look for yourslef

C:\ruby\bin>ri Queue
----------------------------------------------------------- Class: Queue
This class provides a way to synchronize communication between
threads.

Example:

require 'thread'

queue = Queue.new

producer = Thread.new do
5.times do |i|
sleep rand(i) # simulate expense
queue << i
puts "#{i} produced"
end
end

consumer = Thread.new do
5.times do |i|
value = queue.pop
sleep rand(i/2) # simulate expense
puts "consumed #{value}"
end
end

consumer.join

 

[Vidar Hokstad]

Sorry but no info on select in ri. Look for yourslef

"select" only works on IO objects - that's it's purpose. The reason for
pointing you to Queue was that you indicated you're using queues, in
which case the ri documentation is your best starting point.

#select is in any case the completely wrong place to look unless you've
implemented your queues as pipes or similar.

If you want any more useful help you'll need to give a more complete
description of what you are trying to achieve.

Vidar

 

[Robert Klemme]

"select" only works on IO objects - that's it's purpose. The reason for
pointing you to Queue was that you indicated you're using queues, in
which case the ri documentation is your best starting point.

#select is in any case the completely wrong place to look unless you've
implemented your queues as pipes or similar.

If you want any more useful help you'll need to give a more complete
description of what you are trying to achieve.

Adding to that, application architectural wise I do not see the benefit
of having multiple queues if you want to read from all of them and treat
objects identically anyway. This is a typical scenario where a single
queue is sufficient and in fact the most efficient design. At the
moment I cannot think of a reason why multiple queues must be there.
Boris, can you elaborate?

Kind regards

robert

 

[Boris Mojo-jojo]

Adding to that, application architectural wise I do not see the benefit
of having multiple queues if you want to read from all of them and treat
objects identically anyway. This is a typical scenario where a single
queue is sufficient and in fact the most efficient design. At the
moment I cannot think of a reason why multiple queues must be there.
Boris, can you elaborate?

Kind regards

robert

Suppose you have 1000 devices which are being sent messages, every
device has its own queue. devices are independent so the messages to
them must be processed in parallel. Processing message in one device
should not stop processing others. However as along as you cannot create
1000 threads, there is a thread pool where every thread waits for
message to device X to arrive on one of the 1000 queues and processed it
in context of device X. If some thread is processing message which was
sent to device X then the queue X is excluded from thread pool to
prevent parallel executing of message on the device by two different
threads.

You don't want to use one queue, because when two messages are sent to
one device one after another then all other messages will wait because
second message will not be processed till first one is processed.

Now once again the question. How do I do select from multiple queues?
in Win32 there's WaitForMultipleObjects
in Java there's Selector and Pipes
in Ruby there's ???

 

[Robert Klemme]

Suppose you have 1000 devices which are being sent messages, every
device has its own queue. devices are independent so the messages to
them must be processed in parallel. Processing message in one device
should not stop processing others. However as along as you cannot create
1000 threads, there is a thread pool where every thread waits for
message to device X to arrive on one of the 1000 queues and processed it
in context of device X. If some thread is processing message which was
sent to device X then the queue X is excluded from thread pool to
prevent parallel executing of message on the device by two different
threads.

You don't want to use one queue, because when two messages are sent to
one device one after another then all other messages will wait because
second message will not be processed till first one is processed.

True, in that case "select" would help. But you can easily write your
own processor with thread pool. You can get ideas from Queue internals
or http://wiki.rubygarden.org/Ruby/page/show/MultiThreading

Kind regards

robert

 

[Boris Mojo-jojo]

True, in that case "select" would help. But you can easily write your
own processor with thread pool. You can get ideas from Queue internals
or http://wiki.rubygarden.org/Ruby/page/show/MultiThreading

Kind regards

robert

Thanks a lot for the link