RabbitMQ vs ApacheQpid (AMQP)

[jacopo]

Short question:
I am considering two solutions for a distributed system: either
RabbitMQ with py-amqplib or ApacheQpid with its own set of API. I
wander is any of you would be able to compare the two. In particular,
I would expect that since Apache comes with its own documented Python
API this would be sufficient to put it as a first choice.

Background:
I have a main machine dispatching heavy calculations to different
machines, collecting the results, performing some calculation on the
merged results and starting all over again with fresher data. I
implemented a first solution with Twisted PB, then I decided it was
more flexible to rely on an AMQP system and I started looking at
RabbitMQ with txAMQP . Now I am getting really frustrated with the
complexity of Twisted and the reactor, I am realizing that probably I
don’t need to handle asynchronicity. Some degree of asynchronicity is
already handled by the Messaging system and I don’t need to take care
of it in my components (they keep waiting for a message, when they get
it they are fully engaged in processing and they cannot do much more).


Thanks,
Jacopo

 

[Roger Binns]

I am considering two solutions for a distributed system: either
RabbitMQ with py-amqplib or ApacheQpid with its own set of API.

Have you considered the multiprocessing module?

http://docs.python.org/library/multiprocessing.html#using-a-remote-manager

Roger

 

[Hendrik van Rooyen]

Background:
I have a main machine dispatching heavy calculations to different
machines, collecting the results, performing some calculation on the
merged results and starting all over again with fresher data. I
implemented a first solution with Twisted PB, then I decided it was
more flexible to rely on an AMQP system and I started looking at
RabbitMQ with txAMQP . Now I am getting really frustrated with the
complexity of Twisted and the reactor, I am realizing that probably I
don’t need to handle asynchronicity. Some degree of asynchronicity is
already handled by the Messaging system and I don’t need to take care
of it in my components (they keep waiting for a message, when they get
it they are fully engaged in processing and they cannot do much more).

Have you looked at Pyro?

- Hendrik