Networked Broadcast Messaging

[squishywaffle]

I'm not exactly sure what the term for this would be, but I was
wondering if there were any Python packages that supported some kind
of ad-hoc message broadcasting. What I'd like to do is something like
this:

* On a number of workhorse machines, a process listens for network
messages from our broadcast service. It subscribes to a certain sub-
set of them and will only respond to the messaging events that it is
subscribed to.
* Any machine can broadcast a message out to the network of machines
without specifying an IP address.
* Machines can come and go. Since messages are not directly sent to a
specific IP address from our Python script, the messages are simply
broadcasted to those who are there to listen. If nobody is subscribed
to the message type being sent, nothing happens.

I know XML-RPC and other friends are an option, but I'm looking for
something that doesn't require managing a set of IP addresses or
hostnames. I'm not sure what to Google for such a package/module, any
direction would be greatly appreciated.

 

[gregarican]

I'm not exactly sure what the term for this would be, but I was
wondering if there were any Python packages that supported some kind
of ad-hoc message broadcasting. What I'd like to do is something like
this:

* On a number of workhorse machines, a process listens for network
messages from our broadcast service. It subscribes to a certain sub-
set of them and will only respond to the messaging events that it is
subscribed to.
* Any machine can broadcast a message out to the network of machines
without specifying an IP address.
* Machines can come and go. Since messages are not directly sent to a
specific IP address from our Python script, the messages are simply
broadcasted to those who are there to listen. If nobody is subscribed
to the message type being sent, nothing happens.

I know XML-RPC and other friends are an option, but I'm looking for
something that doesn't require managing a set of IP addresses or
hostnames. I'm not sure what to Google for such a package/module, any
direction would be greatly appreciated.

Offhand I'd suggest binding a specific UDP port on the listening
workstations. Then the broadcasting workstation(s) could just pull a
standard list of IP's based on its own subnet. Then there'd be no hard-
coded machine names or IP addresses to manage. Just send the message
to all hosts on the same subnet as the broadcaster(s). Plus the UDP
connection would be stateless and not care if the receivers actually
got the message or not...

 

[Kushal Kumaran]

Offhand I'd suggest binding a specific UDP port on the listening
workstations. Then the broadcasting workstation(s) could just pull a
standard list of IP's based on its own subnet. Then there'd be no hard-
coded machine names or IP addresses to manage. Just send the message
to all hosts on the same subnet as the broadcaster(s). Plus the UDP
connection would be stateless and not care if the receivers actually
got the message or not...

You could use the socket module to broadcast. Using INADDR_BROADCAST
as the destination should do it. I fail to recollect whether that
will need root privileges...

 

[Greg Taylor]

You could use the socket module to broadcast.  Using INADDR_BROADCAST
as the destination should do it.  I fail to recollect whether that
will need root privileges...

Awesome, I think this is exactly what I'm looking for. Much
appreciated!

Greg

 

[David Bolen]

* Machines can come and go. Since messages are not directly sent to a
specific IP address from our Python script, the messages are simply
broadcasted to those who are there to listen. If nobody is subscribed
to the message type being sent, nothing happens.

What sort of delivery guarantees are you looking for if there is in
fact a machine that is trying to listen to a particular message or
message group? Is it ok if someone is listening for a certain type of
message, is it ok if it misses one that is sent?

If you do simple direct broadcasting (e.g., UDP), you'd need your own
reliability layer above that if you cared if the message actually got
to an intended destination if that destination was present.

If you want better guarantees, you might look into a distributed
message bus like Spread (http://www.spread.org/) or perhaps a
messaging protocol like XMPP (http://xmpp.org/) through its PubSub
extension. Both have Python interfaces, though I have no personal
experience with either. But perhaps that will also give you some
terms or starting points for searching for other options.

-- David

 

[Greg Taylor]

If you want better guarantees, you might look into a distributed
message bus like Spread (http://www.spread.org/) or perhaps a
messaging protocol like XMPP (http://xmpp.org/) through its PubSub
extension.  Both have Python interfaces, though I have no personal
experience with either.  But perhaps that will also give you some
terms or starting points for searching for other options.

This is also very helpful. In particular Spread, looks very
attractive.

Thanks for the tip,

Greg