UDP reading on multiple sockets

[Thomas Vogel]

Hi all,

I'm currently have the problem that I try to read UDP messages from
multiple sockets in parallel. So let's say I get UDP packets from the
same IP on the ports 2000, 2001, 2002,...

Therefore I created the following class.
But if I try to instantiate it several times with different ports I
experience that if I get a broadcast message all ports are waking up,
the 1st socket is using the message and the others close itself as the
buffer is empty.

Does anybody have an idea how to do this in a better way?
I just need non-blocking UDP-receiver that are listening on several
ports in parallel and return me the incomming messages. Are there
examples in the net that describe this problem?

class Receiver(asyncore.dispatcher, threading.Thread):
'''
This is the command receiver entity.
'''

def __init__(self, api, localIp, localPort):

asyncore.dispatcher.__init__(self)
threading.Thread.__init__(self)
self.create_socket(socket.AF_INET, socket.SOCK_DGRAM)
self.bind((localIp, localPort))
self.cv = threading.Condition()
self.data = []
self.sorbasApi = api

def handle_connect(self):
return

def writable(self):
return False

def handle_error(self):
self.logger.error('error')
return asyncore.dispatcher.handle_error(self)

def handle_read(self):
s = self.recv(4096)
self.cv.acquire()
self.data.append(s)
self.cv.notifyAll()
self.cv.release()

def handle_close(self):
self.close()

def run(self):
asyncore.loop()

def flush(self):
self.data = []

def readNextMsg(self, paramSet, timeout=1):
if len(self.data) == 0:
self.cv.acquire()
self.cv.wait(timeout)
self.cv.release()
if len(self.data) > 0:
buffer = array('B', self.data.pop(0)).tolist()
m = ReceiveMessage(self.sorbasApi, paramSet)
m.decode(buffer)
return m
else:
return None

def __del__(self):
self.close()

Thanks a lot in advance
Thomas

 

[thomas.vogel]

Is there any reason you can't do it the easy way by using
select?

http://docs.python.org/library/select.html

The only honest answer would be that I'm totaly unfamiliar with select
and also the documentation I found wasn't able to clear the picture.
So are there examples of using select together with sockets available?

Kind regards
Thomas

 

[Guido Goldstein]

Hi!

[...]

The only honest answer would be that I'm totaly unfamiliar with select
and also the documentation I found wasn't able to clear the picture.
So are there examples of using select together with sockets available?

If you want to write something related to networks you should know
this book:
http://www.kohala.com/start/unpv12e.html
Also found at
http://www.amazon.de/UNIX-Network-Programming-Sockets-Networking/dp/013490012X

MfT
Guido

 

[Steve Howell]

The only honest answer would be that I'm totaly unfamiliar with select
and also the documentation I found wasn't able to clear the picture.
So are there examples of using select together with sockets available?

My two cents on this issue is that you should stick with asyncore for
now and just figure out how to debug better what's going on.

I understand the rationale behind the alternatives--there are
certainly advantages (even in debugging) to using something closer to
the metal, like select, or in using something that might be a better
abstraction, like Twisted.

But really, if you look at the source code for asyncore, it's just a
thin layer on top of select.select, and the entire module is under 600
lines of code. Asyncore is much maligned, and there are some
legitimate beefs with it, but it does well, for the most part, what it
intends to do, which is to solve some of the more difficult, or at
least tedious, problems with regards to dealing with select.select.

My fear is that if you try to do this without asyncore, via select,
you'll end up reinventing lots of asyncore with lots of your own
bugs. I have less fear about going higher in the stack, except that
it just puts another layer on top of the problem, and I think the
issue you're dealing with is fairly low-level.

So my suggestion is to figure out a good debugging strategy, either by
getting a bit more familiar with asyncore internals (with a debugger,
print statements, analysis, etc.), or by better using system tools
(like snoop, truss, etc.) to see what's happening at the I/O level, or
both.

Good luck!



Steve

 

[Gabriel Rossetti]

The only honest answer would be that I'm totaly unfamiliar with select
and also the documentation I found wasn't able to clear the picture.
So are there examples of using select together with sockets available?

Kind regards
Thomas

You could also try Twisted : http://twistedmatrix.com/trac/

Best,
Gabriel

 

[Lawrence D'Oliveiro]

In message <27bd949f-80b5-44c9-8e3b-

The only honest answer would be that I'm totaly unfamiliar with select
and also the documentation I found wasn't able to clear the picture.
So are there examples of using select together with sockets available?

select is easy to use. In Python, you just pass it three lists of file-like
objects (anything that implements fileno in the way described in the docs),
together with an optional timeout. It will return three corresponding lists,
being subsets of the ones you passed (which might all be empty if the
timeout expired). You then just go through each item in each returned list,
doing reads (or accepts, as appropriate) from the files that have something
waiting to be read, writing to the ones waiting for something to be written,
and perhaps reporting errors and closing the ones that report problems.
That's basically all there is to it.

This example, of how do implement a timeout on I/O
<http://codecodex.com/wiki/index.php?title=I/O_With_Timeout>, is in C, but
it should help you get the idea.