select.epoll question

[Paul Rubin]

I'm trying to listen to a bunch of sockets using epoll under Linux, e.g.

import select, socket
socket1 = socket.socket() ...

p = select.epoll()
p.register(socket1); p.register(socket2); ...
result = p.poll()

This returns `result' as a list of 2-tuples (fd, status) where fd
is a Linux file descriptor, i.e. a small integer like comes back
from socket.fileno(). That's different from select.select, which
returns a list of actual socket objects that I can read from with
socket.recv().

Any idea of a good way to map the file descriptors back to socket
objects? Is there some kind of hidden interface that I don't know
about, that gives back sockets directly?

The docs for epoll in select.html don't seem very good. I haven't
yet examined the source code.

Thanks for any advice.

--Paul

 

[Chris Angelico]

Any idea of a good way to map the file descriptors back to socket
objects? Is there some kind of hidden interface that I don't know
about, that gives back sockets directly?

I don't know of any, but you can get the file descriptor from a socket
via fileno(), and build your own dictionary:

fd_to_sock={sock.fileno():sock for sock in list_of_sockets}

You'd need to manually maintain that as sockets get created/destroyed, though.

ChrisA

 

[Paul Rubin]

fd_to_sock={sock.fileno():sock for sock in list_of_sockets}
You'd need to manually maintain that as sockets get created/destroyed,
though

Thanks, I was hoping to avoid that. I'll have to check how
select.select manages to return sockets. Maybe it builds such a dict
from the object list before it calls the system's select function, then
maps the result back afterwards. Ugh.

 

[Chris Angelico]

Thanks, I was hoping to avoid that. I'll have to check how
select.select manages to return sockets. Maybe it builds such a dict
from the object list before it calls the system's select function, then
maps the result back afterwards. Ugh.

Yeah, I figured fileno() probably wouldn't be news to you. I don't
suppose there's anything convenient in the rest of your application
that makes such a list/dict plausible? For instance, if you need to
maintain a list of all current socket connections to support broadcast
operations, then it's not much harder to also maintain the fd->socket
mapping.

ChrisA

 

[Paul Rubin]

Yeah, I figured fileno() probably wouldn't be news to you. I don't
suppose there's anything convenient in the rest of your application
that makes such a list/dict plausible?

In fact it's rather annoying, sockets are created and destroyed in
multiple places in the program. I have to be careful about leaking them
in case a thread crashes, etc. I dealt with the poll issue by manually
tracking what was happening, but it wasn't pretty, so I wondered if
there was a better solution I was overlooking. I don't want to mess
with it too much more since I'm planning a completely different approach
for the next version of the program.

 

[Kushal Kumaran]

In fact it's rather annoying, sockets are created and destroyed in
multiple places in the program. I have to be careful about leaking them
in case a thread crashes, etc. I dealt with the poll issue by manually
tracking what was happening, but it wasn't pretty, so I wondered if
there was a better solution I was overlooking. I don't want to mess
with it too much more since I'm planning a completely different approach
for the next version of the program.

You don't have to maintain the mapping globally. You could wrap
socket.epoll in a class of your own that manages the mapping, and passes
through calls to socket.epoll. That would make the wrapper a drop-in
replacement for socket.epoll.