select.epoll question

Discussion in 'Python' started by Paul Rubin, Feb 7, 2013.

  1. Paul Rubin

    Paul Rubin Guest

    I'm trying to listen to a bunch of sockets using epoll under Linux, e.g.

    import select, socket
    socket1 = socket.socket() ...

    p = select.epoll()
    p.register(socket1); p.register(socket2); ...
    result = p.poll()

    This returns `result' as a list of 2-tuples (fd, status) where fd
    is a Linux file descriptor, i.e. a small integer like comes back
    from socket.fileno(). That's different from select.select, which
    returns a list of actual socket objects that I can read from with
    socket.recv().

    Any idea of a good way to map the file descriptors back to socket
    objects? Is there some kind of hidden interface that I don't know
    about, that gives back sockets directly?

    The docs for epoll in select.html don't seem very good. I haven't
    yet examined the source code.

    Thanks for any advice.

    --Paul
     
    Paul Rubin, Feb 7, 2013
    #1
    1. Advertising

  2. On Thu, Feb 7, 2013 at 6:08 PM, Paul Rubin <> wrote:
    > Any idea of a good way to map the file descriptors back to socket
    > objects? Is there some kind of hidden interface that I don't know
    > about, that gives back sockets directly?


    I don't know of any, but you can get the file descriptor from a socket
    via fileno(), and build your own dictionary:

    fd_to_sock={sock.fileno():sock for sock in list_of_sockets}

    You'd need to manually maintain that as sockets get created/destroyed, though.

    ChrisA
     
    Chris Angelico, Feb 7, 2013
    #2
    1. Advertising

  3. Paul Rubin

    Paul Rubin Guest

    Chris Angelico <> writes:
    > fd_to_sock={sock.fileno():sock for sock in list_of_sockets}
    > You'd need to manually maintain that as sockets get created/destroyed,
    > though


    Thanks, I was hoping to avoid that. I'll have to check how
    select.select manages to return sockets. Maybe it builds such a dict
    from the object list before it calls the system's select function, then
    maps the result back afterwards. Ugh.
     
    Paul Rubin, Feb 7, 2013
    #3
  4. On Fri, Feb 8, 2013 at 3:15 AM, Paul Rubin <> wrote:
    > Chris Angelico <> writes:
    >> fd_to_sock={sock.fileno():sock for sock in list_of_sockets}
    >> You'd need to manually maintain that as sockets get created/destroyed,
    >> though

    >
    > Thanks, I was hoping to avoid that. I'll have to check how
    > select.select manages to return sockets. Maybe it builds such a dict
    > from the object list before it calls the system's select function, then
    > maps the result back afterwards. Ugh.


    Yeah, I figured fileno() probably wouldn't be news to you. I don't
    suppose there's anything convenient in the rest of your application
    that makes such a list/dict plausible? For instance, if you need to
    maintain a list of all current socket connections to support broadcast
    operations, then it's not much harder to also maintain the fd->socket
    mapping.

    ChrisA
     
    Chris Angelico, Feb 7, 2013
    #4
  5. Paul Rubin

    Paul Rubin Guest

    Chris Angelico <> writes:
    > Yeah, I figured fileno() probably wouldn't be news to you. I don't
    > suppose there's anything convenient in the rest of your application
    > that makes such a list/dict plausible?


    In fact it's rather annoying, sockets are created and destroyed in
    multiple places in the program. I have to be careful about leaking them
    in case a thread crashes, etc. I dealt with the poll issue by manually
    tracking what was happening, but it wasn't pretty, so I wondered if
    there was a better solution I was overlooking. I don't want to mess
    with it too much more since I'm planning a completely different approach
    for the next version of the program.
     
    Paul Rubin, Feb 7, 2013
    #5
  6. Paul Rubin <> writes:

    > Chris Angelico <> writes:
    >> Yeah, I figured fileno() probably wouldn't be news to you. I don't
    >> suppose there's anything convenient in the rest of your application
    >> that makes such a list/dict plausible?

    >
    > In fact it's rather annoying, sockets are created and destroyed in
    > multiple places in the program. I have to be careful about leaking them
    > in case a thread crashes, etc. I dealt with the poll issue by manually
    > tracking what was happening, but it wasn't pretty, so I wondered if
    > there was a better solution I was overlooking. I don't want to mess
    > with it too much more since I'm planning a completely different approach
    > for the next version of the program.


    You don't have to maintain the mapping globally. You could wrap
    socket.epoll in a class of your own that manages the mapping, and passes
    through calls to socket.epoll. That would make the wrapper a drop-in
    replacement for socket.epoll.

    --
    regards,
    kushal
     
    Kushal Kumaran, Feb 8, 2013
    #6
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. James Mills

    epoll socket server

    James Mills, Oct 10, 2008, in forum: Python
    Replies:
    0
    Views:
    741
    James Mills
    Oct 10, 2008
  2. _wolf
    Replies:
    0
    Views:
    918
    _wolf
    Mar 24, 2010
  3. sonet

    perl and epoll

    sonet, Nov 3, 2006, in forum: Perl Misc
    Replies:
    0
    Views:
    217
    sonet
    Nov 3, 2006
  4. lovecreatesbeauty

    epoll networking code review

    lovecreatesbeauty, Aug 30, 2012, in forum: C Programming
    Replies:
    4
    Views:
    705
    Ben Bacarisse
    Aug 30, 2012
  5. Miki Tebeka
    Replies:
    2
    Views:
    288
    Miki Tebeka
    Sep 12, 2012
Loading...

Share This Page