Re: filedescriptor out of range in select()

Discussion in 'Python' started by Andrew Bennetts, Jun 26, 2003.

  1. On Thu, Jun 26, 2003 at 09:45:19AM +0200, Paolo Invernizzi wrote:
    > Spambayes CVS, Python 2.2.3, FreeBSD 5.1
    >
    > On my machine I'm lunching the pop3proxy with cron at startup, and all
    > is fine.
    >
    > But... sometimes during startup, or after a one or two days, the process
    > exits with the following error...
    >
    > Is a BSD select related problem? Is an asyncore problem? Is a spambayes
    > problem?


    I'm not very familiar with BSD, asyncore or spambayes, but I think
    "filedescriptor out of range in select()" usually means that the process is
    trying to handle too many connections at once -- maybe the sockets aren't
    getting closed, or something.

    You might like to check the output of netstat when it happens, too see if
    there's an outrageously large number of connections to your pop3proxy. I'm
    just guessing, though.

    -Andrew.
     
    Andrew Bennetts, Jun 26, 2003
    #1
    1. Advertising

  2. Andrew Bennetts wrote:

    > I'm not very familiar with BSD, asyncore or spambayes, but I think
    > "filedescriptor out of range in select()" usually means that the
    > process is
    > trying to handle too many connections at once -- maybe the sockets
    > aren't
    > getting closed, or something.


    I strongly suspect it means that the file descriptor (or the result of
    calling the .fileno method of a socket-like object) passed in one of the
    objects is way out of range for being a valid file descriptor number.
    If it were simply bad (i.e., not corresponding to any valid file within
    that process but a legal value), the error would be "Bad file
    descriptor":

    max@oxygen:~% python
    Python 2.2.3 (#1, May 31 2003, 21:31:33)
    [GCC egcs-2.91.66 19990314/Linux (egcs-1.1.2 release)] on linux2
    Type "help", "copyright", "credits" or "license" for more information.
    >>> import select
    >>> select.select([4], [], [], 0)

    Traceback (most recent call last):
    File "<stdin>", line 1, in ?
    select.error: (9, 'Bad file descriptor')
    >>> select.select([10000000], [], [], 0)

    Traceback (most recent call last):
    File "<stdin>", line 1, in ?
    ValueError: filedescriptor out of range in select()

    --
    Erik Max Francis && && http://www.alcyone.com/max/
    __ San Jose, CA, USA && 37 20 N 121 53 W && &tSftDotIotE
    / \ Only the ephemeral is of lasting value.
    \__/ Ionesco
     
    Erik Max Francis, Jun 26, 2003
    #2
    1. Advertising

  3. Fredrik Lundh wrote:

    > but that doesn't necessarily mean that the #define in an include file
    > somewhere automagically picks up the kernel configuration, does it?


    You are right, obviously.

    > (a quick googling brings up some posts about FreeBSD machines with
    > FD_SETSIZE=256 and maxuserfiles>1024 etc. dunno if they've made
    > their configuration tools smarter, but in any case, "poll" is the "right
    > way" to deal with large number of file handles (compared to "select",
    > at least -- there are better but platform-specific ways to do things
    > if you want to...))


    Right again.

    Many thanks for the help

    ---
    Paolo
     
    Paolo Invernizzi, Jun 26, 2003
    #3
  4. Fredrik Lundh wrote:

    > but I'm pretty sure asyncore supports the "poll" interface. try changing
    > the asyncore.loop call in Dibbler.py to:
    >
    > asyncore.loop(map=context._map, use_poll=1)
    >
    > and see what happens.


    Here we are again....
    Too many open files... with poll too...

    ---
    Paolo

    SpamBayes POP3 Proxy Beta1, version 0.1 (May 2003),
    using SpamBayes POP3 Proxy Web Interface Alpha2, version 0.02
    and engine SpamBayes Beta1, version 0.1 (May 2003).

    Loading database... Done.
    SMTP Listener on port 2525 is proxying 127.0.0.1:25
    Listener on port 8110 is proxying kpnqwest.it:110
    User interface url is http://localhost:8880/
    error: uncaptured python exception, closing channel
    <__main__.BayesProxyListener listening :8110 at 0x83c9f6c>
    (socket.error:(24, 'Too many open files')
    [/usr/local/lib/python2.2/asyncore.py|poll3|184]
    [/usr/local/lib/python2.2/asyncore.py|handle_read_event|391]
    [/usr/local/lib/python2.2/site-packages/spambayes/Dibbler.py|handle_accept|280]
    [/usr/local/bin/pop3proxy.py|__init__|337]
    [/usr/local/bin/pop3proxy.py|__init__|180]
    [/usr/local/bin/pop3proxy.py|__init__|134]
    [/usr/local/lib/python2.2/asyncore.py|create_socket|260])
    error: uncaptured python exception, closing channel <__main__.BayesProxy
    connected 192.168.0.2:63485 at 0x84d8aac>
    (exceptions.AttributeError:'_socket.socket' object has no attribute
    'isClosed' [/usr/local/lib/python2.2/asyncore.py|poll3|184]
    [/usr/local/lib/python2.2/asyncore.py|handle_read_event|397]
    [/usr/local/lib/python2.2/asynchat.py|handle_read|82]
    [/usr/local/bin/pop3proxy.py|recv|360]
    [/usr/local/lib/python2.2/asyncore.py|recv|353]
    [/usr/local/lib/python2.2/asynchat.py|handle_close|149]
    [/usr/local/bin/pop3proxy.py|close|368]
    [/usr/local/lib/python2.2/asyncore.py|__getattr__|372])
    Traceback (most recent call last):
    File "/usr/local/bin/pop3proxy.py", line 731, in ?
    run()
    File "/usr/local/bin/pop3proxy.py", line 725, in run
    main(state.servers, state.proxyPorts, state.uiPort, state.launchUI)
    File "/usr/local/bin/pop3proxy.py", line 664, in main
    Dibbler.run(launchBrowser=launchUI)
    File "/usr/local/lib/python2.2/site-packages/spambayes/Dibbler.py",
    line 528, in run
    asyncore.loop(map=context._map,use_poll=1)
    File "/usr/local/lib/python2.2/asyncore.py", line 206, in loop
    poll_fun (timeout, map)
    File "/usr/local/lib/python2.2/asyncore.py", line 190, in poll3
    obj.handle_error()
    File "/usr/local/lib/python2.2/site-packages/spambayes/Dibbler.py",
    line 205, in handle_error
    asynchat.async_chat.handle_error(self)
    File "/usr/local/lib/python2.2/asyncore.py", line 427, in handle_error
    self.close()
    File "/usr/local/bin/pop3proxy.py", line 368, in close
    if not self.isClosed:
    File "/usr/local/lib/python2.2/asyncore.py", line 372, in __getattr__
    return getattr (self.socket, attr)
    AttributeError: '_socket.socket' object has no attribute 'isClosed'
     
    Paolo Invernizzi, Jun 27, 2003
    #4
  5. Andrew Bennetts

    Steve Holden Guest

    "Paolo Invernizzi" <> wrote in message
    news:...
    > Fredrik Lundh wrote:
    >
    > > but I'm pretty sure asyncore supports the "poll" interface. try

    changing
    > > the asyncore.loop call in Dibbler.py to:
    > >
    > > asyncore.loop(map=context._map, use_poll=1)
    > >
    > > and see what happens.

    >
    > Here we are again....
    > Too many open files... with poll too...
    >
    > ---
    > Paolo
    >
    > SpamBayes POP3 Proxy Beta1, version 0.1 (May 2003),
    > using SpamBayes POP3 Proxy Web Interface Alpha2, version 0.02
    > and engine SpamBayes Beta1, version 0.1 (May 2003).
    >
    > Loading database... Done.
    > SMTP Listener on port 2525 is proxying 127.0.0.1:25
    > Listener on port 8110 is proxying kpnqwest.it:110
    > User interface url is http://localhost:8880/
    > error: uncaptured python exception, closing channel
    > <__main__.BayesProxyListener listening :8110 at 0x83c9f6c>
    > (socket.error:(24, 'Too many open files')

    [...]
    > Traceback (most recent call last):
    > File "/usr/local/bin/pop3proxy.py", line 731, in ?
    > run()
    > File "/usr/local/bin/pop3proxy.py", line 725, in run
    > main(state.servers, state.proxyPorts, state.uiPort, state.launchUI)
    > File "/usr/local/bin/pop3proxy.py", line 664, in main
    > Dibbler.run(launchBrowser=launchUI)
    > File "/usr/local/lib/python2.2/site-packages/spambayes/Dibbler.py",
    > line 528, in run
    > asyncore.loop(map=context._map,use_poll=1)
    > File "/usr/local/lib/python2.2/asyncore.py", line 206, in loop
    > poll_fun (timeout, map)
    > File "/usr/local/lib/python2.2/asyncore.py", line 190, in poll3
    > obj.handle_error()
    > File "/usr/local/lib/python2.2/site-packages/spambayes/Dibbler.py",
    > line 205, in handle_error
    > asynchat.async_chat.handle_error(self)
    > File "/usr/local/lib/python2.2/asyncore.py", line 427, in handle_error
    > self.close()
    > File "/usr/local/bin/pop3proxy.py", line 368, in close
    > if not self.isClosed:
    > File "/usr/local/lib/python2.2/asyncore.py", line 372, in __getattr__
    > return getattr (self.socket, attr)
    > AttributeError: '_socket.socket' object has no attribute 'isClosed'
    >


    I suspect that the "too many open files" is simply a symptom of a deeper
    problem, in that your sockets aren't being correctly closed due to a
    programming error. The asyncore framework will happily continue to run even
    if errors occur in the event-handling routines, cneatly demonstrating the
    dangers of an unlimited "try:/except:".

    There's a clear implication here that a BayesProxy instance isn't running
    its __init__() method, because if it *were* running it the isClosed
    attribute would be defined.

    Are you perchance subclassing Bayesproxy and overriding its __init__()
    rather than extending it? If so then try adding

    BayesProxy.__init__(self, clientSocket, serverName, serverPort)

    or equivalent to your subclass' __init__().

    regards
    --
    Steve Holden http://www.holdenweb.com/
    Python Web Programming http://pydish.holdenweb.com/pwp/
     
    Steve Holden, Jun 27, 2003
    #5
  6. Steve Holden wrote:

    > I suspect that the "too many open files" is simply a symptom of a deeper
    > problem, in that your sockets aren't being correctly closed due to a
    > programming error. The asyncore framework will happily continue to run even
    > if errors occur in the event-handling routines, cneatly demonstrating the
    > dangers of an unlimited "try:/except:".
    >
    > There's a clear implication here that a BayesProxy instance isn't running
    > its __init__() method, because if it *were* running it the isClosed
    > attribute would be defined.


    I'll check this today...

    > Are you perchance subclassing Bayesproxy and overriding its __init__()
    > rather than extending it?



    Nono, I'm using the 'plain' pop3proxy.py of spambayes... I'm not
    extending the framework...
     
    Paolo Invernizzi, Jun 30, 2003
    #6
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Paolo Invernizzi

    filedescriptor out of range in select()

    Paolo Invernizzi, Jun 26, 2003, in forum: Python
    Replies:
    0
    Views:
    913
    Paolo Invernizzi
    Jun 26, 2003
  2. Replies:
    1
    Views:
    329
    Jack Klein
    Sep 16, 2006
  3. Laszlo Nagy
    Replies:
    0
    Views:
    383
    Laszlo Nagy
    Mar 17, 2009
  4. Laszlo Nagy
    Replies:
    0
    Views:
    445
    Laszlo Nagy
    Mar 17, 2009
  5. k3xji
    Replies:
    5
    Views:
    722
    Steven D'Aprano
    Sep 16, 2010
Loading...

Share This Page