Windows - select.select, timeout and KeyboardInterrupt

Discussion in 'Python' started by Paul Moore, May 6, 2010.

  1. Paul  Moore

    Paul Moore Guest

    From a quick experiment, it seems that select.select with a timeout
    doesn't react to a keyboard interrupt until the timeout expires.
    Specifically, if I do

    s = socket.socket()
    select.select(, [], [], 30)

    and then press Ctrl-C, Python waits for the 30 seconds before raising
    KeyboardInterrupt.

    Is this a known limitation on Windows? I see no mention of it in the
    documentation. Assuming it is a known limitation, is there a way round
    it? (I'm writing a tiny server using asyncore/asynchat, and the
    delayed response to Ctrl-C is a mild nuisance. Solutions such as "use
    twisted", while probably the sensible option in a wider context, don't
    really apply here - I need something within the confines of the stdlib
    if it's to be worth doing).

    Thanks,
    Paul
     
    Paul Moore, May 6, 2010
    #1
    1. Advertising

  2. Paul Moore schrieb:
    >>From a quick experiment, it seems that select.select with a timeout

    > doesn't react to a keyboard interrupt until the timeout expires.
    > Specifically, if I do
    >
    > s = socket.socket()
    > select.select(, [], [], 30)
    >
    > and then press Ctrl-C, Python waits for the 30 seconds before raising
    > KeyboardInterrupt.
    >
    > Is this a known limitation on Windows? I see no mention of it in the
    > documentation. Assuming it is a known limitation, is there a way round
    > it? (I'm writing a tiny server using asyncore/asynchat, and the
    > delayed response to Ctrl-C is a mild nuisance. Solutions such as "use
    > twisted", while probably the sensible option in a wider context, don't
    > really apply here - I need something within the confines of the stdlib
    > if it's to be worth doing).


    If you look at the source code for time.sleep(), which CAN be interrupted
    by pressing Ctrl-C, you will find that it is carefully programmed to be
    interruptible (sp?). Which is not the case for select.select(), obviously.

    I guess the best way might be to split your select.select() call into several
    ones, using a smaller timeout like 1 second for example.

    BTW: I have experimented with calling the win32 function SetConsoleCtrlHandler()
    before the call to select.select(). This allows to install a python callback
    function which is called when Ctrl+C is pressed. However it seems this callback
    is not able to interrupt the select() call - but it can 'raise SystemExit()'
    which will terminate the script. Here is the code:

    """
    import ctypes, select, socket

    @ctypes.WINFUNCTYPE(ctypes.c_int, ctypes.c_uint)
    def HandlerRoutine(dwCtrlType):
    print "hoppla", dwCtrlType
    if dwCtrlType == 0: # CTRL+C
    raise SystemExit()
    return 1

    s = socket.socket()

    print "Waiting."
    ctypes.windll.kernel32.SetConsoleCtrlHandler(HandlerRoutine, 1)
    select.select(, [], [], 30)
    ctypes.windll.kernel32.SetConsoleCtrlHandler(HandlerRoutine, 0)
    print "Done."
    """
     
    Thomas Heller, May 6, 2010
    #2
    1. Advertising

  3. Paul  Moore

    Paul Moore Guest

    On 6 May, 20:58, Thomas Heller <> wrote:
    > If you look at the source code for time.sleep(), which CAN be interrupted
    > by pressing Ctrl-C, you will find that it is carefully programmed to be
    > interruptible (sp?).  Which is not the case for select.select(), obviously.


    Thanks - given this, would it be worth me submitting a documentation
    patch noting that select.select is not interruptible on Windows?

    > I guess the best way might be to split your select.select() call into several
    > ones, using a smaller timeout like 1 second for example.


    Yes, that's probably good enough for my case.

    > BTW: I have experimented with calling the win32 function SetConsoleCtrlHandler()
    > before the call to select.select().  This allows to install a python callback
    > function which is called when Ctrl+C is pressed.  However it seems this callback
    > is not able to interrupt the select() call - but it can 'raise SystemExit()'
    > which will terminate the script.  Here is the code:


    That's useful - I doubt I'll need it for this case, but I'll keep it
    in mind for the future.

    Thanks for the help.
    Paul.
     
    Paul Moore, May 7, 2010
    #3
  4. You can easily avoid this by setting a lower timeout when calling
    asyncore.loop(), like 1 second or less (for example, Twisted uses
    0.001 secs).
    Actually there's no reason for asyncore to have such a high default
    timeout (30 seconds).
    I think this should be signaled on the bug tracker.

    --- Giampaolo
    http://code.google.com/p/pyftpdlib
    http://code.google.com/p/psutil


    2010/5/6 Paul Moore <>:
    > >From a quick experiment, it seems that select.select with a timeout

    > doesn't react to a keyboard interrupt until the timeout expires.
    > Specifically, if I do
    >
    > s = socket.socket()
    > select.select(, [], [], 30)
    >
    > and then press Ctrl-C, Python waits for the 30 seconds before raising
    > KeyboardInterrupt.
    >
    > Is this a known limitation on Windows? I see no mention of it in the
    > documentation. Assuming it is a known limitation, is there a way round
    > it? (I'm writing a tiny server using asyncore/asynchat, and the
    > delayed response to Ctrl-C is a mild nuisance. Solutions such as "use
    > twisted", while probably the sensible option in a wider context, don't
    > really apply here - I need something within the confines of the stdlib
    > if it's to be worth doing).
    >
    > Thanks,
    > Paul
    > --
    > http://mail.python.org/mailman/listinfo/python-list
    >
     
    Giampaolo Rodolà, May 7, 2010
    #4
  5. Le Fri, 07 May 2010 16:36:44 +0200, Giampaolo Rodolà a écrit :
    > You can easily avoid this by setting a lower timeout when calling
    > asyncore.loop(), like 1 second or less (for example, Twisted uses 0.001
    > secs).
    > Actually there's no reason for asyncore to have such a high default
    > timeout (30 seconds).


    The reason for a high default timeout would be to avoid waking the CPU
    and do useless work too often. This is important on laptops and smaller
    devices, in order to conserve power.

    Under Unix, it's easy to have a separate fd on which you write a byte
    when an signal comes, and which wakes up your select() call. Under
    Windows, it may be more involved -- first because select() only takes
    sockets, not pipes.
     
    Antoine Pitrou, May 7, 2010
    #5
  6. Paul  Moore

    Paul Moore Guest

    On 7 May 2010 15:36, Giampaolo Rodolà <> wrote:
    > You can easily avoid this by setting a lower timeout when calling
    > asyncore.loop(), like 1 second or less (for example, Twisted uses
    > 0.001 secs).


    Thanks, that's what I was considering.

    > Actually there's no reason for asyncore to have such a high default
    > timeout (30 seconds).


    I assumed it was to avoid busy waiting.

    > I think this should be signaled on the bug tracker.


    If a longer timeout doesn't have issues with busy waiting, then I'd agree.
    Paul
     
    Paul Moore, May 7, 2010
    #6
  7. 2010/5/7 Antoine Pitrou <>:
    > Le Fri, 07 May 2010 16:36:44 +0200, Giampaolo Rodolà a écrit :
    >> You can easily avoid this by setting a lower timeout when calling
    >> asyncore.loop(), like 1 second or less (for example, Twisted uses 0.001
    >> secs).
    >> Actually there's no reason for asyncore to have such a high default
    >> timeout (30 seconds).

    >
    > The reason for a high default timeout would be to avoid waking the CPU
    > and do useless work too often. This is important on laptops and smaller
    > devices, in order to conserve power.
    >


    Of course, but 30 seconds look a little bit too much to me, also
    because (I might be wrong here) I noticed that a smaller timeout seems
    to result in better performances.
    Plus, if scheduled callbacks are ever gonna be added to asyncore we
    would be forced to lower the default timeout anyway in order to have a
    decent reactivity.


    --- Giampaolo
    http://code.google.com/p/pyftpdlib
    http://code.google.com/p/psutil
     
    Giampaolo Rodolà, May 7, 2010
    #7
  8. Le Fri, 07 May 2010 21:55:15 +0200, Giampaolo Rodolà a écrit :
    > Of course, but 30 seconds look a little bit too much to me, also because
    > (I might be wrong here) I noticed that a smaller timeout seems to result
    > in better performances.


    That's probably bogus.

    > Plus, if scheduled callbacks are ever gonna be added to asyncore we
    > would be forced to lower the default timeout anyway in order to have a
    > decent reactivity.


    Why?
     
    Antoine Pitrou, May 7, 2010
    #8
  9. Paul  Moore

    Guest

    On 7 May, 07:25 pm, wrote:
    >On 7 May 2010 15:36, Giampaolo Rodolà <> wrote:
    >>You can easily avoid this by setting a lower timeout when calling
    >>asyncore.loop(), like 1 second or less (for example, Twisted uses
    >>0.001 secs).

    >
    >Thanks, that's what I was considering.


    This is a good example of why it's a bad idea to use select on Windows.
    Instead, use WaitForMultipleObjects.
    >>Actually there's no reason for asyncore to have such a high default
    >>timeout (30 seconds).

    >
    >I assumed it was to avoid busy waiting.
    >>I think this should be signaled on the bug tracker.

    >
    >If a longer timeout doesn't have issues with busy waiting, then I'd
    >agree.


    The *default* timeout is only the default. An application that knows
    better can supply a different value. I'm not sure how much good can be
    done by fiddling with the default.

    Jean-Paul
     
    , May 8, 2010
    #9
  10. In message <>,
    wrote:

    > This is a good example of why it's a bad idea to use select on Windows.
    > Instead, use WaitForMultipleObjects.


    How are you supposed to write portable code, then?
     
    Lawrence D'Oliveiro, May 8, 2010
    #10
  11. 2010/5/7 Antoine Pitrou <>:
    > Le Fri, 07 May 2010 21:55:15 +0200, Giampaolo Rodolà a écrit :
    >> Of course, but 30 seconds look a little bit too much to me, also because
    >> (I might be wrong here) I noticed that a smaller timeout seems to result
    >> in better performances.

    >
    > That's probably bogus.


    Probably, I'll try to write a benchmark script and see what happens.

    >> Plus, if scheduled callbacks are ever gonna be added to asyncore we
    >> would be forced to lower the default timeout anyway in order to have a
    >> decent reactivity.

    >
    > Why?


    Assuming loop() function does something like this:

    ...
    select.select(r, w, e, timeout)
    scheduler() # checks for scheduled calls to be fired
    ...

    ....imagine a case where there's a connection (aka a dispatcher
    instance) which does not receive or send any data *and* a scheduled
    call which is supposed to be fired after, say, 5 seconds.
    The entire loop would hang on select.select() which won't return for
    30 seconds because the socket is not ready for reading and/or writing
    resulting in scheduler() be called too late.


    --- Giampaolo
    http://code.google.com/p/pyftpdlib
    http://code.google.com/p/psutil
     
    Giampaolo Rodolà, May 8, 2010
    #11
  12. Paul  Moore

    Guest

    On 07:48 am, _zealand wrote:
    >In message <>,
    > wrote:
    >>This is a good example of why it's a bad idea to use select on
    >>Windows.
    >>Instead, use WaitForMultipleObjects.

    >
    >How are you supposed to write portable code, then?


    With WaitForMultipleObjects on Windows, epoll on Linux, kqueue on BSD,
    event completion on Solaris, etc...

    Sound like more work than using select() everywhere? Yea, a bit. But
    not once you abstract it away from your actual application code. After
    all, it's not like these *do* different things. They all do the same
    thing (basically) - differently.

    Jean-Paul
     
    , May 8, 2010
    #12
  13. Paul  Moore

    Guest

    On 11:47 am, wrote:
    >2010/5/7 Antoine Pitrou <>:
    >>Le Fri, 07 May 2010 21:55:15 +0200, Giampaolo Rodolà a écrit :
    >>>Of course, but 30 seconds look a little bit too much to me, also
    >>>because
    >>>(I might be wrong here) I noticed that a smaller timeout seems to
    >>>result
    >>>in better performances.

    >>
    >>That's probably bogus.

    >
    >Probably, I'll try to write a benchmark script and see what happens.
    >>>Plus, if scheduled callbacks are ever gonna be added to asyncore we
    >>>would be forced to lower the default timeout anyway in order to have
    >>>a
    >>>decent reactivity.

    >>
    >>Why?

    >
    >Assuming loop() function does something like this:
    >
    > ...
    > select.select(r, w, e, timeout)
    > scheduler() # checks for scheduled calls to be fired
    > ...
    >
    >...imagine a case where there's a connection (aka a dispatcher
    >instance) which does not receive or send any data *and* a scheduled
    >call which is supposed to be fired after, say, 5 seconds.
    >The entire loop would hang on select.select() which won't return for
    >30 seconds because the socket is not ready for reading and/or writing
    >resulting in scheduler() be called too late.


    This would be an intensely lame way to implement support for scheduled
    callbacks. Try this trivial modification instead, to make it awesome:

    ...
    select.select(r, w, e, scheduler.time_until_next_call())
    scheduler.run()
    ...

    (Or maybe just use Twisted. ;)

    Jean-Paul
     
    , May 8, 2010
    #13
  14. On Sat, 8 May 2010 13:47:53 +0200
    Giampaolo Rodolà <> wrote:
    >
    > Assuming loop() function does something like this:
    >
    > ...
    > select.select(r, w, e, timeout)
    > scheduler() # checks for scheduled calls to be fired
    > ...
    >
    > ...imagine a case where there's a connection (aka a dispatcher
    > instance) which does not receive or send any data *and* a scheduled
    > call which is supposed to be fired after, say, 5 seconds.
    > The entire loop would hang on select.select() which won't return for
    > 30 seconds because the socket is not ready for reading and/or writing
    > resulting in scheduler() be called too late.


    Well, obviously you have to change the timeout based on the delay for
    the next scheduled call. If your patch doesn't do that, it probably
    needs fixing :)

    Regards

    Antoine.
     
    Antoine Pitrou, May 8, 2010
    #14
  15. 2010/5/8 Antoine Pitrou <>:
    > On Sat, 8 May 2010 13:47:53 +0200
    > Giampaolo Rodolà <> wrote:
    >>
    >> Assuming loop() function does something like this:
    >>
    >>      ...
    >>      select.select(r, w, e, timeout)
    >>      scheduler()  # checks for scheduled calls to be fired
    >>      ...
    >>
    >> ...imagine a case where there's a connection (aka a dispatcher
    >> instance) which does not receive or send any data *and* a scheduled
    >> call which is supposed to be fired after, say, 5 seconds.
    >> The entire loop would hang on select.select() which won't return for
    >> 30 seconds because the socket is not ready for reading and/or writing
    >> resulting in scheduler() be called too late.

    >
    > Well, obviously you have to change the timeout based on the delay for
    > the next scheduled call. If your patch doesn't do that, it probably
    > needs fixing :)
    >
    > Regards
    >
    > Antoine.


    No it doesn't and I didn't consider this solution at the time.
    Thanks. =)


    --- Giampaolo
    http://code.google.com/p/pyftpdlib
    http://code.google.com/p/psutil
     
    Giampaolo Rodolà, May 8, 2010
    #15
  16. In message <>,
    wrote:

    > On 07:48 am, _zealand wrote:
    >
    >>In message <>,
    >> wrote:
    >>
    >>>This is a good example of why it's a bad idea to use select on
    >>>Windows.
    >>>Instead, use WaitForMultipleObjects.

    >>
    >>How are you supposed to write portable code, then?

    >
    > With WaitForMultipleObjects on Windows, epoll on Linux, kqueue on BSD,
    > event completion on Solaris, etc...
    >
    > Sound like more work than using select() everywhere? Yea, a bit. But
    > not once you abstract it away from your actual application code. After
    > all, it's not like these *do* different things. They all do the same
    > thing (basically) - differently.


    Do you understand what “portable†means?
     
    Lawrence D'Oliveiro, May 9, 2010
    #16
  17. Am 09.05.2010 11:59, schrieb Lawrence D'Oliveiro:
    > In message<>,
    > wrote:
    >
    >> On 07:48 am, _zealand wrote:
    >>
    >>> In message<>,
    >>> wrote:
    >>>
    >>>> This is a good example of why it's a bad idea to use select on
    >>>> Windows.
    >>>> Instead, use WaitForMultipleObjects.
    >>>
    >>> How are you supposed to write portable code, then?

    >>
    >> With WaitForMultipleObjects on Windows, epoll on Linux, kqueue on BSD,
    >> event completion on Solaris, etc...
    >>
    >> Sound like more work than using select() everywhere? Yea, a bit. But
    >> not once you abstract it away from your actual application code. After
    >> all, it's not like these *do* different things. They all do the same
    >> thing (basically) - differently.

    >
    > Do you understand what “portable†means?

    Yes. For me it means run as best as possible on all platforms I care
    about. It does *not* mean force one method from *one* platform upon all
    platforms for purely ideological reasons (like GTK being too stupid to
    tell the difference between %HOMEPATH% and %APPDATA%).

    cheers
    Paul
     
    Paul Kölle, May 9, 2010
    #17
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Ivan Nestlerode

    KeyboardInterrupt and threading

    Ivan Nestlerode, Jan 2, 2004, in forum: Python
    Replies:
    8
    Views:
    557
    Anand Pillai
    Jan 8, 2004
  2. PantherSE
    Replies:
    0
    Views:
    365
    PantherSE
    May 16, 2005
  3. darren kirby
    Replies:
    1
    Views:
    532
    Diez B. Roggisch
    Nov 27, 2005
  4. Replies:
    3
    Views:
    813
    Donn Cave
    Nov 28, 2005
  5. Mark Probert

    Timeout::timeout and Socket timeout

    Mark Probert, Oct 6, 2004, in forum: Ruby
    Replies:
    1
    Views:
    1,306
    Brian Candler
    Oct 6, 2004
Loading...

Share This Page