Windows - select.select, timeout and KeyboardInterrupt

[Paul Moore]

From a quick experiment, it seems that select.select with a timeout
doesn't react to a keyboard interrupt until the timeout expires.
Specifically, if I do

s = socket.socket()
select.select(, [], [], 30)

and then press Ctrl-C, Python waits for the 30 seconds before raising
KeyboardInterrupt.

Is this a known limitation on Windows? I see no mention of it in the
documentation. Assuming it is a known limitation, is there a way round
it? (I'm writing a tiny server using asyncore/asynchat, and the
delayed response to Ctrl-C is a mild nuisance. Solutions such as "use
twisted", while probably the sensible option in a wider context, don't
really apply here - I need something within the confines of the stdlib
if it's to be worth doing).

Thanks,
Paul

 

[Thomas Heller]

From a quick experiment, it seems that select.select with a timeout

doesn't react to a keyboard interrupt until the timeout expires.
Specifically, if I do

s = socket.socket()
select.select(, [], [], 30)

and then press Ctrl-C, Python waits for the 30 seconds before raising
KeyboardInterrupt.

Is this a known limitation on Windows? I see no mention of it in the
documentation. Assuming it is a known limitation, is there a way round
it? (I'm writing a tiny server using asyncore/asynchat, and the
delayed response to Ctrl-C is a mild nuisance. Solutions such as "use
twisted", while probably the sensible option in a wider context, don't
really apply here - I need something within the confines of the stdlib
if it's to be worth doing).

If you look at the source code for time.sleep(), which CAN be interrupted
by pressing Ctrl-C, you will find that it is carefully programmed to be
interruptible (sp?). Which is not the case for select.select(), obviously.

I guess the best way might be to split your select.select() call into several
ones, using a smaller timeout like 1 second for example.

BTW: I have experimented with calling the win32 function SetConsoleCtrlHandler()
before the call to select.select(). This allows to install a python callback
function which is called when Ctrl+C is pressed. However it seems this callback
is not able to interrupt the select() call - but it can 'raise SystemExit()'
which will terminate the script. Here is the code:

"""
import ctypes, select, socket

@ctypes.WINFUNCTYPE(ctypes.c_int, ctypes.c_uint)
def HandlerRoutine(dwCtrlType):
print "hoppla", dwCtrlType
if dwCtrlType == 0: # CTRL+C
raise SystemExit()
return 1

s = socket.socket()

print "Waiting."
ctypes.windll.kernel32.SetConsoleCtrlHandler(HandlerRoutine, 1)
select.select(, [], [], 30)
ctypes.windll.kernel32.SetConsoleCtrlHandler(HandlerRoutine, 0)
print "Done."
"""

 

[Paul Moore]

If you look at the source code for time.sleep(), which CAN be interrupted
by pressing Ctrl-C, you will find that it is carefully programmed to be
interruptible (sp?).  Which is not the case for select.select(), obviously.

Thanks - given this, would it be worth me submitting a documentation
patch noting that select.select is not interruptible on Windows?

I guess the best way might be to split your select.select() call into several
ones, using a smaller timeout like 1 second for example.

Yes, that's probably good enough for my case.

BTW: I have experimented with calling the win32 function SetConsoleCtrlHandler()
before the call to select.select().  This allows to install a python callback
function which is called when Ctrl+C is pressed.  However it seems this callback
is not able to interrupt the select() call - but it can 'raise SystemExit()'
which will terminate the script.  Here is the code:

That's useful - I doubt I'll need it for this case, but I'll keep it
in mind for the future.

Thanks for the help.
Paul.

 

[Giampaolo Rodolà]

You can easily avoid this by setting a lower timeout when calling
asyncore.loop(), like 1 second or less (for example, Twisted uses
0.001 secs).
Actually there's no reason for asyncore to have such a high default
timeout (30 seconds).
I think this should be signaled on the bug tracker.

--- Giampaolo
http://code.google.com/p/pyftpdlib
http://code.google.com/p/psutil

From a quick experiment, it seems that select.select with a timeout

doesn't react to a keyboard interrupt until the timeout expires.
Specifically, if I do

s = socket.socket()
select.select(, [], [], 30)

and then press Ctrl-C, Python waits for the 30 seconds before raising
KeyboardInterrupt.

Is this a known limitation on Windows? I see no mention of it in the
documentation. Assuming it is a known limitation, is there a way round
it? (I'm writing a tiny server using asyncore/asynchat, and the
delayed response to Ctrl-C is a mild nuisance. Solutions such as "use
twisted", while probably the sensible option in a wider context, don't
really apply here - I need something within the confines of the stdlib
if it's to be worth doing).

Thanks,
Paul

 

[Antoine Pitrou]

Le Fri, 07 May 2010 16:36:44 +0200, Giampaolo Rodolà a écrit :

You can easily avoid this by setting a lower timeout when calling
asyncore.loop(), like 1 second or less (for example, Twisted uses 0.001
secs).
Actually there's no reason for asyncore to have such a high default
timeout (30 seconds).

The reason for a high default timeout would be to avoid waking the CPU
and do useless work too often. This is important on laptops and smaller
devices, in order to conserve power.

Under Unix, it's easy to have a separate fd on which you write a byte
when an signal comes, and which wakes up your select() call. Under
Windows, it may be more involved -- first because select() only takes
sockets, not pipes.

 

[Paul Moore]

You can easily avoid this by setting a lower timeout when calling
asyncore.loop(), like 1 second or less (for example, Twisted uses
0.001 secs).

Thanks, that's what I was considering.

Actually there's no reason for asyncore to have such a high default
timeout (30 seconds).

I assumed it was to avoid busy waiting.

I think this should be signaled on the bug tracker.

If a longer timeout doesn't have issues with busy waiting, then I'd agree.
Paul

 

[Giampaolo Rodolà]

Le Fri, 07 May 2010 16:36:44 +0200, Giampaolo Rodolà a écrit :

The reason for a high default timeout would be to avoid waking the CPU
and do useless work too often. This is important on laptops and smaller
devices, in order to conserve power.

Of course, but 30 seconds look a little bit too much to me, also
because (I might be wrong here) I noticed that a smaller timeout seems
to result in better performances.
Plus, if scheduled callbacks are ever gonna be added to asyncore we
would be forced to lower the default timeout anyway in order to have a
decent reactivity.


--- Giampaolo
http://code.google.com/p/pyftpdlib
http://code.google.com/p/psutil

 

[Antoine Pitrou]

Le Fri, 07 May 2010 21:55:15 +0200, Giampaolo Rodolà a écrit :

Of course, but 30 seconds look a little bit too much to me, also because
(I might be wrong here) I noticed that a smaller timeout seems to result
in better performances.

That's probably bogus.

Plus, if scheduled callbacks are ever gonna be added to asyncore we
would be forced to lower the default timeout anyway in order to have a
decent reactivity.

Why?

 

[exarkun]

Thanks, that's what I was considering.

This is a good example of why it's a bad idea to use select on Windows.
Instead, use WaitForMultipleObjects.

I assumed it was to avoid busy waiting.

If a longer timeout doesn't have issues with busy waiting, then I'd
agree.

The *default* timeout is only the default. An application that knows
better can supply a different value. I'm not sure how much good can be
done by fiddling with the default.

Jean-Paul

 

[Lawrence D'Oliveiro]

In message <[email protected]>,

This is a good example of why it's a bad idea to use select on Windows.
Instead, use WaitForMultipleObjects.

How are you supposed to write portable code, then?

 

[Giampaolo Rodolà]

Le Fri, 07 May 2010 21:55:15 +0200, Giampaolo Rodolà a écrit :

That's probably bogus.

Probably, I'll try to write a benchmark script and see what happens.

Why?

Assuming loop() function does something like this:

...
select.select(r, w, e, timeout)
scheduler() # checks for scheduled calls to be fired
...

....imagine a case where there's a connection (aka a dispatcher
instance) which does not receive or send any data *and* a scheduled
call which is supposed to be fired after, say, 5 seconds.
The entire loop would hang on select.select() which won't return for
30 seconds because the socket is not ready for reading and/or writing
resulting in scheduler() be called too late.


--- Giampaolo
http://code.google.com/p/pyftpdlib
http://code.google.com/p/psutil

 

[exarkun]

How are you supposed to write portable code, then?

With WaitForMultipleObjects on Windows, epoll on Linux, kqueue on BSD,
event completion on Solaris, etc...

Sound like more work than using select() everywhere? Yea, a bit. But
not once you abstract it away from your actual application code. After
all, it's not like these *do* different things. They all do the same
thing (basically) - differently.

Jean-Paul

 

[exarkun]

Probably, I'll try to write a benchmark script and see what happens.

Assuming loop() function does something like this:

...
select.select(r, w, e, timeout)
scheduler() # checks for scheduled calls to be fired
...

...imagine a case where there's a connection (aka a dispatcher
instance) which does not receive or send any data *and* a scheduled
call which is supposed to be fired after, say, 5 seconds.
The entire loop would hang on select.select() which won't return for
30 seconds because the socket is not ready for reading and/or writing
resulting in scheduler() be called too late.

This would be an intensely lame way to implement support for scheduled
callbacks. Try this trivial modification instead, to make it awesome:

...
select.select(r, w, e, scheduler.time_until_next_call())
scheduler.run()
...

(Or maybe just use Twisted.

Jean-Paul

 

[Antoine Pitrou]

Assuming loop() function does something like this:

...
select.select(r, w, e, timeout)
scheduler() # checks for scheduled calls to be fired
...

...imagine a case where there's a connection (aka a dispatcher
instance) which does not receive or send any data *and* a scheduled
call which is supposed to be fired after, say, 5 seconds.
The entire loop would hang on select.select() which won't return for
30 seconds because the socket is not ready for reading and/or writing
resulting in scheduler() be called too late.

Well, obviously you have to change the timeout based on the delay for
the next scheduled call. If your patch doesn't do that, it probably
needs fixing

Regards

Antoine.

 

[Giampaolo Rodolà]

Well, obviously you have to change the timeout based on the delay for
the next scheduled call. If your patch doesn't do that, it probably
needs fixing

Regards

Antoine.

No it doesn't and I didn't consider this solution at the time.
Thanks. =)


--- Giampaolo
http://code.google.com/p/pyftpdlib
http://code.google.com/p/psutil

 

[Lawrence D'Oliveiro]

In message <[email protected]>,

With WaitForMultipleObjects on Windows, epoll on Linux, kqueue on BSD,
event completion on Solaris, etc...

Sound like more work than using select() everywhere? Yea, a bit. But
not once you abstract it away from your actual application code. After
all, it's not like these *do* different things. They all do the same
thing (basically) - differently.

Do you understand what “portable†means?

 

[Paul Kölle]

Am 09.05.2010 11:59, schrieb Lawrence D'Oliveiro:

In message<[email protected]>,


Do you understand what “portable†means?

Yes. For me it means run as best as possible on all platforms I care
about. It does *not* mean force one method from *one* platform upon all
platforms for purely ideological reasons (like GTK being too stupid to
tell the difference between %HOMEPATH% and %APPDATA%).

cheers
Paul