Threads, signals and sockets (on UNIX)

[geoffbache]

Hi all,

I have a Python program (on UNIX) whose main job is to listen on a
socket, for which I use the SocketServer module. However, I would also
like it to be sensitive to signals received, which it isn't if it's
listening on the socket. ("signals can only be received between atomic
actions of the python interpreter", presumably - and control will not
return to Python unless something appears on the socket). Does anyone
have a tip of a good way to do this?

I can of course put the SocketServer in a thread and call
signal.pause() in the main thread, but this falls down when the
SocketServer terminates normally, as the signal.pause() cannot be
interrupted except via signals. So I tried sending a "dummy" signal
(SIGCHLD) from the thread when the SocketServer terminates, which
seems to work on Linux but not Solaris. And which in any case feels a
bit hackish - surely there has to be a better way?

Regards,
Geoff Bache

 

[Stefan Behnel]

I have a Python program (on UNIX) whose main job is to listen on a
socket, for which I use the SocketServer module. However, I would also
like it to be sensitive to signals received, which it isn't if it's
listening on the socket. ("signals can only be received between atomic
actions of the python interpreter", presumably - and control will not
return to Python unless something appears on the socket). Does anyone
have a tip of a good way to do this?

Twisted *should* be able to do this, as it uses non-blocking IO.

http://twistedmatrix.com/trac/

Stefan

 

[geoffbache]

Twisted *should* be able to do this, as it uses non-blocking IO.

http://twistedmatrix.com/trac/

Thanks for the tip. I'll take a look if nobody has any better
suggestions.

It still seems to me that what I'm trying to do is essentially quite
simple, and shouldn't require
as large a tool as Twisted to fix it. Isn't Twisted basically for web
applications?

Geoff

 

[Chaz Ginger]

Thanks for the tip. I'll take a look if nobody has any better
suggestions.

It still seems to me that what I'm trying to do is essentially quite
simple, and shouldn't require
as large a tool as Twisted to fix it. Isn't Twisted basically for web
applications?

Geoff

You could probably use the Asyncore stuff to do it as well (with a lot
less stuff).

C.

 

[geoffbache]

You could probably use the Asyncore stuff to do it as well (with a lot
less stuff).

This looked interesting. But it seems the asyncore stuff operates at
the socket level,
whereas I've currently just got a standard synchronous SocketServer
and the socket
operations themselves are kind of hidden beneath this layer. Can you
point me at anything
that might tell me how to combine Asyncore with SocketServer,
preferably without having to
mess with the internals of SocketServer too much

Geoff

 

[geoffbache]

Twisted is a pretty good suggestion in general.

Twisted supports HTTP, but it does plenty of other things too. Generally
speaking, it's useful for any network application, plus some other stuff.

My application is only incidentally a network application. It doesn't
have clients
and servers as such, it just distributes its work via a grid engine
and then lets
these workers communicate back their results via sockets.

You're half right about this being simple though, and not needing Twisted
to solve the problem. The only thing you need to do to solve the problem
is avoid using either signals or threads. Interaction between the two is
very complicated and, as you've noticed, varies across platforms. Twisted
is solving the problem for you here by letting you do I/O without using
threads, making signals *almost* simple.

Yes, I would avoid signals or threads if I could, but it's tough. The
program
is supposed to "appear" to just be a batch process, so handling e.g.
ctrl-C is essential.
The standard SocketServer doesn't allow for this, so I need some other
thread of control
that will, or some means of "asynchronising" SocketServer internally.

Or there's always the really hacky low tech solution which has a
certain appeal : have
the main thread check all the others for being alive and sleep in
between...

Geoff

 

[fumanchu]

I have a Python program (on UNIX) whose main job is to listen on a
socket, for which I use the SocketServer module. However, I would also
like it to be sensitive to signals received, which it isn't if it's
listening on the socket. ("signals can only be received between atomic
actions of the python interpreter", presumably - and control will not
return to Python unless something appears on the socket). Does anyone
have a tip of a good way to do this?

I can of course put the SocketServer in a thread and call
signal.pause() in the main thread, but this falls down when the
SocketServer terminates normally, as the signal.pause() cannot be
interrupted except via signals. So I tried sending a "dummy" signal
(SIGCHLD) from the thread when the SocketServer terminates, which
seems to work on Linux but not Solaris. And which in any case feels a
bit hackish - surely there has to be a better way?

Use a timeout on your socket and put socket.accept() in a loop:

mainsocket.settimeout(1)

while True:
try:
childsocket, addr = mainsocket.accept()
handle(childsocket, addr)
except socket.timeout:
pass


Robert Brewer
System Architect
Amor Ministries
(e-mail address removed)