robust clean-up with SIGTERM (was Re: Again, how to write a cleanup function for a module in C )

[Jane Austine]

I'm writing a cgi script. The user can push the stop button at any
moment while the page is loading, and the apache will raise SIGTERM
and then SIGKILL in 3 secs. And the garbage is not collected
automatically when the signals are raised. It's an abnormal
termination.

I need to clean up a few things(like closing the db connection) before
the interpreter terminates.

How do I?

And according to Alex's suggestions... (see my questions below)


[snip]

Are you implying that atexit doesn't? Run the following script:

import atexit
def f(*args):
print 'f',args
atexit.register(f,'goo','bar','baz')
1/0

The output I see for it is as follows:

[alex@lancelot src]$ python2.2 a.py
Traceback (most recent call last):
File "a.py", line 5, in ?
1/0
ZeroDivisionError: integer division or modulo by zero
f ('goo', 'bar', 'baz')
[alex@lancelot src]$


Signals are nastier -- by default they terminate the process
WITHOUT cleanup. For example, if instead of the 1/0 you have
at the end of the script:

import os, signal
os.kill(os.getpid(), signal.SIGIO)

then f won't get run. Easy solution (won't work all of the
time, but, pretty often): handle the signal to turn it into
an exception! e.g. change the above two lines to:

import os, signal
def sig2exc(sig, frm): raise SystemError(sig)
signal.signal(signal.SIGIO, sig2exc)
os.kill(os.getpid(), signal.SIGIO)

and f will again execute.

[snip]

So, handle the signal, and make the program die "cleanly" with the
exception of your choice -- THEN, cleanup code DOES run.

[snip]

This "signal to exception translation" looks nice. But Alex said
"won't work all of the time, but, pretty often". I am afraid to use
this method if its safety is not guaranteed. I wonder in what cases it
does not work, and how I can make it work always.

 

[Changjune Kim]

Though I'm not Alex Martelli, hope this helps: (look at the end of this
post)

I'm writing a cgi script. The user can push the stop button at any
moment while the page is loading, and the apache will raise SIGTERM
and then SIGKILL in 3 secs. And the garbage is not collected
automatically when the signals are raised. It's an abnormal
termination.

I need to clean up a few things(like closing the db connection) before
the interpreter terminates.

How do I?

And according to Alex's suggestions... (see my questions below)


[snip]

Are you implying that atexit doesn't? Run the following script:

import atexit
def f(*args):
print 'f',args
atexit.register(f,'goo','bar','baz')
1/0

The output I see for it is as follows:

[alex@lancelot src]$ python2.2 a.py
Traceback (most recent call last):
File "a.py", line 5, in ?
1/0
ZeroDivisionError: integer division or modulo by zero
f ('goo', 'bar', 'baz')
[alex@lancelot src]$


Signals are nastier -- by default they terminate the process
WITHOUT cleanup. For example, if instead of the 1/0 you have
at the end of the script:

import os, signal
os.kill(os.getpid(), signal.SIGIO)

then f won't get run. Easy solution (won't work all of the
time, but, pretty often): handle the signal to turn it into
an exception! e.g. change the above two lines to:

import os, signal
def sig2exc(sig, frm): raise SystemError(sig)
signal.signal(signal.SIGIO, sig2exc)
os.kill(os.getpid(), signal.SIGIO)

and f will again execute.

[snip]

So, handle the signal, and make the program die "cleanly" with the
exception of your choice -- THEN, cleanup code DOES run.

[snip]

This "signal to exception translation" looks nice. But Alex said
"won't work all of the time, but, pretty often". I am afraid to use
this method if its safety is not guaranteed. I wonder in what cases it
does not work, and how I can make it work always.

When signals are generated successively in a very short time, python
interpreter might screw up some of them. Basically, the interpreter checks
every checkinterval(default 100) if it has pending signal handler calls,
and process them. When a signal is generated, it registers the signal and
put its handler on the pending calls. As you can see there are potential
race conditions.

Check signalmodule.c and ceval.c.

I don't have a good idea on making signal handling always safe &
guaranteed. Maybe someone else can help?

 

[Donn Cave]

Quoth "Changjune Kim" <[email protected]>:
[ ... re why signal handling may not be 100% reliable ]

| When signals are generated successively in a very short time, python
| interpreter might screw up some of them. Basically, the interpreter checks
| every checkinterval(default 100) if it has pending signal handler calls,
| and process them. When a signal is generated, it registers the signal and
| put its handler on the pending calls. As you can see there are potential
| race conditions.
|
| Check signalmodule.c and ceval.c.
|
| I don't have a good idea on making signal handling always safe &
| guaranteed. Maybe someone else can help?

Well, there are ways to defer signal delivery while you process
the signals that have already been delivered -

/* (untested) */
sigset_t mask, prev_mask;
sigfillset(&mask);
sigprocmask(SIG_SETMASK, &mask, &prev_mask);
/* All signals now blocked. */
... process state from signal deliveries
sigprocmask(SIG_SETMASK, &prev_mask, 0);

and you can set a signal handler up such that signals are blocked
while it executes -

sa.sa_handler = signal_handler;
sa.sa_flags = 0;
sigfillset(&sa.sa_mask);
sigaction(SIGTERM, &sa, 0);

This assumes signal handling per POSIX 1003.1, which may be part
of the reason if Python isn't trying to do it. Then there are
some naturally intrinsic problems with signal handling in Python
that make delivery unreliable in a different way - your signal
handler may be invoked much later than the signal is delivered.
(And of course if you're using threads, there are more problems.)
That's only relevant to the present question because, given the
unavoidable weaknesses in Python signal handling, it may not be
worth even a slight headache from platforms that don't support
POSIX signal handling or don't support it correctly.

Donn Cave, (e-mail address removed)