no cleanup on TERM signal

[Yves Dorfsman]

I did a few tests with this script:

class byebye:

def __del__(self):
print 'Bye, bye...'


x = byebye()


x.del() gets executed if:
-I del x, then run gc.collect()
-simply exit the script
-get the script to abort on an exception

But if I kill it with the default signal TERM, the script dies, but I don't
get the message, so I am assuming that python isn't taking the time to
cleanup, even though that is (was) what TERM was intended for.

Has this been discussed before ? Is worth a suggestion (PEP) ?

 

[Marc 'BlackJack' Rintsch]

x.del() gets executed if:
-I del x, then run gc.collect()
-simply exit the script
-get the script to abort on an exception

But if I kill it with the default signal TERM, the script dies, but I don't
get the message, so I am assuming that python isn't taking the time to
cleanup, even though that is (was) what TERM was intended for.

Has this been discussed before ? Is worth a suggestion (PEP) ?

There is the docs for `__del__()` saying this method is not guaranteed to
be called at all. Don't use it if you *need* that method to be called.
Just like `finalize()` in Java, it can't be used for deterministic
destruction, so it's not that useful after all.

Ciao,
Marc 'BlackJack' Rintsch

 

[Laszlo Nagy]

I did a few tests with this script:

class byebye:

def __del__(self):
print 'Bye, bye...'


x = byebye()


x.del() gets executed if:
-I del x, then run gc.collect()
-simply exit the script
-get the script to abort on an exception

But if I kill it with the default signal TERM, the script dies, but I
don't get the message, so I am assuming that python isn't taking the
time to cleanup, even though that is (was) what TERM was intended for.

TERM signal is unix specific. There is no special syntax/programming
structure for signals inside the Python language, since it would be
platform dependent. For simple client programs, usually it is not needed
to setup signal handlers because they can easily be controlled in other
ways.

For sensitive resources, instead of writing __del__ methods, you should
create a "close()" method. Python does this with file objects, DB API
2.0 with database connection objects etc. Then you can do

res = create_resource()
try:
use_resource()
finally:
res.close() # Must free resource, but the object can still be alive...

It is more common to use signals when you have more threads or child
processes. You can use something like:

import threading
import signal

stop_requested = threading.Event()
exited_on_sigterm = False

def free_all_resources():
pass # Free your resources here!

def sigterm_handler(signum, frame):
"""Stop the server gracefully when on SIGTERM."""
global stop_requested
global exited_on_sigterm
exited_on_sigterm = True
stop_requested.set()
free_all_resources()

def main():
global stop_requested
global exited_on_sigterm

logger = servicelog.getLogger('main',filename=LOGFILENAME)

logger.info('Setting up the SIGTERM signal handler.')
signal.signal(signal.SIGTERM, sigterm_handler) # Setup signal handler

logger.info('Starting watchdog thread')
watchdog = WatchDog()
watchdog.start()

worker1 = create_worker(stop_requested)
worker2 = create_worker(stop_requested)
# etc. Prepare things here...

try:
try:
server = create_my_server()
server.serve_until_not_stopped()
except Exception, e:
logger.error(dumpexc(e))
raise e
finally:
stop_requested.set() # Ask other threads to stop cooperatively

# Join all threads here
watchdog.join()
worker1.join()
worder2.join()
# etc. wait for all threads to exit cooperatively.
if exited_on_sigterm:
logger.warning('Exited on SIGTERM!')

Best,

Laszlo

 

[Christian Heimes]

For sensitive resources, instead of writing __del__ methods, you should
create a "close()" method. Python does this with file objects, DB API
2.0 with database connection objects etc. Then you can do

res = create_resource()
try:
use_resource()
finally:
res.close() # Must free resource, but the object can still be alive...

You can replace the try/finally code with a "with resource:
do_something()" block if the object supporst the context manager protocol.

If you want to run some code during the shutdown phase of the Python
process you can register a callback function in the "atexit" module.

It is more common to use signals when you have more threads or child
processes. You can use something like:

Do NOT mix threads and signals. It's usually a very bad idea and may
lead to surprising side effects. Signals are only handled by the main
thread.

Christian

 

[Laszlo Nagy]

You can replace the try/finally code with a "with resource:
do_something()" block if the object supporst the context manager protocol.
I'm using 2.4 so...
If you want to run some code during the shutdown phase of the Python
process you can register a callback function in the "atexit" module.

I also wanted to tell this to the OP but he was asking about signals.
AFAIK atexit is only called on SIGTERM, not on others.

Do NOT mix threads and signals. It's usually a very bad idea and may
lead to surprising side effects. Signals are only handled by the main
thread.

Well, I have to, because:

#1. I have a server that uses threads. Using fork() instead of threading
would be a nightmare in my case.
#2. It must support signals, just because some users tend to use things
like "killall python" and try/finally is not useful in those cases. I
cannot do anything to prevent the users killing the server. They will
just do it...

I know that signals are handled in the main thread only. This is why I'm
using a threading.Event() object to allow all threads shut down
cooperatively: sigterm will set the event, and then all threads
(including the main thread) will exit gracefully (within a given
timeout). Do you know a better solution?

Thanks,

Laszlo