no cleanup on TERM signal

Discussion in 'Python' started by Yves Dorfsman, May 2, 2008.

  1. I did a few tests with this script:

    class byebye:

    def __del__(self):
    print 'Bye, bye...'


    x = byebye()


    x.del() gets executed if:
    -I del x, then run gc.collect()
    -simply exit the script
    -get the script to abort on an exception

    But if I kill it with the default signal TERM, the script dies, but I don't
    get the message, so I am assuming that python isn't taking the time to
    cleanup, even though that is (was) what TERM was intended for.

    Has this been discussed before ? Is worth a suggestion (PEP) ?


    --
    Yves.
    http://www.SollerS.ca
    Yves Dorfsman, May 2, 2008
    #1
    1. Advertising

  2. On Fri, 02 May 2008 04:36:06 +0000, Yves Dorfsman wrote:

    > x.del() gets executed if:
    > -I del x, then run gc.collect()
    > -simply exit the script
    > -get the script to abort on an exception
    >
    > But if I kill it with the default signal TERM, the script dies, but I don't
    > get the message, so I am assuming that python isn't taking the time to
    > cleanup, even though that is (was) what TERM was intended for.
    >
    > Has this been discussed before ? Is worth a suggestion (PEP) ?


    There is the docs for `__del__()` saying this method is not guaranteed to
    be called at all. Don't use it if you *need* that method to be called.
    Just like `finalize()` in Java, it can't be used for deterministic
    destruction, so it's not that useful after all.

    Ciao,
    Marc 'BlackJack' Rintsch
    Marc 'BlackJack' Rintsch, May 2, 2008
    #2
    1. Advertising

  3. Yves Dorfsman

    Laszlo Nagy Guest

    Yves Dorfsman wrote:
    > I did a few tests with this script:
    >
    > class byebye:
    >
    > def __del__(self):
    > print 'Bye, bye...'
    >
    >
    > x = byebye()
    >
    >
    > x.del() gets executed if:
    > -I del x, then run gc.collect()
    > -simply exit the script
    > -get the script to abort on an exception
    >
    > But if I kill it with the default signal TERM, the script dies, but I
    > don't get the message, so I am assuming that python isn't taking the
    > time to cleanup, even though that is (was) what TERM was intended for.

    TERM signal is unix specific. There is no special syntax/programming
    structure for signals inside the Python language, since it would be
    platform dependent. For simple client programs, usually it is not needed
    to setup signal handlers because they can easily be controlled in other
    ways.

    For sensitive resources, instead of writing __del__ methods, you should
    create a "close()" method. Python does this with file objects, DB API
    2.0 with database connection objects etc. Then you can do

    res = create_resource()
    try:
    use_resource()
    finally:
    res.close() # Must free resource, but the object can still be alive...

    It is more common to use signals when you have more threads or child
    processes. You can use something like:

    import threading
    import signal

    stop_requested = threading.Event()
    exited_on_sigterm = False

    def free_all_resources():
    pass # Free your resources here!

    def sigterm_handler(signum, frame):
    """Stop the server gracefully when on SIGTERM."""
    global stop_requested
    global exited_on_sigterm
    exited_on_sigterm = True
    stop_requested.set()
    free_all_resources()

    def main():
    global stop_requested
    global exited_on_sigterm

    logger = servicelog.getLogger('main',filename=LOGFILENAME)

    logger.info('Setting up the SIGTERM signal handler.')
    signal.signal(signal.SIGTERM, sigterm_handler) # Setup signal handler

    logger.info('Starting watchdog thread')
    watchdog = WatchDog()
    watchdog.start()

    worker1 = create_worker(stop_requested)
    worker2 = create_worker(stop_requested)
    # etc. Prepare things here...

    try:
    try:
    server = create_my_server()
    server.serve_until_not_stopped()
    except Exception, e:
    logger.error(dumpexc(e))
    raise e
    finally:
    stop_requested.set() # Ask other threads to stop cooperatively

    # Join all threads here
    watchdog.join()
    worker1.join()
    worder2.join()
    # etc. wait for all threads to exit cooperatively.
    if exited_on_sigterm:
    logger.warning('Exited on SIGTERM!')

    Best,

    Laszlo
    Laszlo Nagy, May 2, 2008
    #3
  4. Laszlo Nagy schrieb:
    > For sensitive resources, instead of writing __del__ methods, you should
    > create a "close()" method. Python does this with file objects, DB API
    > 2.0 with database connection objects etc. Then you can do
    >
    > res = create_resource()
    > try:
    > use_resource()
    > finally:
    > res.close() # Must free resource, but the object can still be alive...


    You can replace the try/finally code with a "with resource:
    do_something()" block if the object supporst the context manager protocol.

    If you want to run some code during the shutdown phase of the Python
    process you can register a callback function in the "atexit" module.

    > It is more common to use signals when you have more threads or child
    > processes. You can use something like:


    Do NOT mix threads and signals. It's usually a very bad idea and may
    lead to surprising side effects. Signals are only handled by the main
    thread.

    Christian
    Christian Heimes, May 2, 2008
    #4
  5. Yves Dorfsman

    Laszlo Nagy Guest


    > You can replace the try/finally code with a "with resource:
    > do_something()" block if the object supporst the context manager protocol.
    >

    I'm using 2.4 so...
    > If you want to run some code during the shutdown phase of the Python
    > process you can register a callback function in the "atexit" module.
    >

    I also wanted to tell this to the OP but he was asking about signals.
    AFAIK atexit is only called on SIGTERM, not on others.
    > Do NOT mix threads and signals. It's usually a very bad idea and may
    > lead to surprising side effects. Signals are only handled by the main
    > thread.
    >

    Well, I have to, because:

    #1. I have a server that uses threads. Using fork() instead of threading
    would be a nightmare in my case.
    #2. It must support signals, just because some users tend to use things
    like "killall python" and try/finally is not useful in those cases. I
    cannot do anything to prevent the users killing the server. They will
    just do it...

    I know that signals are handled in the main thread only. This is why I'm
    using a threading.Event() object to allow all threads shut down
    cooperatively: sigterm will set the event, and then all threads
    (including the main thread) will exit gracefully (within a given
    timeout). Do you know a better solution?

    Thanks,

    Laszlo
    Laszlo Nagy, May 2, 2008
    #5
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. John C. Worsley
    Replies:
    0
    Views:
    1,399
    John C. Worsley
    Jan 21, 2004
  2. Diez B. Roggisch

    twisted signal handler override or cleanup code

    Diez B. Roggisch, Sep 1, 2004, in forum: Python
    Replies:
    1
    Views:
    538
    Jp Calderone
    Sep 1, 2004
  3. Michael Pronath
    Replies:
    1
    Views:
    1,166
    Diez B. Roggisch
    Jan 3, 2005
  4. Jack Orenstein

    threading.Thread vs. signal.signal

    Jack Orenstein, Sep 18, 2005, in forum: Python
    Replies:
    0
    Views:
    464
    Jack Orenstein
    Sep 18, 2005
  5. Weng Tianxiang
    Replies:
    2
    Views:
    658
    Jonathan Bromley
    Jan 30, 2007
Loading...

Share This Page