robust clean-up with SIGTERM (was Re: Again, how to write a cleanup function for a module in C )

Discussion in 'Python' started by Jane Austine, Aug 13, 2003.

  1. Jane Austine

    Jane Austine Guest

    I'm writing a cgi script. The user can push the stop button at any
    moment while the page is loading, and the apache will raise SIGTERM
    and then SIGKILL in 3 secs. And the garbage is not collected
    automatically when the signals are raised. It's an abnormal
    termination.

    I need to clean up a few things(like closing the db connection) before
    the interpreter terminates.

    How do I?

    And according to Alex's suggestions... (see my questions below)

    "Alex Martelli" <> wrote:

    [snip]
    >
    > Are you implying that atexit doesn't? Run the following script:
    >
    > import atexit
    > def f(*args):
    > print 'f',args
    > atexit.register(f,'goo','bar','baz')
    > 1/0
    >
    > The output I see for it is as follows:
    >
    > [alex@lancelot src]$ python2.2 a.py
    > Traceback (most recent call last):
    > File "a.py", line 5, in ?
    > 1/0
    > ZeroDivisionError: integer division or modulo by zero
    > f ('goo', 'bar', 'baz')
    > [alex@lancelot src]$
    >
    >
    > Signals are nastier -- by default they terminate the process
    > WITHOUT cleanup. For example, if instead of the 1/0 you have
    > at the end of the script:
    >
    > import os, signal
    > os.kill(os.getpid(), signal.SIGIO)
    >
    > then f won't get run. Easy solution (won't work all of the
    > time, but, pretty often): handle the signal to turn it into
    > an exception! e.g. change the above two lines to:
    >
    > import os, signal
    > def sig2exc(sig, frm): raise SystemError(sig)
    > signal.signal(signal.SIGIO, sig2exc)
    > os.kill(os.getpid(), signal.SIGIO)
    >
    > and f will again execute.
    >
    >

    [snip]
    >
    > So, handle the signal, and make the program die "cleanly" with the
    > exception of your choice -- THEN, cleanup code DOES run.
    >

    [snip]

    This "signal to exception translation" looks nice. But Alex said
    "won't work all of the time, but, pretty often". I am afraid to use
    this method if its safety is not guaranteed. I wonder in what cases it
    does not work, and how I can make it work always.
    Jane Austine, Aug 13, 2003
    #1
    1. Advertising

  2. Though I'm not Alex Martelli, hope this helps: (look at the end of this
    post)

    "Jane Austine" <> wrote in message
    news:...
    > I'm writing a cgi script. The user can push the stop button at any
    > moment while the page is loading, and the apache will raise SIGTERM
    > and then SIGKILL in 3 secs. And the garbage is not collected
    > automatically when the signals are raised. It's an abnormal
    > termination.
    >
    > I need to clean up a few things(like closing the db connection) before
    > the interpreter terminates.
    >
    > How do I?
    >
    > And according to Alex's suggestions... (see my questions below)
    >
    > "Alex Martelli" <> wrote:
    >
    > [snip]
    > >
    > > Are you implying that atexit doesn't? Run the following script:
    > >
    > > import atexit
    > > def f(*args):
    > > print 'f',args
    > > atexit.register(f,'goo','bar','baz')
    > > 1/0
    > >
    > > The output I see for it is as follows:
    > >
    > > [alex@lancelot src]$ python2.2 a.py
    > > Traceback (most recent call last):
    > > File "a.py", line 5, in ?
    > > 1/0
    > > ZeroDivisionError: integer division or modulo by zero
    > > f ('goo', 'bar', 'baz')
    > > [alex@lancelot src]$
    > >
    > >
    > > Signals are nastier -- by default they terminate the process
    > > WITHOUT cleanup. For example, if instead of the 1/0 you have
    > > at the end of the script:
    > >
    > > import os, signal
    > > os.kill(os.getpid(), signal.SIGIO)
    > >
    > > then f won't get run. Easy solution (won't work all of the
    > > time, but, pretty often): handle the signal to turn it into
    > > an exception! e.g. change the above two lines to:
    > >
    > > import os, signal
    > > def sig2exc(sig, frm): raise SystemError(sig)
    > > signal.signal(signal.SIGIO, sig2exc)
    > > os.kill(os.getpid(), signal.SIGIO)
    > >
    > > and f will again execute.
    > >
    > >

    > [snip]
    > >
    > > So, handle the signal, and make the program die "cleanly" with the
    > > exception of your choice -- THEN, cleanup code DOES run.
    > >

    > [snip]
    >
    > This "signal to exception translation" looks nice. But Alex said
    > "won't work all of the time, but, pretty often". I am afraid to use
    > this method if its safety is not guaranteed. I wonder in what cases it
    > does not work, and how I can make it work always.


    When signals are generated successively in a very short time, python
    interpreter might screw up some of them. Basically, the interpreter checks
    every checkinterval(default 100) if it has pending signal handler calls,
    and process them. When a signal is generated, it registers the signal and
    put its handler on the pending calls. As you can see there are potential
    race conditions.

    Check signalmodule.c and ceval.c.

    I don't have a good idea on making signal handling always safe &
    guaranteed. Maybe someone else can help?
    Changjune Kim, Aug 15, 2003
    #2
    1. Advertising

  3. Jane Austine

    Donn Cave Guest

    Quoth "Changjune Kim" <>:
    [ ... re why signal handling may not be 100% reliable ]

    | When signals are generated successively in a very short time, python
    | interpreter might screw up some of them. Basically, the interpreter checks
    | every checkinterval(default 100) if it has pending signal handler calls,
    | and process them. When a signal is generated, it registers the signal and
    | put its handler on the pending calls. As you can see there are potential
    | race conditions.
    |
    | Check signalmodule.c and ceval.c.
    |
    | I don't have a good idea on making signal handling always safe &
    | guaranteed. Maybe someone else can help?

    Well, there are ways to defer signal delivery while you process
    the signals that have already been delivered -

    /* (untested) */
    sigset_t mask, prev_mask;
    sigfillset(&mask);
    sigprocmask(SIG_SETMASK, &mask, &prev_mask);
    /* All signals now blocked. */
    ... process state from signal deliveries
    sigprocmask(SIG_SETMASK, &prev_mask, 0);

    and you can set a signal handler up such that signals are blocked
    while it executes -

    sa.sa_handler = signal_handler;
    sa.sa_flags = 0;
    sigfillset(&sa.sa_mask);
    sigaction(SIGTERM, &sa, 0);

    This assumes signal handling per POSIX 1003.1, which may be part
    of the reason if Python isn't trying to do it. Then there are
    some naturally intrinsic problems with signal handling in Python
    that make delivery unreliable in a different way - your signal
    handler may be invoked much later than the signal is delivered.
    (And of course if you're using threads, there are more problems.)
    That's only relevant to the present question because, given the
    unavoidable weaknesses in Python signal handling, it may not be
    worth even a slight headache from platforms that don't support
    POSIX signal handling or don't support it correctly.

    Donn Cave,
    Donn Cave, Aug 16, 2003
    #3
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Teresa
    Replies:
    2
    Views:
    639
    Rob Thorpe
    Apr 6, 2004
  2. Zoran Bosnjak

    popen3,4 and SIGTERM

    Zoran Bosnjak, Oct 15, 2003, in forum: Python
    Replies:
    0
    Views:
    399
    Zoran Bosnjak
    Oct 15, 2003
  3. Replies:
    8
    Views:
    500
  4. Replies:
    0
    Views:
    231
  5. Mr.M
    Replies:
    6
    Views:
    754
Loading...

Share This Page