C API: module cleanup function

[Mr.M]

Hi,

I can't figure out if there is a way to run a specialized cleanup
function when a module needs to be "unloaded" (i.e. just before a
reload() or when i quit the interpreter).

I'm thinking of something like tp_dealloc.

If I call Py_InitModule3 and look at module->ob_type->tp_dealloc, I find
that Python provides a default tp_dealloc for me.

Now, suppose my module needs to allocate some resources at startup, I'm
not sure, but I think I'd have to do it in my PyMODINIT_FUNC, right?

But, if I reload() my module or if I quit the Python interpreter, I'd
like to free those resources (before allocate them again, in case of a
reload).

Is there a way to make this work?

Thank you, Luca.

 

[Gabriel Genellina]

I can't figure out if there is a way to run a specialized cleanup
function when a module needs to be "unloaded" (i.e. just before a
reload() or when i quit the interpreter).

I think what you want to do isn't possible with Python 2, and it's one of
the reasons the module handling was redesigned in Python 3.x; see PEP 3121.

I'm thinking of something like tp_dealloc.

m_free (PyModuleDef member, in Python 3) might work for this.

 

[Mr.M]

Gabriel Genellina ha scritto:

I think what you want to do isn't possible with Python 2, and it's one
of the reasons the module handling was redesigned in Python 3.x; see PEP
3121.

Thank you Gabriel for your help. Unlucky I can't use Python 3.x in my
project, sob!

Luca.

 

[Stefan Behnel]

Mr.M, 29.01.2010 23:50:

I can't figure out if there is a way to run a specialized cleanup
function when a module needs to be "unloaded" (i.e. just before a
reload() or when i quit the interpreter).

I'm thinking of something like tp_dealloc.

If I call Py_InitModule3 and look at module->ob_type->tp_dealloc, I find
that Python provides a default tp_dealloc for me.

Now, suppose my module needs to allocate some resources at startup, I'm
not sure, but I think I'd have to do it in my PyMODINIT_FUNC, right?

But, if I reload() my module or if I quit the Python interpreter, I'd
like to free those resources (before allocate them again, in case of a
reload).

Is there a way to make this work?

Gabriel already pointed you to the module cleanup support in Py3, which can
be used to provide reload capabilities to your module.

In Py2, there are at least some ways to free resources when terminating the
interpreter. See the "atexit" module and the Py_AtExit() function:

http://docs.python.org/library/atexit.html
http://docs.python.org/c-api/sys.html

Note that both have their specific limitations, though, as you can see from
the docs.

Also note that it might help you to take a look at Cython, a Python-to-C
compiler for writing fast C extensions. It has an option for generating
module-level cleanup code automatically, and generally simplifies writing
binary extension modules quite a bit.

Stefan

 

[Mr.M]

Stefan Behnel ha scritto:

Gabriel already pointed you to the module cleanup support in Py3, which can
be used to provide reload capabilities to your module.

In Py2, there are at least some ways to free resources when terminating the
interpreter. See the "atexit" module and the Py_AtExit() function:

http://docs.python.org/library/atexit.html
http://docs.python.org/c-api/sys.html

Note that both have their specific limitations, though, as you can see from
the docs.

Also note that it might help you to take a look at Cython, a Python-to-C
compiler for writing fast C extensions. It has an option for generating
module-level cleanup code automatically, and generally simplifies writing
binary extension modules quite a bit.

Stefan

Thank you very much Stefan for your reply, I'll study the sources you
have pointed me to.

Could I allocate my resources in a "static" object (without publishing
the type of that object so that I can't instantiate another one) linked
to my module?
This way, when I stop the interpreter, the object will be destroyed
calling its destructor.

There's something I'm missing?

Thank you, Luca.

 

[Stefan Behnel]

Mr.M, 30.01.2010 14:24:

Could I allocate my resources in a "static" object linked to my module?

Sure, but you will have to let CPython know about it so that it can see
that the reference is only held by the module that it is cleaning up.
Otherwise it can't collect the reference. This works easily in Py3 but not
in Py2.

In Cython, you would just write

cdef ResourceKeeper resource_keeper = ResourceKeeper()

at the module level and let Cython generate the cleanup code for it. In C,
you have to implement an atexit function and decref the resource object
either there or in the module cleanup function of Py3.

(without publishing
the type of that object so that I can't instantiate another one)

Note that there is the type() builtin function which returns the type given
an instance. So you can't hide the type.

Stefan

 

[Mr.M]

Stefan Behnel ha scritto:

Note that there is the type() builtin function which returns the type given
an instance. So you can't hide the type.

Argh! Yes, you are right.

So I'd better have a look to Cython, right?

L-