Circular references not being cleaned up by Py_Finalize()

[blackpawn]

I've been trying to get garbage collection of circular references to
work properly with no success then I noticed the documentation stating
that it just doesn't:

From documentation on Py_Finalize() -> "Memory tied up in circular
references between objects is not freed. "

I copy pasted the Noddy example for circular reference breaking from
the documentation and set a break point in its Noddy_dealloc function
and sure enough even that simple example doesn't work properly.

So what's the deal here? I want all objects to be freed when I
shut down and their destruction functions to be properly called. Is
there really no way to make this happen?

Many thanks for any insights,
Jim

 

[Marc 'BlackJack' Rintsch]

So what's the deal here? I want all objects to be freed when I
shut down and their destruction functions to be properly called.

Then you want something that's not guaranteed by the language.

Ciao,
Marc 'BlackJack' Rintsch

 

[Christian Heimes]

So what's the deal here? I want all objects to be freed when I
shut down and their destruction functions to be properly called. Is
there really no way to make this happen?

Does the Noddy example use GC (Py_TPFLAGS_HAVE_GC)? Container objects
must use the cycle GC or circular referneces aren't broken. Have you
tried calling PyGC_Collect() multiple times?

Christian

 

[Paul McGuire]

I've been trying to get garbage collection of circular references to
work properly with no success then I noticed the documentation stating
that it just doesn't:

From documentation on Py_Finalize() -> "Memory tied up in circular
references between objects is not freed. "

I copy pasted the Noddy example for circular reference breaking from
the documentation and set a break point in its Noddy_dealloc function
and sure enough even that simple example doesn't work properly.

So what's the deal here?    I want all objects to be freed when I
shut down and their destruction functions to be properly called.  Is
there really no way to make this happen?

Many thanks for any insights,
Jim

It sounds like you are using finalizers like destructors in C++.
Don't do this. Garbage collection is not deterministic, and is not
supposed to be.

Instead, use language features that do similar scope-driven code
execution:
- try/except/finally
- with

Put your cleanup code in the finally block, or in the context managers
__exit__ method. These will run when the program exits the given
scope, without having to play GC shenanigans.

-- Paul

 

[blackpawn]

Does the Noddy example use GC (Py_TPFLAGS_HAVE_GC)? Container objects

must use the cycle GC or circular referneces aren't broken. Have you
tried calling PyGC_Collect() multiple times?

Yeah the Noddy example is from "2.1.3 Supporting cyclic garbage
collection" part of the Python docs. They list sample C and Python
code which I cut and pasted with no changes and on Py_Finalize and app
shut down it leaks the object. I know the garbage collector is
tracking the object because it properly calls the traverse function
but for whatever reason it never calls the clear function. Does
anyone have experience with circular references and had success with
it or the example in the docs?

 

[Hrvoje Niksic]

I know the garbage collector is tracking the object because it
properly calls the traverse function but for whatever reason it
never calls the clear function. Does anyone have experience with
circular references and had success with it or the example in the
docs?

I've now tried it, and it seems to work. Modifying Nody_dealloc with
printf("collecting %p\n", self), it works like this:
collecting 0xb7d3bf2c

So the garbage-collector is invoked at least once before shutdown.
However, it doesn't work without "del n" line, so it would appear that
the gc is not invoked after the module cleanup and therefore doesn't
explicitly dealloc global objects that contain cycles. You can work
around that by using the atexit module to register a cleanup function
that removes your global objects from the module namespace(s) and, if
necessary, invokes gc.collect() manually. That way your objects end
up as reclaimable garbage.

This won't work if your cycle-containing objects are indirectly
reachable from the module namespace through other containers not under
your control. In that case, you should really rethink your strategy
of depending on finalizers to run and perhaps use atexit to do your
exit processing or, as others pointed out, use scope tools
(try/finally, with) at the entry point of your code.