Does Boost.Python participate in cyclic gc?

[skip]

We encountered a situation today where it appeared that a
Boost.Python-provided class didn't participate in Python's cyclic garbage
collection. The wrapped C++ instance held a reference to a method in the
Python object which referenced the Boostified C++ instance, e.g.:

class Foo:
def __init__(self, ...):
self.over_there = BoostifiedClass()
self.over_there.set_callback(self.boost_callback)
...

def boost_callback(self, ...):
...

When an instance of Foo went out of scope the BoostifiedClass instance it
referred to didn't disappear. (We could tell because it made connections to
another server which didn't disappear.) The solution was to recognize when
we where finished with it to set self.over_there to None. It would be nice
if Boost.Python knew how to play in the gc sandbox, but I suspect it may
not.

Thx,

Skip

 

[Matimus]

The solution was to recognize when we where finished with it to set self.over_there to None.

To my knowledge python does not handle all cyclic gc anyway. Here is
some information from the gc documentation:

[doc]
garbage
A list of objects which the collector found to be unreachable but
could not be freed (uncollectable objects). By default, this list
contains only objects with __del__() methods.3.1Objects that have
__del__() methods and are part of a reference cycle cause the entire
reference cycle to be uncollectable, including objects not necessarily
in the cycle but reachable only from it. Python doesn't collect such
cycles automatically because, in general, it isn't possible for Python
to guess a safe order in which to run the __del__() methods. If you
know a safe order, you can force the issue by examining the garbage
list, and explicitly breaking cycles due to your objects within the
list. Note that these objects are kept alive even so by virtue of
being in the garbage list, so they should be removed from garbage too.
For example, after breaking cycles, do del gc.garbage[:] to empty the
list. It's generally better to avoid the issue by not creating cycles
containing objects with __del__() methods, and garbage can be examined
in that case to verify that no such cycles are being created.
[/doc]

So, do BoostFieldClass and Foo both implement __del__? Is there some
way you can eliminate one of them?

I can reproduce something similar with the code below, and I'm not
using Boost.Python.

class Bar:
    def set_callback(self,cb):
        self.cb = cb
    def __del__(self):
        pass


class Foo:
    def __init__(self):
        self.over_there = Bar()
        self.over_there.set_callback(self.boost_callback)

    def boost_callback(self):
        print "Boost callback"

    def __del__(self):
        pass

if __name__ == "__main__":
    import gc
    f = Foo()

    del(f)

    gc.collect()

    print dir()
    print gc.garbage

 

[Giovanni Bajo]

We encountered a situation today where it appeared that a
Boost.Python-provided class didn't participate in Python's cyclic garbage
collection. The wrapped C++ instance held a reference to a method in the
Python object which referenced the Boostified C++ instance, e.g.:

Those situations get more and more common as you keep wrapping your code. To
the best of my knowledge, SIP is the only one which does the right thing here.