Cyclic garbage collection and segfaults...

Discussion in 'Python' started by Thomas Mailund, Jan 14, 2004.

  1. Hi group.

    I have a problem with some C extensions I am working with and
    hope that some of you can help. Basically, I am wrapping a a tree
    structure from C where I have python methods for extracting either the
    entire tree or subtrees; since I don't want the full tree to be
    deallocated while python has references to the subtrees, I INCREF the
    full tree whenever I hand out a subtree reference. I don't want any of
    the subtrees to be deallocated while the full tree lives, either, so I
    want the fulltree to have a reference to each of the subtrees. Naturally,
    this gives me a cyclik reference structure, and I want to be able to
    garbage collect it. But this is where my problems begin...

    For some reason, when I turn on the cyclic garbage collection for my
    types, a deallocation automatically gives me a segfault. That is, if my
    type new type contains any data whatsoever, I cannot garbage collect
    without dumping core. I have boiled the problem down to the code shown
    below. It contains no cyclic structure, it is as simple as it gets, but
    it still seqfaults for me. If I comment out the magic void-pointer (which
    isn't used for anything) I don't get the segfault.


    #include <Python.h>

    struct SimpleObject;

    static PyObject * Simple_new (PyTypeObject *type,
    PyObject *args,
    PyObject *kwds);
    static int Simple_init (struct SimpleObject *self,
    PyObject *args,
    PyObject *kwds);
    static int Simple_traverse(struct SimpleObject *self,
    visitproc visit,
    void *arg);
    static void Simple_dealloc (struct SimpleObject *self);




    typedef struct SimpleObject {
    /* object stuff */
    PyObject_HEAD

    /* dark magic */
    void *magic;

    } SimpleObject;


    static PyTypeObject simple_SimpleType = {
    PyObject_HEAD_INIT(NULL)
    0, /*ob_size*/
    "simple.Simple", /*tp_name*/
    sizeof(SimpleObject), /*tp_basicsize*/
    0, /*tp_itemsize*/
    (destructor)Simple_dealloc, /*tp_dealloc*/
    0, /*tp_print*/
    0, /*tp_getattr*/
    0, /*tp_setattr*/
    0, /*tp_compare*/
    0, /*tp_repr*/
    0, /*tp_as_number*/
    0, /*tp_as_sequence*/
    0, /*tp_as_mapping*/
    0, /*tp_hash */
    0, /*tp_call*/
    0, /*tp_str*/
    0, /*tp_getattro*/
    0, /*tp_setattro*/
    0, /*tp_as_buffer*/
    Py_TPFLAGS_DEFAULT | Py_TPFLAGS_BASETYPE | Py_TPFLAGS_HAVE_GC, /*tp_flags*/
    "Simple objects", /*tp_doc*/
    (traverseproc)Simple_traverse, /*tp_traverse*/
    0, /*tp_clear*/
    0, /*tp_richcompare*/
    0, /*tp_weaklistoffset*/
    0, /*tp_iter*/
    0, /*tp_iternext*/
    0, /*tp_methods*/
    0, /*tp_members*/
    0, /*tp_getset*/
    0, /*tp_base*/
    0, /*tp_dict*/
    0, /*tp_descr_get*/
    0, /*tp_descr_set*/
    0, /*tp_dictoffset*/
    (initproc)Simple_init, /*tp_init*/
    0, /*tp_alloc*/
    Simple_new, /*tp_new*/

    0 /*...the rest...*/
    };


    static PyObject *
    Simple_new(PyTypeObject *type, PyObject *args, PyObject *kwds)
    {
    SimpleObject *self = (SimpleObject *)type->tp_alloc(type, 0);
    return (PyObject*)self;
    }

    static int
    Simple_init(SimpleObject *self, PyObject *args, PyObject *kwds)
    {
    return 0;
    }

    /* cyclic gc */
    static int
    Simple_traverse (SimpleObject *self, visitproc visit, void *arg)
    {
    fprintf(stderr,"Simple_traverse...\n");
    return 0;
    }

    static int
    Simple_clear(SimpleObject *self)
    {
    fprintf(stderr,"Simple_clear...\n");
    return 0;
    }

    static void
    Simple_dealloc(SimpleObject *self)
    {
    fprintf(stderr,"Simple_dealloc %p\n", self);
    self->ob_type->tp_free((PyObject*)self); /* <= segfault here */
    return;
    }



    static PyMethodDef simple_methods[] = {
    {0} /* sentinel */
    };



    void
    initsimple(void)
    {
    if (PyType_Ready(&simple_SimpleType) < 0) return;
    Py_INCREF(&simple_SimpleType);

    PyObject* m = Py_InitModule3("simple", simple_methods, "Simple module.");
    PyModule_AddObject(m,"Simple",(PyObject*)&simple_SimpleType);
    }


    A script sufficient to provoke the fault is this:

    import simple
    simple.Simple()


    Can anyone explain what I'm doing wrong? Or perhaps suggest a better
    solution to my "real" problem, if I'm approaching the problem completely
    wrong :)

    Yours,
    /mailund
    Thomas Mailund, Jan 14, 2004
    #1
    1. Advertising

  2. Thomas Mailund

    Simon Burton Guest


    > Can anyone explain what I'm doing wrong? Or perhaps suggest a better
    > solution to my "real" problem, if I'm approaching the problem completely
    > wrong :)
    >
    > Yours,
    > /mailund


    In my extensions python seems to segfault in the GC most of all...
    Just yesterday I removed a malloc that was (why??) causing a segfault.

    Have you seen pyrex ? It's absolutely brilliant. Take 2 hours to try it
    out if you have the time.

    Simon.
    Simon Burton, Jan 15, 2004
    #2
    1. Advertising

  3. "Thomas Mailund" <> writes:

    > Hi group.
    >
    > I have a problem with some C extensions I am working with and
    > hope that some of you can help.


    [snippety]

    > static void
    > Simple_dealloc(SimpleObject *self)
    > {
    > fprintf(stderr,"Simple_dealloc %p\n", self);
    > self->ob_type->tp_free((PyObject*)self); /* <= segfault here */


    Well, you're calling tp_free from a tp_dealloc. That doesn't *sound*
    sensible to me.

    > Can anyone explain what I'm doing wrong? Or perhaps suggest a better
    > solution to my "real" problem, if I'm approaching the problem completely
    > wrong :)


    There are docs on this sort of thing.

    Cheers,
    mwh

    --
    "Sturgeon's Law (90% of everything is crap) applies to Usenet."
    "Nothing guarantees that the 10% isn't crap, too."
    -- Gene Spafford's Axiom #2 of Usenet, and a corollary
    Michael Hudson, Jan 15, 2004
    #3
  4. On Thu, 15 Jan 2004 11:15:51 +0000, Michael Hudson wrote:

    > "Thomas Mailund" <> writes:
    >
    >> Hi group.
    >>
    >> I have a problem with some C extensions I am working with and
    >> hope that some of you can help.

    >
    > [snippety]
    >
    >> static void
    >> Simple_dealloc(SimpleObject *self)
    >> {
    >> fprintf(stderr,"Simple_dealloc %p\n", self);
    >> self->ob_type->tp_free((PyObject*)self); /* <= segfault here */

    >
    > Well, you're calling tp_free from a tp_dealloc. That doesn't *sound*
    > sensible to me.


    I'm suprised to hear that; the documentation I was working from,
    <URL:http://www.python.org/doc/current/ext/node22.html>, does exactly
    that. Of course, this is from the section that uses reference counting,
    not cyclic gc, but later on, when the garbage collector is introduced, the
    deallocator still calls tp_free, it just calls clear first
    <URL:http://www.python.org/doc/current/ext/node24.html>.

    If I shouldn't free self in this way, how should I do it?

    >> Can anyone explain what I'm doing wrong? Or perhaps suggest a better
    >> solution to my "real" problem, if I'm approaching the problem completely
    >> wrong :)

    >
    > There are docs on this sort of thing.


    About the garbage collection? In that case, I thought I *was* following
    the documentation ;-)

    If about the general problem with accessing the whole and parts of a C
    structure from python, if you have any specific references in mind, I
    would be very grateful if you would post them. I know that I am not the
    first to have this problem, and that very likely there are general
    patterns for solving the problem, but my googling didn't find anything
    (which probably means I wasn't asking the right questions, but never the
    less...)

    Yours,
    /mailund
    Thomas Mailund, Jan 15, 2004
    #4
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Øyvind Isaksen
    Replies:
    1
    Views:
    948
    Øyvind Isaksen
    May 18, 2007
  2. Aaron Brady

    garbage collection / cyclic references

    Aaron Brady, Mar 21, 2009, in forum: Python
    Replies:
    9
    Views:
    306
    Aaron Brady
    Mar 22, 2009
  3. moerchendiser2k3

    Python leaks in cyclic garbage collection

    moerchendiser2k3, Feb 19, 2011, in forum: Python
    Replies:
    6
    Views:
    232
  4. Chris Angelico
    Replies:
    0
    Views:
    109
    Chris Angelico
    Apr 26, 2013
  5. Dave Angel
    Replies:
    0
    Views:
    99
    Dave Angel
    Apr 26, 2013
Loading...

Share This Page