Storing a C pointer in a Python class instance

Discussion in 'Python' started by lallous, Sep 29, 2009.

  1. lallous

    lallous Guest

    Hello

    From my C extension module I want to store a C pointer in a given PyObject.

    The only way I figure how to do it is to use Py_BuildValues and store the
    poiner casted to Py_ssize_t, thus:

    Py_BuildValues("n", (Py_ssize_t)my_ptr)

    Can it be done differently?

    Regards,
    Elias
    lallous, Sep 29, 2009
    #1
    1. Advertising

  2. lallous

    Falcolas Guest

    On Sep 29, 2:27 am, "lallous" <> wrote:
    > Hello
    >
    > From my C extension module I want to store a C pointer in a given PyObject.
    >
    > The only way I figure how to do it is to use Py_BuildValues and store the
    > poiner casted to Py_ssize_t, thus:
    >
    > Py_BuildValues("n", (Py_ssize_t)my_ptr)
    >
    > Can it be done differently?
    >
    > Regards,
    > Elias


    You can use a "PyCObject_FromVoidPtr"

    http://docs.python.org/c-api/cobject.html

    PyArg_ParseTuple(args, "O", &pyVoidPointer);
    castPointer = (type *) PyCObject_AsVoidPtr(pyVoidPointer);
    return PyCObject_FromVoidPtr((void *) castPointer, NULL);
    Falcolas, Sep 29, 2009
    #2
    1. Advertising

  3. lallous

    sturlamolden Guest

    On 29 Sep, 10:27, "lallous" <> wrote:
    > Hello
    >
    > From my C extension module I want to store a C pointer in a given PyObject.
    >
    > The only way I figure how to do it is to use Py_BuildValues and store the
    > poiner casted to Py_ssize_t,


    Formally, you should cast the pointer to Py_intptr_t, as it has the
    same size as void*. Py_ssize_t has the same size as size_t, but the C
    standard does not mandate that sizeof(void*) == sizeof(size_t). In
    fact there are segment and offset architectures where this is not
    true. Casting a pointer to Py_ssize_t accidentally works if you have a
    flat address space.


    > Can it be done differently?


    You can use PyCObject, or write your own extension type that wraps the
    pointer (very easy to to with Cython or Pyrex). The advantage of using
    an extension type is you have a guarantee from Python on the
    deallocator method being called (cdef __dealloc__ in Cython). If the
    pointer references a resource that needs to be closed, this is safer
    than using a __del__ method in a Python class.
    sturlamolden, Sep 29, 2009
    #3
  4. lallous

    Carl Banks Guest

    On Sep 29, 9:42 am, sturlamolden <> wrote:
    > You can use PyCObject, or write your own extension type that wraps the
    > pointer (very easy to to with Cython or Pyrex). The advantage of using
    > an extension type is you have a guarantee from Python on the
    > deallocator method being called (cdef __dealloc__ in Cython).


    CObjects can be passed a C function as a deallocator; this should work
    as reliably as a custom class deallocator.


    Carl Banks
    Carl Banks, Sep 29, 2009
    #4
  5. lallous

    sturlamolden Guest

    On 29 Sep, 19:11, Carl Banks <> wrote:

    > CObjects can be passed a C function as a deallocator; this should work
    > as reliably as a custom class deallocator.
    >
    > Carl Banks


    Except that __del__ prevents cyclic GC.
    sturlamolden, Sep 29, 2009
    #5
  6. lallous

    lallous Guest

    Thanks everyone.

    Finally, I used Falcolas suggestion and took into consideration
    sturlamolden's comments.

    Regards,
    Elias
    "lallous" <> wrote in message news:h9sgcn$ivo$...
    > Hello
    >
    > From my C extension module I want to store a C pointer in a given
    > PyObject.
    >
    > The only way I figure how to do it is to use Py_BuildValues and store the
    > poiner casted to Py_ssize_t, thus:
    >
    > Py_BuildValues("n", (Py_ssize_t)my_ptr)
    >
    > Can it be done differently?
    >
    > Regards,
    > Elias
    lallous, Sep 30, 2009
    #6
  7. lallous

    lallous Guest

    Hello

    After using the PyCObject, I cannot pickle the class anymore.
    Any simple solution to this problem? (or resorting to __reduce__ is the only
    solution?)

    Thanks,
    Elias

    "Falcolas" <> wrote in message
    news:...
    > On Sep 29, 2:27 am, "lallous" <> wrote:
    >> Hello
    >>
    >> From my C extension module I want to store a C pointer in a given
    >> PyObject.
    >>
    >> The only way I figure how to do it is to use Py_BuildValues and store the
    >> poiner casted to Py_ssize_t, thus:
    >>
    >> Py_BuildValues("n", (Py_ssize_t)my_ptr)
    >>
    >> Can it be done differently?
    >>
    >> Regards,
    >> Elias

    >
    > You can use a "PyCObject_FromVoidPtr"
    >
    > http://docs.python.org/c-api/cobject.html
    >
    > PyArg_ParseTuple(args, "O", &pyVoidPointer);
    > castPointer = (type *) PyCObject_AsVoidPtr(pyVoidPointer);
    > return PyCObject_FromVoidPtr((void *) castPointer, NULL);
    lallous, Sep 30, 2009
    #7
  8. lallous

    Carl Banks Guest

    On Sep 30, 5:24 am, "lallous" <> wrote:
    > Hello
    >
    > After using the PyCObject, I cannot pickle the class anymore.
    > Any simple solution to this problem? (or resorting to __reduce__ is the only
    > solution?)



    You can't pickle a CObject, you'd have to create a custom type (one
    that implements one of the pickling methods) for that. Or arrange for
    whatever object contains the CObject to pack and unpack it manually.

    Out of curiosity, what kind of data you storing in this CObject?
    Maybe we can help you choose a better way to handle it at the C level.


    Carl Banks
    Carl Banks, Sep 30, 2009
    #8
  9. lallous

    Carl Banks Guest

    On Sep 29, 11:16 am, sturlamolden <> wrote:
    > On 29 Sep, 19:11, Carl Banks <> wrote:
    >
    > > CObjects can be passed a C function as a deallocator; this should work
    > > as reliably as a custom class deallocator.

    >
    > Except that __del__ prevents cyclic GC.


    You are mistaken on two counts.

    First of all, a CObject is not a container. It can't prevent cyclic
    GC because it's never a part of a cycle.

    Second, CObjects do not have a __del__ method. They call the supplied
    constructor from the type's tp_dealloc slot. Use of the tp_dealloc
    slot does not, by itself, prevent cyclic GC.

    Bottom line is, the CObject's deallocator is as reliable as a custom
    type's tp_dealloc.


    Carl Banks
    Carl Banks, Sep 30, 2009
    #9
  10. lallous

    sturlamolden Guest

    On 30 Sep, 19:03, Carl Banks <> wrote:

    > Second, CObjects do not have a __del__ method.  They call the supplied
    > constructor from the type's tp_dealloc slot.  Use of the tp_dealloc
    > slot does not, by itself, prevent cyclic GC.
    >
    > Bottom line is, the CObject's deallocator is as reliable as a custom
    > type's tp_dealloc.


    You are right. I did not look at the PyCObject_* API close enough.

    I thought of wrapping the CObject with a Python class, and calling the
    destructor from __del__. That would be less reliable.

    S.M.
    sturlamolden, Sep 30, 2009
    #10
  11. lallous

    lallous Guest

    "Carl Banks" <> wrote in message
    news:...
    > On Sep 30, 5:24 am, "lallous" <> wrote:
    >> Hello
    >>
    >> After using the PyCObject, I cannot pickle the class anymore.
    >> Any simple solution to this problem? (or resorting to __reduce__ is the
    >> only
    >> solution?)

    >
    >
    > You can't pickle a CObject, you'd have to create a custom type (one
    > that implements one of the pickling methods) for that. Or arrange for
    > whatever object contains the CObject to pack and unpack it manually.
    >
    > Out of curiosity, what kind of data you storing in this CObject?
    > Maybe we can help you choose a better way to handle it at the C level.
    >
    >


    I am wrapping a C++ pointer with the python object. That way I can tell with
    which C++ object a given python class instance is associated.

    The thing is when developing, I need to pickle but I don't need the C++
    pointer, so I solved the problem with conditional compilation:
    - testing: pickle allowed and "This" is stored in the py object
    - production code: no need to pickle and "this" and "pyobject" are bound

    Thanks,
    Elias
    lallous, Oct 6, 2009
    #11
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Whitney Kew
    Replies:
    10
    Views:
    579
    Whitney Kew
    Dec 7, 2004
  2. Sridhar R
    Replies:
    14
    Views:
    1,396
    =?iso-8859-1?Q?Fran=E7ois?= Pinard
    Feb 10, 2004
  3. toton
    Replies:
    11
    Views:
    704
    toton
    Oct 13, 2006
  4. Jonathan Wood
    Replies:
    1
    Views:
    505
    Jonathan Wood
    Jun 2, 2008
  5. Martin P. Hellwig
    Replies:
    1
    Views:
    373
    Martin P. Hellwig
    Mar 26, 2010
Loading...

Share This Page