Storing a C pointer in a Python class instance

[lallous]

Hello

From my C extension module I want to store a C pointer in a given PyObject.

The only way I figure how to do it is to use Py_BuildValues and store the
poiner casted to Py_ssize_t, thus:

Py_BuildValues("n", (Py_ssize_t)my_ptr)

Can it be done differently?

Regards,
Elias

 

[Falcolas]

Hello

From my C extension module I want to store a C pointer in a given PyObject.

The only way I figure how to do it is to use Py_BuildValues and store the
poiner casted to Py_ssize_t, thus:

Py_BuildValues("n", (Py_ssize_t)my_ptr)

Can it be done differently?

Regards,
Elias

You can use a "PyCObject_FromVoidPtr"

http://docs.python.org/c-api/cobject.html

PyArg_ParseTuple(args, "O", &pyVoidPointer);
castPointer = (type *) PyCObject_AsVoidPtr(pyVoidPointer);
return PyCObject_FromVoidPtr((void *) castPointer, NULL);

 

[sturlamolden]

Hello

From my C extension module I want to store a C pointer in a given PyObject.

The only way I figure how to do it is to use Py_BuildValues and store the
poiner casted to Py_ssize_t,

Formally, you should cast the pointer to Py_intptr_t, as it has the
same size as void*. Py_ssize_t has the same size as size_t, but the C
standard does not mandate that sizeof(void*) == sizeof(size_t). In
fact there are segment and offset architectures where this is not
true. Casting a pointer to Py_ssize_t accidentally works if you have a
flat address space.

Can it be done differently?

You can use PyCObject, or write your own extension type that wraps the
pointer (very easy to to with Cython or Pyrex). The advantage of using
an extension type is you have a guarantee from Python on the
deallocator method being called (cdef __dealloc__ in Cython). If the
pointer references a resource that needs to be closed, this is safer
than using a __del__ method in a Python class.

 

[Carl Banks]

You can use PyCObject, or write your own extension type that wraps the
pointer (very easy to to with Cython or Pyrex). The advantage of using
an extension type is you have a guarantee from Python on the
deallocator method being called (cdef __dealloc__ in Cython).

CObjects can be passed a C function as a deallocator; this should work
as reliably as a custom class deallocator.


Carl Banks

 

[sturlamolden]

CObjects can be passed a C function as a deallocator; this should work
as reliably as a custom class deallocator.

Carl Banks

Except that __del__ prevents cyclic GC.

 

[lallous]

Thanks everyone.

Finally, I used Falcolas suggestion and took into consideration
sturlamolden's comments.

Regards,
Elias

 

[lallous]

Hello

After using the PyCObject, I cannot pickle the class anymore.
Any simple solution to this problem? (or resorting to __reduce__ is the only
solution?)

Thanks,
Elias

 

[Carl Banks]

Hello

After using the PyCObject, I cannot pickle the class anymore.
Any simple solution to this problem? (or resorting to __reduce__ is the only
solution?)

You can't pickle a CObject, you'd have to create a custom type (one
that implements one of the pickling methods) for that. Or arrange for
whatever object contains the CObject to pack and unpack it manually.

Out of curiosity, what kind of data you storing in this CObject?
Maybe we can help you choose a better way to handle it at the C level.


Carl Banks

 

[Carl Banks]

Except that __del__ prevents cyclic GC.

You are mistaken on two counts.

First of all, a CObject is not a container. It can't prevent cyclic
GC because it's never a part of a cycle.

Second, CObjects do not have a __del__ method. They call the supplied
constructor from the type's tp_dealloc slot. Use of the tp_dealloc
slot does not, by itself, prevent cyclic GC.

Bottom line is, the CObject's deallocator is as reliable as a custom
type's tp_dealloc.


Carl Banks

 

[sturlamolden]

Second, CObjects do not have a __del__ method.  They call the supplied
constructor from the type's tp_dealloc slot.  Use of the tp_dealloc
slot does not, by itself, prevent cyclic GC.

Bottom line is, the CObject's deallocator is as reliable as a custom
type's tp_dealloc.

You are right. I did not look at the PyCObject_* API close enough.

I thought of wrapping the CObject with a Python class, and calling the
destructor from __del__. That would be less reliable.

S.M.

 

[lallous]

You can't pickle a CObject, you'd have to create a custom type (one
that implements one of the pickling methods) for that. Or arrange for
whatever object contains the CObject to pack and unpack it manually.

Out of curiosity, what kind of data you storing in this CObject?
Maybe we can help you choose a better way to handle it at the C level.

I am wrapping a C++ pointer with the python object. That way I can tell with
which C++ object a given python class instance is associated.

The thing is when developing, I need to pickle but I don't need the C++
pointer, so I solved the problem with conditional compilation:
- testing: pickle allowed and "This" is stored in the py object
- production code: no need to pickle and "this" and "pyobject" are bound

Thanks,
Elias