problem with Py_BuildValue

[Christian Meesters]

Hi,

currently I have a problem understanding Py_BuildValue. I have this code:

static PyObject *function(PyObject *self, PyObject *args) {
PyObject * python_return_value = NULL;
PyObject * dummy = NULL;
double * internal_list;
<snip and forget the rest>

/* converting to python representation */
for (i=0; i < limit; i++) {
dummy = Py_BuildValue("d", internal_list);
if (!dummy) return NULL;
PyList_Append(python_return_value, dummy);
Py_DECREF(dummy); dummy = NULL;
}
return python_return_value
}

This doesn't work. What I see, when invoking the function "function()" in
Python is a list of refcounts, like: [<refcnt 0 at 0x94a29d4>, <refcnt 0 at
0x94a29e4>, ...]. However, if I change the Py_BuildValue-line to be
dummy = Py_BuildValue("i", (int)internal_list);
I do get the 'right' integer return values. Point is that I really would
like to work with Python-floats afterwards.

Any idea where a pitfall might be here?

TIA
Christian

PS Oh, and I tried casting to float and explicitly to double, too. Same
result as without the casts.

 

[Cédric Lucantis]

Hi,

Hi,

currently I have a problem understanding Py_BuildValue. I have this code:

static PyObject *function(PyObject *self, PyObject *args) {
PyObject * python_return_value = NULL;
PyObject * dummy = NULL;
double * internal_list;
<snip and forget the rest>

/* converting to python representation */
for (i=0; i < limit; i++) {
dummy = Py_BuildValue("d", internal_list);
if (!dummy) return NULL;
PyList_Append(python_return_value, dummy);
Py_DECREF(dummy); dummy = NULL;
}
return python_return_value
}

This doesn't work. What I see, when invoking the function "function()" in
Python is a list of refcounts, like: [<refcnt 0 at 0x94a29d4>, <refcnt 0 at
0x94a29e4>, ...]. However, if I change the Py_BuildValue-line to be
dummy = Py_BuildValue("i", (int)internal_list);
I do get the 'right' integer return values. Point is that I really would
like to work with Python-floats afterwards.

Any idea where a pitfall might be here?

I see nothing wrong with your code so I'd say it is somewhere else (did you
snip any code between the end of the loop and the return?). I've never seen
those 'refcnt' objects but a refcount of 0 sounds like you unrefed your
objects one extra time by mistake. This would produce a segfault on unix, but
maybe not on all platforms ? You should check the return value of
PyList_Append() and if it doesn't help trace the content of your list after
each iteration to see when the bad things happen (you can check the reference
count of an object with obj->ob_refcnt).

Finally note that in your case it would be much simpler and more efficient to
use the float constructor directly:

dummy = PyFloat_FromDouble(internal_list())

PS: always use Py_CLEAR(dummy) instead of Py_DECREF(dummy); dummy=NULL;
(though it doesn't really matter in this simple case - see
http://docs.python.org/api/countingRefs.html)

 

[Christian Meesters]

Thank you. At least I can exclude another few error sources, now.

I see nothing wrong with your code so I'd say it is somewhere else (did
you snip any code between the end of the loop and the return?).

No. (Apart from freeing allocated memory.)

I've never
seen those 'refcnt' objects but a refcount of 0 sounds like you unrefed
your objects one extra time by mistake. This would produce a segfault on
unix, but maybe not on all platforms ?

Well, I am working on Linux. Python 2.5.1, gcc 4.1.3 . And I do not see
segfaults until I start working in Python with the return value of that
function, of course.

You should check the return value
of PyList_Append()

It is always 0, regardless of what I do.

and if it doesn't help trace the content of your list
after each iteration to see when the bad things happen (you can check the
reference count of an object with obj->ob_refcnt).

Seems ok. What I did to check this was placing this after building the list:

for (i=0; i < limit; i++) {
dummy = PyList_GetItem(python_return_value, i);
printf("%f\n", PyFloat_AsDouble(dummy));
Py_CLEAR(dummy);
}

Which gives reasonable numbers.

Finally note that in your case it would be much simpler and more efficient
to use the float constructor directly:

dummy = PyFloat_FromDouble(internal_list())

I tried that (actually PyFloat_FromDouble(internal_list) ): Same thing,

PS: always use Py_CLEAR(dummy) instead of Py_DECREF(dummy); dummy=NULL;
(though it doesn't really matter in this simple case - see
http://docs.python.org/api/countingRefs.html)

Good idea! Since I require 2.4 for users anyway, there is no harm in
reducing the code.

Christian

 

[Cédric Lucantis]

Thank you. At least I can exclude another few error sources, now.

No. (Apart from freeing allocated memory.)

I'm pretty sure we'll find something interesting here

Seems ok. What I did to check this was placing this after building the
list:

for (i=0; i < limit; i++) {
dummy = PyList_GetItem(python_return_value, i);
printf("%f\n", PyFloat_AsDouble(dummy));
Py_CLEAR(dummy);
}

PyList_GetItem returns a borrowed reference so you shoud _not_ unref it (this
explains the refcnt -1 I think)

Here's the code producing your message (from Objects/object.c in the python
sources) :

/* Implementation of PyObject_Print with recursion checking */
static int
internal_print(PyObject *op, FILE *fp, int flags, int nesting)
{
<snip>
if (op->ob_refcnt <= 0)
/* XXX(twouters) cast refcount to long until %zd is
universally available */
fprintf(fp, "<refcnt %ld at %p>", (long)op->ob_refcnt, op);
}
<snip>
}

I don't really understand its purpose, but it confirms that it's a ref count
problem. Maybe the full source of your function could help ?

 

[Christian Meesters]

Thank you so much - I was such an idiot (see below).

I'm pretty sure we'll find something interesting here

Still not. I was about to prove it and already uploaded the file, when I
saw, what was really going wrong ...

PyList_GetItem returns a borrowed reference so you shoud _not_ unref it
(this explains the refcnt -1 I think)

This is THE crucial point. If I just delete the Py_CLEAR-line, everything is
working smoothly and calling PyFloat_FromDouble-is working too.

Again: Thanks a lot. 'Reference counting' won't become one of my
hobbies ;-).

Best,
Christian