reference counting and PyTuple_SetItem

[Anne Wilson]

Hi,

I'm embedding some Python code in my C code. I've spent way too much
time trying to ensure my code works properly. I'm finding the docs and
discussions very ambiguous. The term "steal" is extremely ambiguous.

I want to thank Edward C. Jones for his reference counting document
(http://members.tripod.com/~edcjones/refcount.html). I found Edward's
additional explanations very helpful, especially about "stealing a
reference" and using PyTuple_SetItem(). It was especially enlightening
to learn that PyTuple_SetItem will not incref the tuple's arguments, and
will decref the tuple's arguments when the tuple is decrefed.

However, that method of reference count management seems to preclude two
things I'm trying to do. In one case, I'm reusing an item in more than
one tuple, hence, more than one PyTuple_SetItem() call. In the second
case, in a loop I'm putting different items into the 4th argument of the
same tuple.

Here's some pseudo code to illustrate what I'm trying to do (leaving out
error checking, etc.):


pyStatsDir = PyString_FromString(statsDir);
...

/* ----------------------------------------
* Set up arg list for 5 minute window size
* ----------------------------------------
*/
pyFiveMinArgs = PyTuple_New(PY_CALL_ARG_CNT);
pyFiveMinWindowSize = PyInt_FromLong(300L);
pyFiveMinBinSize = PyInt_FromLong(1L);
pyFiveMinOutputFileStr = PyString_FromString("5min");
/* ---> Note 1 <--- */
PyTuple_SetItem(pyFiveMinArgs, 0, pyFiveMinWindowSize);
PyTuple_SetItem(pyFiveMinArgs, 1, pyFiveMinBinSize);
PyTuple_SetItem(pyFiveMinArgs, 2, pyFiveMinOutputFileStr);
/* ---> Note 2 <--- */
PyTuple_SetItem(pyFiveMinArgs, 3, pyStatsDir);
/* Field 4, rHost, we'll get inside the loop */

/* ----------------------------------------
* Set up arg list for one hour window size
* ----------------------------------------
*/
pyOneHourArgs = PyTuple_New(PY_CALL_ARG_CNT);

pyOneHourWindowSize = PyInt_FromLong(3600L);
pyOneHourBinSize = PyInt_FromLong(15L);
pyOneHourOutputFileStr = PyString_FromString("1hr");

PyTuple_SetItem(pyOneHourArgs, 0, pyOneHourWindowSize);
PyTuple_SetItem(pyOneHourArgs, 1, pyOneHourBinSize);
PyTuple_SetItem(pyOneHourArgs, 2, pyOneHourOutputFileStr);
/* ---> Note 2 <--- */
PyTuple_SetItem(pyOneHourArgs, 3, pyStatsDir);
/* Field 4, rHost, we'll get inside the loop */

...

while (<some condition>)
{
pyRHost = PyString_FromString(rHost);

Py_INCREF(pyRHost); /* ---> Note 3 <--- */
PyTuple_SetItem(pyFiveMinArgs, 4, pyRHost);
PyTuple_SetItem(pyOneHourArgs, 4, pyRHost);

pyValue = PyObject_CallObject(pyFunc, pyFiveMinArgs);
Py_DECREF(pyValue);

pyValue = PyObject_CallObject(pyFunc, pyOneDayArgs);
Py_DECREF(pyValue);

py_DECREF(pyRHost); /* ---> Note 3 <--- */
}
Py_DECREF(pyStatsDir);
Py_DECREF(pyFeedtype);
Py_DECREF(pyFiveMinArgs);
Py_DECREF(pyOneHourArgs);


---> Note 1: This looks straightforward - don't increment the ref count
for pyFiveMinWindowSize, pyFiveMinBinSize, pyFiveMinOutputFileStr, as
when pyFiveMinArgs is decref'ed it will handle the ref counting for
those objects.

---> Note 2: Here I am reusing pyStatsDir in multiple tuples. Now, when
the loop ends, each tuple will decref pyStatsDir?? Since the pyStatsDir
isn't increfed, seems like that's too many decrefs!

---> Note 3: Here, pyRHost varies across each iteration of the while
loop. I *want* the pyRHost object space to be freed each time through
the loop. So, I thought I should keep a ref myself and then decrement
it at the bottom of the loop, except for possibly the last iteration.
Is this right??? (Also, not sure if I can determine if I'm in the last
iteration of the loop or not.)

I'm still getting spurious crashes. Maybe I just can't use
PyTuple_SetItem? If so, then what should I be using?

Any help or comments would be greatly appreciated.

Thanks!

Anne