how to pass C++ object to another C++ function via Python function

[grbgooglefan]

I am trying to pass a C++ object to Python function. This Python
function then calls another C++ function which then uses this C++
object to call methods of that object's class.

I tried something like this, but it did not work, gave core dump.

class myclass {
public:
myclass(){};
~myclass(){};
void printmyname() { printf("I am myclass, num=%d\n",num); };
};

main(){
myclass myobj

char funcbody[]=\
"def pyfunction(t167,classobj):\n\
onemorefunc(t167,\"NONAMe\")\n\
return 10\n\
\n\n";

// compile this Python function using PyRun_String & get its pointer
by PyObject_GetAttrString.
// Later call it as below:
myclass myobj(23);
PyObject *pTuple = 0;
pTuple = PyTuple_New(2);
PyTuple_SetItem(pTuple,0,PyString_FromString("NAME")); // for t167
parameter
PyObject *inputarg = Py_BuildValue("OO&",pTuple,myobj); // for
classobj parameter

result = PyObject_CallObject(pPyEvalFunction,inputarg);
}

How can I pass this class object to Python function?
Is it possible to set it in tuple using PyTuple_SetItem, because I may
have varying number of arguments for my Python functions & that's why
I can't use Py_BuildValue.

 

[Diez B. Roggisch]

I am trying to pass a C++ object to Python function. This Python
function then calls another C++ function which then uses this C++
object to call methods of that object's class.

You might consider using a C++-wrapper like SIP, Swig or Boost:ython to do
this.

If you don't like that, all I can think of would be to return the address of
the object as integer, and pass that around. Then in the appropriate
C++-call, cast that integer to the object. butt-ugly and -10 style-points
though.

Diez

 

[Gabriel Genellina]

I am trying to pass a C++ object to Python function. This Python
function then calls another C++ function which then uses this C++
object to call methods of that object's class.

I tried something like this, but it did not work, gave core dump.

You can't pass any arbitrary C object to a Python function.
In this case you can use a PyCObject, a Python box around a void* pointer.
See http://docs.python.org/api/cObjects.html

// compile this Python function using PyRun_String & get its pointer
by PyObject_GetAttrString.
// Later call it as below:
myclass myobj(23);
PyObject *pTuple = 0;
pTuple = PyTuple_New(2);
PyTuple_SetItem(pTuple,0,PyString_FromString("NAME")); // for t167
parameter
PyObject *inputarg = Py_BuildValue("OO&",pTuple,myobj); // for
classobj parameter

result = PyObject_CallObject(pPyEvalFunction,inputarg);
}

How can I pass this class object to Python function?
Is it possible to set it in tuple using PyTuple_SetItem, because I may
have varying number of arguments for my Python functions & that's why
I can't use Py_BuildValue.

You have to check every call for errors, and pay attention to the reference counts!
The argument passing is wrong. You never set the second tuple element. An easier way is using PyObject_CallFunctionObjArgs(function, arg1, arg2, NULL)

 

[Diez B. Roggisch]

You can't pass any arbitrary C object to a Python function.
In this case you can use a PyCObject, a Python box around a void* pointer.
See http://docs.python.org/api/cObjects.html

Neat! Didn't know about that one.

Diez

 

[grbgooglefan]

You can't pass any arbitrary C object to a Python function.
In this case you can use a PyCObject, a Python box around a void* pointer.
Seehttp://docs.python.org/api/cObjects.html

Yup, I looked at http://www.python.org/doc/ext/using-cobjects.html
also. I could not find in this example where is CObject used or
converted back from Python to C.
Is there any tutorial which I can use for this?

 

[Gabriel Genellina]

Yup, I looked at http://www.python.org/doc/ext/using-cobjects.html
also. I could not find in this example where is CObject used or
converted back from Python to C.
Is there any tutorial which I can use for this?

If you have a C function that receives a PyCObject, just include the
relevant headers (cobject.h) and you can retrieve the original pointer
using PyCObject_AsVoidPtr:

void foo(PyObject *pyobj)
{
TOriginalType *porig;
porig = (TOriginalType *)PyCObject_AsVoidPtr(pyobj);
// do something with porig
}

 

[Gabriel Genellina]

Neat! Didn't know about that one.

I found it just by chance. The title alone in the API reference isn't very
descriptive...

 

[grbgooglefan]

If you have a C function that receives a PyCObject, just include the  
relevant headers (cobject.h) and you can retrieve the original pointer  
using PyCObject_AsVoidPtr:

void foo(PyObject *pyobj)
{
   TOriginalType *porig;
   porig = (TOriginalType *)PyCObject_AsVoidPtr(pyobj);
   // do something with porig
}

This works. But only once, means if I try to use this object second
time in Python function it causes crash.

What I am doing in my program is that I am putting STL map in a
structure & passing that structure as object to a Python function
alongwith other agurments of that Python function. This briefly as
below:

// In pyinterface.h file:---
typedef hash_map<char*,int> Elements;
typedef hash_map<char*,Elements*,StringHash,eqstr>
PerExchGroupsElementsTable;
typedef struct capsule {
PerExchGroupsElementsTable* refgrps;
} *pcapsule;

// In pyinterface.cpp file:---
numvars = // value set depending on number of variables of that python
function
PyObject *pTuple = PyTuple_New(numvars);

// Set variables as below:
for(nCtr = 0; nCtr < numvars; nCtr++){
slot = prul->pVarLst[nCtr].slot;
ndtyp = ordvars[slot].dtype;
switch(ndtyp){
case(INT_T):

PyTuple_SetItem(pTuple,nCtr,PyInt_FromLong(ordvars[slot].nvalue));
break;
case(FLOAT_T):

PyTuple_SetItem(pTuple,nCtr,PyFloat_FromDouble(ordvars[slot].fvalue));
break;
case(STRING_T):

PyTuple_SetItem(pTuple,nCtr,PyString_FromString(ordvars[slot].cvalue));
break;
default:
printf("\nUnknown data type [%d] for %s\n",ndtyp,prul-

pVarLst[nCtr].var);

bUnknownDataType = true;
break;
}
if(bUnknownDataType){
ret = -1;
break;
}
}

// Then set the C++ object as below:
if(ret == 0){
capsule grpob;
if(pGroups){
grpob.refgrps = pGroups; // pGroups is pointer to
PerExchGroupsElementsTable map & is global.
int ret = PyTuple_SetItem(pTuple,
(numvars-1),PyCObject_FromVoidPtr((void *)&grpob, NULL));
}
PyObject *pResult = PyObject_CallObject(pfunc,pTuple);
if(PyErr_Occurred()){
printf("error occured in PyObject_CallObject for %s\n",prul-

pyobj.szPyRouteName);

PyErr_Print();
} else {
printf("Python function passed, use its result for other
purposes as designed\n");
}
Py_XDECREF(pResult);
Py_XDECREF(pTuple);
}

//---------- My Pythong module & functions in it -------------------
//PyObject* pfunc is a Python function which I compile dynamically &
add to my Python module as below:
// One of such dynamically compiled function is as below:
char pyfunction[]=\
"def TSE581(t22,t52,t1012,ob):
if(co1(ob,t22,\"TSE_FUTURE_GRP\") and
like1(ob,t52,\"TSE_SECID_LST2\")):\n\
print \"result is pass\"\n\
return 1\n\
else:\n\
print \"result is fail\"\n\
return 0\n";

// function parameter "ob" in this function definition is the one
which Im passing as CObject.

PyObject *result = NULL;
result =
PyRun_String(pyfunction,Py_file_input,_pPyDictionary,_pPyDictionary);
if(PyErr_Occurred() || result == NULL){
printf("Failed to compile function [%s]\n",func);
PyErr_Print();
return;
}
Py_XDECREF(result);
result = NULL;
PyObject *ptr = PyObject_GetAttrString(_pPyModule,fname);
if(PyErr_Occurred() || *ptr == NULL){
printf("PyObject_GetAttrString failed:%s",fname);
return;
}
if(!PyCallable_Check(*ptr)){
printf("%s not a callable Python code\n",fname);
Py_XDECREF(*ptr);
*ptr = NULL;
return;
}

// I've created dynamically loadble Python module & multiple functions
similar to above gets added dynamically to this module.
// Module has functions "co1" & "like1" in module's .cpp file. These
functions then use CObject - the struct capsule & map pointer in it.
static PyObject* merorderrouter_co1(PyObject* self, PyObject* args)
{
printf("Contains function\n");
int ret=0;
char *arg1=NULL, *arg2=NULL;
PyObject* voidcap=NULL;
if(!PyArg_ParseTuple(args, "Oss", &voidcap,&arg1,&arg2)){
printf("failed to get args\n");
if(PyErr_Occurred())
PyErr_Print();
return(PyInt_FromLong(ret));
}
printf("key=%s, grpname=[%s]\n",arg1,arg2);
if(voidcap && PyCObject_Check(voidcap))
printf("valid Py-C-Object\n");
else
printf("NOT a valid Py-C-Object\n");
pcapsule pobj = (pcapsule)PyCObject_AsVoidPtr(voidcap);
if(pobj){
PerExchGroupsElementsTable grpmap = *pobj->refgrps;
if(grpmap.count(arg2)){
PerExchGroupsElementsTable::iterator grpmi =
grpmap.find(arg2);
Elements grpelems = *(Elements*)grpmi->second;
Elements::iterator ei = grpelems.find(arg1);
if(ei != grpelems.end()){
printf("has elm.\n");
ret=1;
} else {
printf("no elm.\n");
ret=0;
}
}
} else {
printf("pcapsule object is null from PyCObject_AsVoidPtr
\n");
if(PyErr_Occurred())
PyErr_Print();
}
printf("------- co returning.....\n");
return PyInt_FromLong(ret);
}
//=======================================================

What happens is that when Python function TSE581 gets called from my C
Program via PyObject_CallObject (as shown at top of this post), co1
function works fine & can access the map pointer properly.
But when next function like1 gets called, crash happens.

My queries are & things on which I need help are:
1) Is there anything wrong I am doing when I am passing the C-Object
from my C++ code->Python -> C++ code again?
2) Am I missing to increase or decrease the reference count somewhere.
3) I dont want map pointer to be ever freed because it is Process
level data structure & requried at every execution of these Python
functions. How do I avoid its cleanup when it gets passed to Python &
Python cleans up those objects.

If want to see my Python module & the CPP code using it, I can send
all my source code to you.
Please help.

 

[Gabriel Genellina]

If you have a C function that receives a PyCObject, just include the  
relevant headers (cobject.h) and you can retrieve the original pointer  
using PyCObject_AsVoidPtr:

This works. But only once, means if I try to use this object second
time in Python function it causes crash.

What I am doing in my program is that I am putting STL map in a
structure & passing that structure as object to a Python function
alongwith other agurments of that Python function. This briefly as
below:

// In pyinterface.h file:---
typedef hash_map<char*,int> Elements;
typedef hash_map<char*,Elements*,StringHash,eqstr>
PerExchGroupsElementsTable;
typedef struct capsule {
PerExchGroupsElementsTable* refgrps;
} *pcapsule;

// In pyinterface.cpp file:---
numvars = // value set depending on number of variables of that python
function
PyObject *pTuple = PyTuple_New(numvars);

// Set variables as below:
for(nCtr = 0; nCtr < numvars; nCtr++){
slot = prul->pVarLst[nCtr].slot;
ndtyp = ordvars[slot].dtype;
switch(ndtyp){
case(INT_T):
PyTuple_SetItem(pTuple,nCtr,PyInt_FromLong(ordvars[slot].nvalue));
break;
case(FLOAT_T):
PyTuple_SetItem(pTuple,nCtr,PyFloat_FromDouble(ordvars[slot].fvalue));
break;
case(STRING_T):
PyTuple_SetItem(pTuple,nCtr,PyString_FromString(ordvars[slot].cvalue));
break;
default:
printf("\nUnknown data type [%d] for %s\n",ndtyp,prul-

pVarLst[nCtr].var);

bUnknownDataType = true;
break;
}
if(bUnknownDataType){
ret = -1;
break;
}
}

// Then set the C++ object as below:
if(ret == 0){
capsule grpob;
if(pGroups){
grpob.refgrps = pGroups; // pGroups is pointer to
PerExchGroupsElementsTable map & is global.
int ret = PyTuple_SetItem(pTuple,
(numvars-1),PyCObject_FromVoidPtr((void *)&grpob, NULL));
}

This look suspicious - what if !pGroups? You can't leave a tuple item
uninitialized - if you create a tuple with PyTuple_New(3), then you have
to fill the 3 items.
And beware of that local variable grpob - you're using a pointer to it,
and it won't be valid when execution goes out of this block. You must
ensure that nobody stores a reference to it.
And you should decref the tuple even if this block isn't executed.

PyObject *ptr = PyObject_GetAttrString(_pPyModule,fname);
if(PyErr_Occurred() || *ptr == NULL){
printf("PyObject_GetAttrString failed:%s",fname);
return;
}
if(!PyCallable_Check(*ptr)){
printf("%s not a callable Python code\n",fname);
Py_XDECREF(*ptr);
*ptr = NULL;
return;
}

All those *ptr should be ptr (the compiler should issue a lot of warnings,
I presume?)

What happens is that when Python function TSE581 gets called from my C
Program via PyObject_CallObject (as shown at top of this post), co1
function works fine & can access the map pointer properly.
But when next function like1 gets called, crash happens.

My queries are & things on which I need help are:
1) Is there anything wrong I am doing when I am passing the C-Object
from my C++ code->Python -> C++ code again?

I see nothing obviously wrong, except the above notes. If you get warnings
from the C++ compiler, try to fix all of them.

2) Am I missing to increase or decrease the reference count somewhere.

In case of error you exit early but without releasing some existing
objects. (Sometimes a goto statement *is* the right thing to do)

3) I dont want map pointer to be ever freed because it is Process
level data structure & requried at every execution of these Python
functions. How do I avoid its cleanup when it gets passed to Python &
Python cleans up those objects.

Python will do nothing with the pointer inside a PyCObject - unless you
want to, and pass a cleanup function as the second argument to the
PyCObject constructor.

 

[grbgooglefan]

Yes, that worked. I "protected" the variable which I did not want to
get freed. I incremented reference for that variable & it started
working.

Thanks for all your guidance.