C Wrapper Function, crashing Python?

J

Java and Swing

static PyObject *wrap_doStuff(PyObject *self, PyObject *args) {
// this will store the result in a Python object
PyObject *finalResult;

// get arguments from Python
char *result = 0;
char *in= 0;
char *aString = 0;
char *bString = 0;
MY_NUM *a;
MY_NUM *b;
int ok = PyArg_ParseTuple(args, "sss", &in, &aString, &bString);
if (!ok) return 0;

// do work to get a and b
// count - returns an int; GetVal - returns a char *
a = GetVal(aString, count(aString, ","));
b = GetVal(bString, count(bString, ","));

// make function call, which returns a char *
result = doStuff(in, a, b);

// save result in Python string
finalResult = PyString_FromString(result);

// free memory
PyMem_Free(result);
PyMem_Free(a);
PyMem_Free(b);

// return the result as a Python string
return finalResult;
}

....from python I can call this function 4 times...works fine. WHen I
call it for the fifth time python.exe crashes. im thinking some memory
problem in the wrapper function perhaps...but I am not sure. The
actually C function, doStuff can be called 5, 6,7...N times without a
problem
so i know its gotta be my wrapper.

Any ideas? Thanks!
 
J

Java and Swing

update:
if I use C's free(result), free(a) free(b) instead of PyMem_Free...I
only get one successfuly use/call of doStuff.

i.e.
// this works
doStuff(...)

// python crashes here
doStuff(...)
 
J

Java and Swing

one more update...

if I remove PyMem_Free and free(...) ...so no memory clean up...I can
still only call doStuff 4 times, the 5th attemp crashes Python.
 
A

Antoon Pardon

Op 2005-10-12 said:
static PyObject *wrap_doStuff(PyObject *self, PyObject *args) {
// this will store the result in a Python object
PyObject *finalResult;

// get arguments from Python
char *result = 0;
char *in= 0;
char *aString = 0;
char *bString = 0;
MY_NUM *a;
MY_NUM *b;
int ok = PyArg_ParseTuple(args, "sss", &in, &aString, &bString);
if (!ok) return 0;

// do work to get a and b
// count - returns an int; GetVal - returns a char *
a = GetVal(aString, count(aString, ","));
b = GetVal(bString, count(bString, ","));

// make function call, which returns a char *
result = doStuff(in, a, b);

// save result in Python string
finalResult = PyString_FromString(result);

// free memory
PyMem_Free(result);
PyMem_Free(a);
PyMem_Free(b);

// return the result as a Python string
return finalResult;
}

...from python I can call this function 4 times...works fine. WHen I
call it for the fifth time python.exe crashes. im thinking some memory
problem in the wrapper function perhaps...but I am not sure. The
actually C function, doStuff can be called 5, 6,7...N times without a
problem
so i know its gotta be my wrapper.

Any ideas? Thanks!
Click to expand...

Well assuming your doStuff is a C function that knows nothing of python.
it might be the PyMem_Free(result).
http://docs.python.org/api/memoryInterface.html says the following:

void PyMem_Free(void *p)
Frees the memory block pointed to by p, which must have been
returned by a previous call to PyMem_Malloc() or PyMem_Realloc().
Otherwise, or if PyMem_Free(p) has been called before, undefined
behavior occurs. If p is NULL, no operation is performed.

But your result wasn't allocated by a PyMem_Malloc, it was returned
to you by a C function.
 
J

Java and Swing

Antoon,
I just saw that to. I updated the code like so...

static PyObject *wrap_doStuff(PyObject *self, PyObject *args) {
// this will store the result in a Python object
PyObject *finalResult;

// get arguments from Python
char *result = 0;
char *in= 0;
char *aString = 0;
char *bString = 0;
MY_NUM *a = (MY_NUM *) PyMem_Malloc((20 * sizeof(MY_NUM) + 1); //
the array will be 20 long
MY_NUM *b = (MY_NUM *) PyMem_Malloc((20 * sizeof(MY_NUM) + 1); //
the array will be 20 long
int ok = PyArg_ParseTuple(args, "sss", &in, &aString, &bString);
if (!ok) return 0;

// do work to get a and b
// count - returns an int; GetVal - returns a MY_NUM * (a pointer
to a MY_NUM array)
a = GetVal(aString, count(aString, ","));
b = GetVal(bString, count(bString, ","));

// make function call, which returns a char *
result = doStuff(in, a, b);

// save result in Python string
finalResult = PyString_FromString(result);

// free memory
PyMem_Free(a);
PyMem_Free(b);
free(aString);
free(bString);
free(result);

// return the result as a Python string
return finalResult;
}

...as you can see, i malloc'ed memory, and free'd the memory. However,
I still have python crashing (after only 3 successful calls to
doStuff). And yes, doStuff is a plain C function...nothing related to
Python.
 
J

Java and Swing

Antoon,
I just saw that to. I updated the code like so...

static PyObject *wrap_doStuff(PyObject *self, PyObject *args) {
// this will store the result in a Python object
PyObject *finalResult;

// get arguments from Python
char *result = 0;
char *in= 0;
char *aString = 0;
char *bString = 0;
MY_NUM *a = (MY_NUM *) PyMem_Malloc((20 * sizeof(MY_NUM) + 1); //
the array will be 20 long
MY_NUM *b = (MY_NUM *) PyMem_Malloc((20 * sizeof(MY_NUM) + 1); //
the array will be 20 long
int ok = PyArg_ParseTuple(args, "sss", &in, &aString, &bString);
if (!ok) return 0;

// do work to get a and b
// count - returns an int; GetVal - returns a MY_NUM * (a pointer
to a MY_NUM array)
a = GetVal(aString, count(aString, ","));
b = GetVal(bString, count(bString, ","));

// make function call, which returns a char *
result = doStuff(in, a, b);

// save result in Python string
finalResult = PyString_FromString(result);

// free memory
PyMem_Free(a);
PyMem_Free(b);
free(aString);
free(bString);
free(result);

// return the result as a Python string
return finalResult;
}

...as you can see, i malloc'ed memory, and free'd the memory. However,
I still have python crashing (after only 3 successful calls to
doStuff). And yes, doStuff is a plain C function...nothing related to
Python.
 
J

Java and Swing

Sorry about the double post...

anyhow, after putting in debug statements I found that it was crashing
when it called, free(result)....so I removed the free(result).

now it crashes when it gets to, b = GetVal(bString, count(bString,
","));

...any ideas?
 
J

Java and Swing

ok, further digging...I found that in the C function GetVal...it is
crashing where I try to malloc some memory. Note, I have no problems
when running this from C..just from Python using my wrapper.

GetVal looks something like..
MY_NUM *GetVal(const char *in, const int x) {
MY_NUM *results, *returnResults;
results = (MY_NUM *) malloc((x * sizeof(MY_NUM) + 1);
returnResults = results;
// ..do more work..
return returnResults;
}

I put in print statements into the C code and found that it is crashing
at the line where I malloc some space for "results".

....any ideas why this is crashing when calling from Python via C
wrapper?
 
B

Bernhard Herzog

Java and Swing said:
static PyObject *wrap_doStuff(PyObject *self, PyObject *args) { [...]
char *aString = 0;
char *bString = 0; [...]
int ok = PyArg_ParseTuple(args, "sss", &in, &aString, &bString); [...]
free(aString);
free(bString);
Click to expand...

aString and bString are pointers to memory managed by the strings in
your args tuple. You must not free them! The memory is automatically
managed by Python.

Bernhard
 
J

Java and Swing

thanks for the tip, however even when I do not free aString or bString,
i'm still crashing at the malloc in the c function, not the wrapper.


Bernhard said:
Java and Swing said:
static PyObject *wrap_doStuff(PyObject *self, PyObject *args) { [...]
char *aString = 0;
char *bString = 0; [...]
int ok = PyArg_ParseTuple(args, "sss", &in, &aString, &bString); [...]
free(aString);
free(bString);
Click to expand...

aString and bString are pointers to memory managed by the strings in
your args tuple. You must not free them! The memory is automatically
managed by Python.

Bernhard
Click to expand...
 
B

Bernhard Herzog

Java and Swing said:
thanks for the tip, however even when I do not free aString or bString,
i'm still crashing at the malloc in the c function, not the wrapper.
Click to expand...

Do you have any more places where you use free incorrectly? In my
experience, calling free with invalid values can corrupt the data
structures used by the memory allocator in such a way that subsequent
malloc calls crash.

Bernhard
 
J

Java and Swing

As far as my C Wrapper functions are concerned...I no longer have the
need for free(...). I do use PyMem_Free, for structures I allocated by
using PyMem_New(...).

In my C code I do have things such as...

char *foo(const char *in) {
char *tmp;
tmp = (char *) malloc((strlen(in) * sizeof(char)) + 1);
strcpy(tmp, in);
...
...
free(tmp);
return someValue;
}

Is that appropriate? I was under the impression that when you malloc
memory, you free it when done.

I also have things like...

char *bar(char *in) {
char *results, *finalResults;
results = (char *) malloc(...);
finalResults = results;

while (...) { *results++ = some_val; }

return finalResults;
}

....is that correct?

As I mentioned earlier, when I run these functions from C I have no
troubles...I can run them 5, 10, 15 times, etc. From Python, using my
wrapper function is when I have trouble.

Thanks in advance.
 
B

Bernhard Herzog

Java and Swing said:
char *foo(const char *in) {
char *tmp;
tmp = (char *) malloc((strlen(in) * sizeof(char)) + 1);
strcpy(tmp, in);
...
...
free(tmp);
return someValue;
}

Is that appropriate? I was under the impression that when you malloc
memory, you free it when done.
Click to expand...

Looks fine. I hope someValue does not point somewhere into the tmp
buffer, though.
I also have things like...

char *bar(char *in) {
char *results, *finalResults;
results = (char *) malloc(...);
finalResults = results;

while (...) { *results++ = some_val; }

return finalResults;
}
Click to expand...

Seems OK, too, assuming the results buffer is big enough.

Bernhard
 
J

Java and Swing

Bernhard said:
Looks fine. I hope someValue does not point somewhere into the tmp
buffer, though.
Click to expand...

someValue doesn't.
Seems OK, too, assuming the results buffer is big enough.
Click to expand...

results = (char *) malloc((sizeof(char) * strlen(in) + 1);
..should be big enough. At any rate, it works four times in a row
before it fails.

I wonder what else or how else to solve this. Why is malloc'ing
the memory crashing it!?
 
T

Tony Nelson

"Java and Swing said:
one more update...

if I remove PyMem_Free and free(...) ...so no memory clean up...I can
still only call doStuff 4 times, the 5th attemp crashes Python.
Click to expand...

I think your wrapper should look something like:

static PyObject *wrap_doStuff(PyObject *self, PyObject *args)
{
// this will store the result in a Python object
PyObject *finalResult;

// get arguments from Python
char *result = 0;
char *in= 0;
char *aString = 0;
char *bString = 0;
MY_NUM *a;
MY_NUM *b;

int ok = PyArg_ParseTuple(args, "sss", &in, &aString, &bString);
if (!ok) return 0;

// do work to get a and b
// count - returns an int; GetVal - returns a char *
a = GetVal(aString, count(aString, ","));
b = GetVal(bString, count(bString, ","));

// make function call, which returns a char *
result = doStuff(in, a, b);

// save result in Python string
finalResult = PyString_FromString(result);

// free memory
free(result);
free(a);
free(b);

// return the result as a Python string
return finalResult;
}

You must match malloc() with free(), and PyMem_Malloc() with
PyMem_Free(). Malloc() and free() usually crash /after/ the call that
did the damage. You may wish to avail yourself of your platform's
malloc debugging facilities.

Note that I don't do this stuff in C, I do it in pyrex, and I'm new to
it anyway, so there may still be something wrong. Unless you are
determined to learn how to do this in C, I think you should switch to
pyrex.
________________________________________________________________________
TonyN.:' *firstname*nlsnews@georgea*lastname*.com
' <http://www.georgeanelson.com/>
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Similar Threads

When to free memory for C wrapper? 1
Wrapper function 5
callling python function in c 2
how avoid delay while returning from C-python api? 0
How to create an instance of a python class from C++ 7
Numpy ndarray to C array 1
Lexical Analysis on C++ 1
Array of structs function pointer 10

Members online

Total: 29 (members: 1, guests: 28)
Robots: 380

Forum statistics

Threads
473,755
Messages
2,569,536
Members
45,020
Latest member
GenesisGai

Latest Threads

Top