Help! Python either hangs or core dumps when calling C malloc

[Lil]

Hi Everyone! I've been trying to figure out this weird bug in my
program. I have a python program that calls a C function that reads in
a binary file into a buffer. In the C program, buffer is allocated by
calling malloc. The C program runs perfectly fine but when I use python
to call the C function, it core dumps at malloc.
I've tried multiple binary files with different sizes and the result
is:

if file size is < 20 bytes , works fine
if file size is > 20 bytes, it hangs or core dumps.

Please help!!
Lil

 

[Larry Bates]

Question: Why not just use Python to read the file?

f=open(filename, 'rb')
fcontents=f.read()

If you need to manipulate what is in fcontents you
can use struct module and/or slicing.

Larry Bates

 

[Lil]

Hi Larry,
It's in the C code mainly because the buffer is an input to the
driver. The driver takes a char* as input and I didn't want to pass a
char* from python -> swig -> C since swig has memory leaks passing
pointers.
Do you think this is a Python issue or a Red Hat issue? I'm going to
try it on my windows machine now and see what happens.

thanks! Lil

 

[Fredrik Lundh]

It's in the C code mainly because the buffer is an input to the
driver. The driver takes a char* as input and I didn't want to pass a
char* from python -> swig -> C since swig has memory leaks passing
pointers.
Do you think this is a Python issue or a Red Hat issue?

I think we would have noticed by now if Python or Red Hat weren't
able to allocate and read 20 bytes. It's a bug in your program, and
you should concentrate on fixing it, not looking for bugs everywhere
else.

(quick guess: did you perhaps type malloc(sizeof(bytes)) instead of
malloc(bytes), or something similar)

</F>

 

[Lil]

I already double checked my C code. It runs perfectly fine in C without
any errors. So in my python program, I added a pdb.set_trace()
and step through the program and it did not dump. But when i took out
the tracing, the core dump came back. "sigh"

 

[Fredrik Lundh]

Lil wrote:'

I already double checked my C code. It runs perfectly fine in C without
any errors. So in my python program, I added a pdb.set_trace()
and step through the program and it did not dump. But when i took out
the tracing, the core dump came back. "sigh"

so triple-check it.

if your code overwrites the malloc buffer, it's perfectly possible
that it appears to run fine when you run it standalone or under the
debugger, but that doesn't mean that your code is fine.

</F>

 

[Mike Meyer]

Lil wrote:'


so triple-check it.

if your code overwrites the malloc buffer, it's perfectly possible
that it appears to run fine when you run it standalone or under the
debugger, but that doesn't mean that your code is fine.

The appropriate aphorism is: "Testing can never reveal the absence of
bugs."

<mike

 

[Christian Stapfer]

I already double checked my C code. It runs perfectly fine in C without
any errors. So in my python program, I added a pdb.set_trace()
and step through the program and it did not dump. But when i took out
the tracing, the core dump came back. "sigh"

Check your program for _uninitialized_variables_.
(Just a guess: but what other side-effect than
changing the values of uninitialized variables
- and the program's timing, of course - might
the stepping through with a debugger have?)

Regards,
Christian

 

[Frithiof Andreas Jensen]

I already double checked my C code. It runs perfectly fine in C without
any errors.

Errr - It runs perfectly fine without *announcing* any errors (while
gleefully urinating all over its memory space).

The programming model for "C" is "*you* asked for it, you got it mate!" - so
you have to "tell" the program how to debug memory by rebuilding it with
f.ex. Electric Fence, or whatever memory debug lib is preferred for your
box, and running that again.

The tools will spot mistakes faster and more reliable than the developer who
wrote the code. Use them.

So in my python program, I added a pdb.set_trace()
and step through the program and it did not dump. But when i took out
the tracing, the core dump came back. "sigh"

So??

You re-jiggled the bits, so something important got thrashed this time.

Run the C program trough with gdb and see what it does. It thrashes
something.

 

[Frithiof Andreas Jensen]

Check your program for _uninitialized_variables_.

good point.

(Just a guess: but what other side-effect than
changing the values of uninitialized variables
- and the program's timing, of course - might
the stepping through with a debugger have?)

that, f.ex. with some debuggers, all application memory is conveniently
zeroed so you might survive popping NOP's off the stack until you eventually
hit a valid return ...

I once traced through an embedded assembly program - that had been working
for years - with a logic analyser to re-create the source from the old flow
charts and the hundreds of patches that had been applied to the thing during
"comissioning", i.e. On Site Development.

I found that the "correct" program would somtimes get hit by an unitialised
interupt, the CPU would land between two instructions and start executing
garbage until it chrashed over something else and magically re-synchronised.
Fine stuff.

The reassembled program executed 40% faster without all the jump-to-patch
stuff so I had to re-adjust all the timers too.