Debug build without PYMALLOC

[Amaury]

Hello,

When debugging C extensions, I find the most difficult is to detect
that an object is Py_DECREF'ed when it should not.
And when you find yourself with an invalid object, it's a nightmare to
find where the error occured!
Normally tools like Purify are good to find this kind of memory
errors, but they are of no use with python's own memory allocator.

That's why I recompiled python (on Windows) in debug mode after
removing the WITH_PYMALLOC option.
This worked very well: Purify warned about a "Free memory read", and
it turned out that a custom sys.stdout was incorrectly DECREF'ed just
before, which caused the crash.

The problem is that this python build (debug/without-pymalloc) is not
compatible with extension modules built WITH_pymalloc. The DLL loader
fails with a "missing _PyObject_DebugMalloc" error.
Note that with release builds, extensions built with pymalloc are
compatible with python.dll built without pymalloc.

My workaround was to add the three functions (_PyObject_DebugMalloc,
*Realloc and *Free) in obmalloc.c.
Isn't there another way?
Otherwise, should this be integrated in Python?

Thanks,
Amaury.

 

[Michael Hudson]

Hello,

When debugging C extensions, I find the most difficult is to detect
that an object is Py_DECREF'ed when it should not.
And when you find yourself with an invalid object, it's a nightmare to
find where the error occured!
Normally tools like Purify are good to find this kind of memory
errors, but they are of no use with python's own memory allocator.

That's why I recompiled python (on Windows) in debug mode after
removing the WITH_PYMALLOC option.
This worked very well: Purify warned about a "Free memory read", and
it turned out that a custom sys.stdout was incorrectly DECREF'ed just
before, which caused the crash.

The problem is that this python build (debug/without-pymalloc) is not
compatible with extension modules built WITH_pymalloc. The DLL loader
fails with a "missing _PyObject_DebugMalloc" error.
Note that with release builds, extensions built with pymalloc are
compatible with python.dll built without pymalloc.

Hmm, that makes sense but I hadn't thought of it before. You could
also have recompiled the extensions, though.

My workaround was to add the three functions (_PyObject_DebugMalloc,
*Realloc and *Free) in obmalloc.c.
Isn't there another way?
Otherwise, should this be integrated in Python?

Possibly. Post a bug report or maybe a patch to SF if you feel
strongly about it...

Cheers,
mwh