Best way of debigging a C extension

[Paul Moore]

I'm writing a C extension. My environment is Python 2.5, with the
mingw compiler, on Windows XP. At the moment I'm debugging by
scattering printf() statements around, but it's not always easy. Is
there a better way of debugging - particularly for diagnosing crashes?

I have gdb (although I've hardly used it, but I can learn ) but if
I try building my extension with python setup.py build --debug, I get
an error because -lpython25_d does not exist. I'm not surprised by
this, as I don't have a debug build of Python - but that should be OK,
I'm only looking for debugging info from my code.

I tried copying libpython25.a to libpython25_d.a - my extension now
builds, but the resulting pyd is XXX_d.pyd, which won't import. If I
rename this to XXX.pyd, I can import and things seem to work - but it
seems a bit of a roundabout way of doing things. Is there a simpler
way that I've missed? It seems to me that this (debugging a C
extension without compiling a debug build of Python) would be a fairly
common situation, so I would have expected a "cleaner" way of doing
it.

At the very least, a documentation patch to explain the best way of
doing things might be useful. I'll see what I can put together based
on the responses I get here.

Thanks,
Paul.

 

[Miki]

Hello Paul,

I'm writing a C extension. My environment is Python 2.5, with the
mingw compiler, on Windows XP. At the moment I'm debugging by
scattering printf() statements around, but it's not always easy. Is
there a better way of debugging - particularly for diagnosing crashes?

No guaranteed to work ...

* Download WinDbg from http://www.microsoft.com/whdc/devtools/debugging/installx86.mspx#a
* Add "hard" breakpoints in your code using __asm int 3;
* Run your program

If everything works well, when a breakpoint is reached you'll get
message box asking if you want to debug the program.
When you hit "OK", WinDbg should open with your code. Just point it to
the source location and you should be set.

HTH,

 

[Paul Moore]

You have to build Python on your own to get debug builds. Only debug
builds allow to do extension debugging like memory leak finding.

The trouble is, I only have mingw to build extensions, not MSVC7.1 -
so I can't build Python (and I don't know if I still have the toolkit
compiler to build with that - I certainly don't have all the pieces
installed). With Python 2.6, I guess things will be better as I have
VS2008 Express, so I can use this to build a debug Python plus my
extension. It's more work than I really want to do to go that way, but
I guess it'll work.

You are working against the system
On Windows all extensions and shared libraries of a debug build have a
_d suffix.

That's what I thought. I was just hoping that using a debug build of
an extension would be usable with a standard release build of Python,
as that's what will be easy for most people to set up.

Ah, well, I'll stick to printf, combined with hacking a debug build to
work with a release Python. It's ugly and unreliable, maybe, but good
enough. Maybe sometime (probably not until I'm targeting 2.6 only)
I'll set up my own debug build of Python, in a virtual machine
somewhere.

Thanks for the explanation.
Paul.

 

[Diez B. Roggisch]

I'm writing a C extension. My environment is Python 2.5, with the
mingw compiler, on Windows XP. At the moment I'm debugging by
scattering printf() statements around, but it's not always easy. Is
there a better way of debugging - particularly for diagnosing crashes?

I have gdb (although I've hardly used it, but I can learn ) but if
I try building my extension with python setup.py build --debug, I get
an error because -lpython25_d does not exist. I'm not surprised by
this, as I don't have a debug build of Python - but that should be OK,
I'm only looking for debugging info from my code.

I tried copying libpython25.a to libpython25_d.a - my extension now
builds, but the resulting pyd is XXX_d.pyd, which won't import. If I
rename this to XXX.pyd, I can import and things seem to work - but it
seems a bit of a roundabout way of doing things. Is there a simpler
way that I've missed? It seems to me that this (debugging a C
extension without compiling a debug build of Python) would be a fairly
common situation, so I would have expected a "cleaner" way of doing
it.

At the very least, a documentation patch to explain the best way of
doing things might be useful. I'll see what I can put together based
on the responses I get here.

I never tried this on windows - but what happens if you start python
inside GDB, and then set breakpoints inside your extension?

This works flawlessly for me under *nix.

The debug-build of python isn't needed for this - and I doubt a bit that
it helps you much, as being inside the interpreter & getting detailed
information isn't your goal - you want to see your extensions functions,
what parameters they get and so on.


Diez

 

[Diez B. Roggisch]

It's probably a lot easier to just stick in a few printf()
calls in key places...

Easier than

$ gdb python
# set args myscript.py
# break some_function_of_mine


?


Diez

 

[Diez B. Roggisch]

It doesn't work well with all debugging symbols stripped and with -O2
optimized code. The compiler rearranges the code. You could try to
compile your own code with different compiler arguments. I don't know if
that's easily possible on Windows with Python 2.5.

As I said - as long as you don't care about the python debug code
itself, it works pretty well. Usually, the problems are within your
extension, which you have the full control over regarding symbols &
optimization settings.

Diez

 

[Paul Moore]

A debug build changes large parts of the ABI. Every PyObject struct
gains additional fields, lots of checks are added to Python's memory
allocation system and asserts are enabled. Debug extensions have a _d
suffix for a very good reason. We aren't just trying to make *your* life
harder.

Ah. That's the bit of information I was missing. If the ABI changes,
it's a wonder my hacking of my extension to build in debug mode worked
at all!

I suspect that if I could get distutils to let me specify that it
keeps debug symbols in a release-mode build, I could get what I
needed. But the arcane contortions needed to get distutils to do this
are beyond me

I'll stick with printf - it's served me well enough so far

Thanks for clarifying,
Paul.