Several questions about embedding Python

[Miki Tebeka]

Hello All,

I'm trying to extend a hardware simulator (written in C/C++) by
embedding Python.
After reading the documentation and playing around several questions
popped up:

1. Is there a way (using SWIG/Boost.Python ...) to export existing C/C++
functions other than hand writing the wrapper functions.
Something like: py_export(a_c_function)
Since the Python module need to get information from the simulator I
can't just write pure SWIG wrapper (or can I?)

2. How do I debug the Python code? Since the interpreter is called from
the simulator application I can't run `pdb' on it.

3. Why on does pyconfig.h decides that `python23_d.lib' should be used
when _DEBUG is on (IMO is should use Py_DEBUG)? I'm debugging *my*
application not Python.

Thanks.
Miki

 

[Cameron Laird]

Hello All,

I'm trying to extend a hardware simulator (written in C/C++) by
embedding Python.
After reading the documentation and playing around several questions
popped up:

1. Is there a way (using SWIG/Boost.Python ...) to export existing C/C++
functions other than hand writing the wrapper functions.
Something like: py_export(a_c_function)
Since the Python module need to get information from the simulator I
can't just write pure SWIG wrapper (or can I?)

2. How do I debug the Python code? Since the interpreter is called from
the simulator application I can't run `pdb' on it.

3. Why on does pyconfig.h decides that `python23_d.lib' should be used
when _DEBUG is on (IMO is should use Py_DEBUG)? I'm debugging *my*
application not Python.

.
.
.
I'm sympathetic on 3. Python's compile options ... well, *my*
preferred mental models don't match them well, yet. I have no
feel for the likelihood of a change or workaround here. With
luck, timbot will say a few words.

I don't know how to debug, in this sort of embedding context
in particular; the reason you cite is a good example of why.
I sit and think a lot, so that I understand what my applica-
tions do. I don't use debuggers. I'm still undecided about
whether it's helpful for me to confess that.

Yes, you can use pure SWIG wrappers. I'll be more precise: it
*sounds* as though you're saying SWIG is only for extending Py-
thon, and not embedding Python. This is false; SWIG can be used
to embed Python. Maybe there's a problem with use of SWIG in a
particular approach to embedding; just from what you've written,
though, I think it's reasonable to expect success with SWIG's
use.

 

[Duncan Booth]

I'm trying to extend a hardware simulator (written in C/C++) by
embedding Python.
After reading the documentation and playing around several questions
popped up:

1. Is there a way (using SWIG/Boost.Python ...) to export existing C/C++
functions other than hand writing the wrapper functions.
Something like: py_export(a_c_function)
Since the Python module need to get information from the simulator I
can't just write pure SWIG wrapper (or can I?)

If you have C-like functions (rather than C++ classes) take a look at Pyrex
(http://www.cosc.canterbury.ac.nz/~greg/python/Pyrex/). You still need to
hand write wrapper functions, but in most cases they are pretty short.

e.g. to wrap char *a_c_function(char *, int)
you might get away with:

cdef extern from "someheader.h":
char *real_a_c_function(char *, int) "a_c_function"

def a_c_function(s, i):
return real_a_c_function(s, i)

This would give you a module loadable in Python with a Python function
a_c_function that passes its arguments through to to the underlying
a_c_function written in C and converts the result back into a Python
string. The quoted string in the function prototype renames the C function
when used by Pyrex so you can reuse the same name.

In most cases you are likely to want to do a bit more with arguments and
results than just passing them through, and as soon as you do this Pyrex
wins over SWIG.

 

[Miki Tebeka]

Hello Cameron,

I don't know how to debug, in this sort of embedding context
in particular; the reason you cite is a good example of why.

Currently what I do is:
from pdb import run
....

def main():
...

if __name__ == "__main__":
run("main())"

If you have a .pdbrc in the current directory of the module you can set
breakpoints to it and it'll be loaded at import time.
When you hit a breakpoint the program will stop and you'll be in pdb
prompt.

I sit and think a lot, so that I understand what my applica-
tions do. I don't use debuggers. I'm still undecided about
whether it's helpful for me to confess that.

I agree. However there are times when a debugger is priceless, I found
it help mainly with "stupid" bug (such as forgetting to declare
"global", misspelling ...).

Yes, you can use pure SWIG wrappers. I'll be more precise: it
*sounds* as though you're saying SWIG is only for extending Py-
thon, and not embedding Python. This is false; SWIG can be used
to embed Python. Maybe there's a problem with use of SWIG in a
particular approach to embedding; just from what you've written,
though, I think it's reasonable to expect success with SWIG's
use.

How?
As I understand it SWIG will produce a DLL (sadly it's win2k).
I need that this DLL will share some data with the main application,
don't have a clue on how to do this.


Bye.

 

[Emmanuel]

Miki Tebeka a écrit :

Hello All,

I'm trying to extend a hardware simulator (written in C/C++) by
embedding Python.
After reading the documentation and playing around several questions
popped up:

1. Is there a way (using SWIG/Boost.Python ...) to export existing C/C++
functions other than hand writing the wrapper functions.
Something like: py_export(a_c_function)
Since the Python module need to get information from the simulator I
can't just write pure SWIG wrapper (or can I?)

? I don't understand...
Just have a header file with the function you want to call from Python, and
let SWIG parse it. That's what I'm doing...

2. How do I debug the Python code? Since the interpreter is called from
the simulator application I can't run `pdb' on it.

I used HAP editor/debugger, that can be quite simply included in a embedded
app.
But I must admit there are still some bugs in it, and I'm not sure it's
still actively developped...

3. Why on does pyconfig.h decides that `python23_d.lib' should be used
when _DEBUG is on (IMO is should use Py_DEBUG)? I'm debugging *my*
application not Python.

You should use :
#ifdef _DEBUG
#undef _DEBUG
#include "python.h"
#define _DEBUG
#else
#include "python.h"
#endif

instead of the main
#include "python.h"

to get rid of this issue.
Actually, I have a special MyPython.h for this. ( I add a special define in
order to allow this trick or not, I remember times where step into python
for debugging )

Hope it helps,

Emmanuel

 

[Miki Tebeka]

? I don't understand...
Just have a header file with the function you want to call from Python, and
let SWIG parse it. That's what I'm doing...

Yes. But the Python code is on the DLL generated by SWIG and the
functions it needs to call are part of the main executable.

I used HAP editor/debugger, that can be quite simply included in a embedded
app.
But I must admit there are still some bugs in it, and I'm not sure it's
still actively developped...

Thanks! I'll check it out.

You should use :
#ifdef _DEBUG
#undef _DEBUG
#include "python.h"
#define _DEBUG
#else
#include "python.h"
#endif

instead of the main
#include "python.h"

Yeah. I just make sure the last #include lines are #undef _DEBUG and
#include <Python.h>

Thanks.
Miki