Embedding Python

[Martin]

Greetings,

I am new to python and wish to embed python in an 3D graphics
application to provide application automation. The high level goal is to
be able to drive my app from a script for batch job like behavior rather
than via the GUI (ie. I would like to run a script and see those changes
reflected in the GUI as if the user had clicked buttons, etc.) The
application is written in C++ and uses QT for the GUI.

I have read the python docs on python.org and (using SWIG) have been
able to create Python shadow wrappers around the C++ classes I want to
allow users to have been able to call them via python scripts. This part
works well. However in terms of embedding, I feel like I am missing
something.

For those of you familiar with QT, you know that QT apps run in a
single-thread (essentially). Although QT supports threads, a
QApplication based program is (by default) single-threaded. Callbacks
(slots in QT speak) can be hooked up based on a users actions, but these
callbacks all run in the main program thread.

So here is my question ... What's the best way to integrate Python into
an app such as the one I am describing.

Here's what I have though of ...

Alternative #1
Add boiler plate python embedding code (ie. Py_Initialize,
PyRun_SimpleFile(...), Py_Finalize) in a QT callback. Unfortunately,
since the QT app is single threaded, this will wedge up the GUI, until
the script is done. Not good.

Alternative #2
Keep the app as a single threaded app and interpret the script piece
meal (ie. one statement at a time) at some timed interval. Allow for GUI
updating between statement interpreting. Unfortunately, groveling over
the Python C API, this does not look particularly easy either and would
more or less require cutting/pasting much of the interpreter guts in my
app. Furthermore, incomplete statements (ie. lines ending in : ) seem to
present a problem to PyRun_SimpleString.

Alternative #3
Split the app up so that it is broken up into 2 threads (GUI and guts).
The interpreter would run in the gut thread and change the internal data
store. The GUI would then have to be smart enough to update itselft.
Unfortunately, this requires major app re-architecting (something I am
trying to avoid).

Alternative #4
Spawn an interpreter thread using multi-threaded interpreter calls. This
would then require locking/unlocking shared resources between
Interpreter thread and the rest of the app. It feels like if I am going
to do this #3 may be a cleaner approach.

I am sure someone out there has done this before.

For me, the key point is that I want to add this functionality without
disrupting the entire product. I am trying to evaluate what is
architecturally the best approach and what will give me the most bang
for my buck.

Any help would be appreciated.

Cheers and Thanks,

Martin

 

[Dmitry Borisov]

Greetings,

I am new to python and wish to embed python in an 3D graphics
application to provide application automation. The high level goal is to
be able to drive my app from a script for batch job like behavior rather

[snip]

Martin,
I would recommend looking at nebula device for that. Their 3D engine
supports couple of different languages like Tcl/Tk, Lua and Python.
http://www.radonlabs.de/
Have fun,
Dmitry/

 

[Martin]

Greetings,

I am new to python and wish to embed python in an 3D graphics
application to provide application automation. The high level goal is to
be able to drive my app from a script for batch job like behavior rather

[snip]

Martin,
I would recommend looking at nebula device for that. Their 3D engine
supports couple of different languages like Tcl/Tk, Lua and Python.
http://www.radonlabs.de/
Have fun,
Dmitry/

Thanks for the suggestion. Unfortunately, the application I am working
on is is quite mature in its life-cycle and overhauling the core library
is not an alternative.

My question really boils down to:

Can someone point me to a (minimal) example showing strategies for
application automation using Python. If the example involves a QT app
that would be a plus.

Thanks

Martin

 

[Martin]

Greetings,

I am new to python and wish to embed python in an 3D graphics
application to provide application automation. The high level goal is to
be able to drive my app from a script for batch job like behavior rather

[snip]

Martin,
I would recommend looking at nebula device for that. Their 3D engine
supports couple of different languages like Tcl/Tk, Lua and Python.
http://www.radonlabs.de/
Have fun,
Dmitry/

Thanks for the suggestion. Unfortunately, the application I am working
on is is quite mature in its life-cycle and overhauling the core library
is not an alternative.

My question really boils down to:

Can someone point me to a (minimal) example showing strategies for
application automation using Python. If the example involves a QT app
that would be a plus.

Thanks

Martin

OOps ... I didn't realize that the nebula device sources were also
available for perusal. I'll take a look at them. Hopefully this will help.

Thanks,

Martin

 

[Tom Cocagne]

I've recently been working with both extending and embedding Python and
have been wondering about some similar issues. Something that just recently
occurred to me is that you might be able to use Python's Global Interpreter
Lock to implement a fairly simple multi-threaded lock-step solution.
Consider the following:

1. Your program starts up, Initializes the Python interpreter, and acquires
the GIL (Global Interpreter Lock).

2. The program then creates a new thread to run the Python scripts. In
theory, this thread will immediately block in an attempt to acquire the GIL

3. In your main GUI loop, you set a periodic timer that expires at a rate of
say.... 20 Hz (this would likely need tuning) that, upon expiry, uses the
following pseudo-code:
 global_interpreter_lock.unlock()
 # Give Python approximately 20 msec of execution time
 main_application_thread.sleep(20 milliseconds)
 global_interpreter_lock.lock()
 .... continue normal QT processing .....

I haven't put a whole lot of thought into it yet but on the surface it seems
like an approach along these lines would be fairly easy to implement and
should guarantee that only 1 thread is ever accessing your application.

Using this approach would involve at least three tuneable parameters. The
GUI timer to indicate how often Python should be given some CPU time, the
delay to determine how long the Python interpreter should be allowed to
run, and byte-code check interval Python uses to release the GIL (described
in the C-API documentation on Python+threads). Getting all these right
might take some time.

  Again, I have never actually tried this so there could be some demons
lurking in the shadows. Anyone see problems with this approach?

 Cheers,

 Tom