Embedding Python

Discussion in 'Python' started by Martin, Oct 2, 2004.

  1. Martin

    Martin Guest

    Greetings,

    I am new to python and wish to embed python in an 3D graphics
    application to provide application automation. The high level goal is to
    be able to drive my app from a script for batch job like behavior rather
    than via the GUI (ie. I would like to run a script and see those changes
    reflected in the GUI as if the user had clicked buttons, etc.) The
    application is written in C++ and uses QT for the GUI.

    I have read the python docs on python.org and (using SWIG) have been
    able to create Python shadow wrappers around the C++ classes I want to
    allow users to have been able to call them via python scripts. This part
    works well. However in terms of embedding, I feel like I am missing
    something.

    For those of you familiar with QT, you know that QT apps run in a
    single-thread (essentially). Although QT supports threads, a
    QApplication based program is (by default) single-threaded. Callbacks
    (slots in QT speak) can be hooked up based on a users actions, but these
    callbacks all run in the main program thread.

    So here is my question ... What's the best way to integrate Python into
    an app such as the one I am describing.

    Here's what I have though of ...

    Alternative #1
    Add boiler plate python embedding code (ie. Py_Initialize,
    PyRun_SimpleFile(...), Py_Finalize) in a QT callback. Unfortunately,
    since the QT app is single threaded, this will wedge up the GUI, until
    the script is done. Not good.

    Alternative #2
    Keep the app as a single threaded app and interpret the script piece
    meal (ie. one statement at a time) at some timed interval. Allow for GUI
    updating between statement interpreting. Unfortunately, groveling over
    the Python C API, this does not look particularly easy either and would
    more or less require cutting/pasting much of the interpreter guts in my
    app. Furthermore, incomplete statements (ie. lines ending in : ) seem to
    present a problem to PyRun_SimpleString.

    Alternative #3
    Split the app up so that it is broken up into 2 threads (GUI and guts).
    The interpreter would run in the gut thread and change the internal data
    store. The GUI would then have to be smart enough to update itselft.
    Unfortunately, this requires major app re-architecting (something I am
    trying to avoid).

    Alternative #4
    Spawn an interpreter thread using multi-threaded interpreter calls. This
    would then require locking/unlocking shared resources between
    Interpreter thread and the rest of the app. It feels like if I am going
    to do this #3 may be a cleaner approach.

    I am sure someone out there has done this before.

    For me, the key point is that I want to add this functionality without
    disrupting the entire product. I am trying to evaluate what is
    architecturally the best approach and what will give me the most bang
    for my buck.

    Any help would be appreciated.

    Cheers and Thanks,

    Martin
    Martin, Oct 2, 2004
    #1
    1. Advertising

  2. > Greetings,
    >
    > I am new to python and wish to embed python in an 3D graphics
    > application to provide application automation. The high level goal is to
    > be able to drive my app from a script for batch job like behavior rather

    [snip]

    Martin,
    I would recommend looking at nebula device for that. Their 3D engine
    supports couple of different languages like Tcl/Tk, Lua and Python.
    http://www.radonlabs.de/
    Have fun,
    Dmitry/
    Dmitry Borisov, Oct 2, 2004
    #2
    1. Advertising

  3. Martin

    Martin Guest

    Dmitry Borisov wrote:
    >>Greetings,
    >>
    >>I am new to python and wish to embed python in an 3D graphics
    >>application to provide application automation. The high level goal is to
    >>be able to drive my app from a script for batch job like behavior rather

    >
    > [snip]
    >
    > Martin,
    > I would recommend looking at nebula device for that. Their 3D engine
    > supports couple of different languages like Tcl/Tk, Lua and Python.
    > http://www.radonlabs.de/
    > Have fun,
    > Dmitry/
    >
    >

    Thanks for the suggestion. Unfortunately, the application I am working
    on is is quite mature in its life-cycle and overhauling the core library
    is not an alternative.

    My question really boils down to:

    Can someone point me to a (minimal) example showing strategies for
    application automation using Python. If the example involves a QT app
    that would be a plus.

    Thanks

    Martin
    Martin, Oct 4, 2004
    #3
  4. Martin

    Martin Guest

    Martin wrote:

    > Dmitry Borisov wrote:
    >
    >>> Greetings,
    >>>
    >>> I am new to python and wish to embed python in an 3D graphics
    >>> application to provide application automation. The high level goal is to
    >>> be able to drive my app from a script for batch job like behavior rather

    >>
    >>
    >> [snip]
    >>
    >> Martin,
    >> I would recommend looking at nebula device for that. Their 3D engine
    >> supports couple of different languages like Tcl/Tk, Lua and Python.
    >> http://www.radonlabs.de/
    >> Have fun,
    >> Dmitry/
    >>
    >>

    > Thanks for the suggestion. Unfortunately, the application I am working
    > on is is quite mature in its life-cycle and overhauling the core library
    > is not an alternative.
    >
    > My question really boils down to:
    >
    > Can someone point me to a (minimal) example showing strategies for
    > application automation using Python. If the example involves a QT app
    > that would be a plus.
    >
    > Thanks
    >
    > Martin


    OOps ... I didn't realize that the nebula device sources were also
    available for perusal. I'll take a look at them. Hopefully this will help.

    Thanks,

    Martin
    Martin, Oct 4, 2004
    #4
  5. Martin

    Tom Cocagne Guest

    I've recently been working with both extending and embedding Python and
    have been wondering about some similar issues. Something that just recently
    occurred to me is that you might be able to use Python's Global Interpreter
    Lock to implement a fairly simple multi-threaded lock-step solution.
    Consider the following:

    1. Your program starts up, Initializes the Python interpreter, and acquires
    the GIL (Global Interpreter Lock).

    2. The program then creates a new thread to run the Python scripts. In
    theory, this thread will immediately block in an attempt to acquire the GIL

    3. In your main GUI loop, you set a periodic timer that expires at a rate of
    say.... 20 Hz (this would likely need tuning) that, upon expiry, uses the
    following pseudo-code:
     global_interpreter_lock.unlock()
     # Give Python approximately 20 msec of execution time
     main_application_thread.sleep(20 milliseconds)
     global_interpreter_lock.lock()
     .... continue normal QT processing .....

    I haven't put a whole lot of thought into it yet but on the surface it seems
    like an approach along these lines would be fairly easy to implement and
    should guarantee that only 1 thread is ever accessing your application.

    Using this approach would involve at least three tuneable parameters. The
    GUI timer to indicate how often Python should be given some CPU time, the
    delay to determine how long the Python interpreter should be allowed to
    run, and byte-code check interval Python uses to release the GIL (described
    in the C-API documentation on Python+threads). Getting all these right
    might take some time.

      Again, I have never actually tried this so there could be some demons
    lurking in the shadows. Anyone see problems with this approach?

     Cheers,

     Tom


    Martin wrote:

    > Greetings,
    >
    > I am new to python and wish to embed python in an 3D graphics
    > application to provide application automation. The high level goal is to
    > be able to drive my app from a script for batch job like behavior rather
    > than via the GUI (ie. I would like to run a script and see those changes
    > reflected in the GUI as if the user had clicked buttons, etc.) The
    > application is written in C++ and uses QT for the GUI.
    >
    > I have read the python docs on python.org and (using SWIG) have been
    > able to create Python shadow wrappers around the C++ classes I want to
    > allow users to have been able to call them via python scripts. This part
    > works well. However in terms of embedding, I feel like I am missing
    > something.
    >
    > For those of you familiar with QT, you know that QT apps run in a
    > single-thread (essentially). Although QT supports threads, a
    > QApplication based program is (by default) single-threaded. Callbacks
    > (slots in QT speak) can be hooked up based on a users actions, but these
    > callbacks all run in the main program thread.
    >
    > So here is my question ... What's the best way to integrate Python into
    > an app such as the one I am describing.
    >
    > Here's what I have though of ...
    >
    > Alternative #1
    > Add boiler plate python embedding code (ie. Py_Initialize,
    > PyRun_SimpleFile(...), Py_Finalize) in a QT callback. Unfortunately,
    > since the QT app is single threaded, this will wedge up the GUI, until
    > the script is done. Not good.
    >
    > Alternative #2
    > Keep the app as a single threaded app and interpret the script piece
    > meal (ie. one statement at a time) at some timed interval. Allow for GUI
    > updating between statement interpreting. Unfortunately, groveling over
    > the Python C API, this does not look particularly easy either and would
    > more or less require cutting/pasting much of the interpreter guts in my
    > app. Furthermore, incomplete statements (ie. lines ending in : ) seem to
    > present a problem to PyRun_SimpleString.
    >
    > Alternative #3
    > Split the app up so that it is broken up into 2 threads (GUI and guts).
    > The interpreter would run in the gut thread and change the internal data
    > store. The GUI would then have to be smart enough to update itselft.
    > Unfortunately, this requires major app re-architecting (something I am
    > trying to avoid).
    >
    > Alternative #4
    > Spawn an interpreter thread using multi-threaded interpreter calls. This
    > would then require locking/unlocking shared resources between
    > Interpreter thread and the rest of the app. It feels like if I am going
    > to do this #3 may be a cleaner approach.
    >
    > I am sure someone out there has done this before.
    >
    > For me, the key point is that I want to add this functionality without
    > disrupting the entire product. I am trying to evaluate what is
    > architecturally the best approach and what will give me the most bang
    > for my buck.
    >
    > Any help would be appreciated.
    >
    > Cheers and Thanks,
    >
    > Martin
    Tom Cocagne, Oct 5, 2004
    #5
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Darryl

    Embedding Python in Python

    Darryl, Oct 8, 2003, in forum: Python
    Replies:
    6
    Views:
    381
    Darryl
    Oct 14, 2003
  2. Phil Frost

    Embedding Python in Python

    Phil Frost, Aug 18, 2004, in forum: Python
    Replies:
    23
    Views:
    750
    Paul Rubin
    Aug 19, 2004
  3. Maurice LING

    embedding python in python

    Maurice LING, Sep 29, 2004, in forum: Python
    Replies:
    8
    Views:
    360
    Jeff Shannon
    Oct 1, 2004
  4. Kakacek
    Replies:
    1
    Views:
    370
    Kakacek
    Jan 12, 2006
  5. abhinav
    Replies:
    1
    Views:
    431
    Armin Steinhoff
    Feb 19, 2006
Loading...

Share This Page