Integrating Python into a C++ app

[Ben Sizer]

I know the conventional wisdom is to write the whole app in Python and
only extend with C++ where speed is an issue, but I already have a
large C++ app that I'd like to add Python to. Ideally I'd rewrite the
whole app in Python but I don't have time to do that and maintain the
old system at the same time. So my next thought was to perhaps
integrate the two and slowly migrate modules and classes across from
C++ to Python. If it matters, the main reason I'm interested in doing
this is because I appreciate the productivity of Python and would like
to take advantage of that as I add features to the current code, to
reduce bugs and cut development time.

I've read a few good things in this group about Elmer
(http://elmer.sourceforge.net), but I'm not sure how simply that
accommodates calls in the reverse direction (from Python code back into
C++). Are there any other options that would require a minimum of
rewriting of code? Does anybody have any experience of such a project?

 

[Alex Martelli]

I know the conventional wisdom is to write the whole app in Python and
only extend with C++ where speed is an issue, but I already have a
large C++ app that I'd like to add Python to. Ideally I'd rewrite the
whole app in Python but I don't have time to do that and maintain the
old system at the same time. So my next thought was to perhaps
integrate the two and slowly migrate modules and classes across from
C++ to Python. If it matters, the main reason I'm interested in doing
this is because I appreciate the productivity of Python and would like
to take advantage of that as I add features to the current code, to
reduce bugs and cut development time.

This seems quite a reasonable approach to me.

I've read a few good things in this group about Elmer
(http://elmer.sourceforge.net), but I'm not sure how simply that
accommodates calls in the reverse direction (from Python code back into
C++). Are there any other options that would require a minimum of
rewriting of code? Does anybody have any experience of such a project?

Sorry, haven't tried elmer yet. When I did such embedding a few years
ago I used Boost Python (www.boost.org) -- Boost has kept growing better
with the years so today it should be even easier. OTOH, the C++
application was already well structured according to Lakos' ideas
(expressed in his book about large-scale program design in C++):
components communicating across interfaces, rather than a free-for-all
of jumbles of dependencies. The interfaces being abstract C++ classes,
and the building of concrete instances always going through Factory
design patterns, Boost's abilities to wrap C++ classes as Python types
and let Python classes inherit from such wrapped C++ classes so that
normal virtual-method-call C++ approaches proved sufficient.

(In the end, after several experiments, we ended up using COM, since the
application only ran on Windows anyway; much of the COM was of course
done in Python, but that's another issue).

I suspect that few legacy C++ applications are well structured in terms
of components and interfaces. But then, trying to do any kind of major
surgery without serious rearchitecting tends to produce inferior results
anyway. And the rearchitecting is quite advisable whether you do end up
using any Python, or not -- you want good, clean components, cohesive
and coherent, with a good graph of dependencies, unit-tests for each
component, dependency inversion (a la Robert Martin) to avoid
dependencies going the wrong way, etc, etc. Lakos and Martin are the
two authors I would suggest reading. Lakos' book is old, and among his
major concerns is using just the subset of C++ you could rely on, years
ago; you can probably ignore those issues safely today (to use Boost,
you need a good recent C++ compiler with impeccable template support,
anyway, for example).


Alex