Thanks for this. I'm enjoying this discussion and I'm learning a lot
about people's views and how they differ from mine.
However, I'm still wondering about my original post.
Can the experts please comment on python's usage for the following:
1. Databases. Assuming I roll my own, does python have any
performance issues for this sort of thing?
CAD software has generally proven to be somewhat less than amenable to
relational database models. Relational databases are good for storing
and querying large numbers of same-structure things. CAD models tend to
be large numbers of different-structure things. An object database is
likely to give you less of an impedance mismatch. With either approach,
expect your database operations to be slow compared to running in
memory. Storing the model in memory while working (if you can fit it)
is going to be a lot snappier than keeping things out in a database.
There are OO databases that have decent caching (e.g. ZODB) that can
give you some of the benefits of storing in the database without needing
to have everything loaded up at once. But keep in mind that most OO
databases are *not* optimised for this type of use-case, you may need to
rather heavily customise a standard package if you decide to use one.
You'll have to handle the partition of the data such that you can run
your simulations with partial data-sets if you want to take advantage of
that (possible with architecture, interior design, game design,
etceteras, not sure if it's possible with electrical engineering).
2. GUI. Can Python command the various gui libraries fast enough to
qualify as game-quality, with transparency, anti-aliasing, animations,
While it might be possible to do this fast enough with Python, you will
almost certainly want to use a retained-mode engine for the graphics
display. There are quite a few retained-mode engines for Python, most
are built on OpenGL. Some of those engines are written in Python, more
in C or C++. Basically what the retained mode engine does for you is it
allows you to simply describe what's supposed to be there and let
something else deal with the problem of how to display it. Retained
mode engines tend to be written by fairly experienced 3D programmers who
are somewhat obsessed by performance issues. You can readily setup the
node-graph in (slower) Python code and then have the engine handle the
heavy per-frame rendering.
The retained mode engine will (should) take care of things such as
caching intermediate rendering data (such as simple data-pointers for
rendering entire swaths of your drawings with a single GL call). They
will only update those structures when you actually change something
that affects the rendering. Their formality of operation allows them to
make quite a few optimisations even though some are quite general
platforms (the more general, the fewer optimisations they can easily
make). (For simple drawings it's easy for you to make your own, better,
optimisations, but with more complex scenes such optimisations become
more and more cumbersome without introducing formal structures).
3. Computational stuff like simulations and rules-checking. I'm just
going to assume this needs to be in C++ or pyrex or SciPy or something.
Numpy will often be sufficient for many scientific simulations, but it's
all about what algorithms you need to implement.
4. Programmability. How do I embed an interpreter which has some
custom CLI commands and knowledge of the graphical conext and which has
exposure to the API used in the main application, such that to the user
all the necessary things are exposed by default, and so that it doesn't
feel like it's just a disconnected console window that he/she needs to
set up manually each time?
Most Python GUI libraries have a console widget into which you can pass
context. Doesn't particularly require a lot of work, just follow along
in the docs for the GUI widget. You can see how it's done with wxPython
and OpenGLContext here:
5. Threads and parallelism. Should I even bother? I've read that
it's possibly more tricky with python than with normal dev tools. I've
never done it, but I'd like to at least think about it up front so
if/when this goes MT it's not a complete rewrite from the ground up.
Threads aren't any more complex with Python than other tools, if
anything they're easier (Python has a good basic API for threading).
The problem is that they are far less *useful* than in C or the like,
because the global interpreter lock means they don't let you take
advantage of multiple processors in the box in most cases. That said,
threading is always tricky, particularly during debugging. If you can
avoid it for most operations and only put things in threads that are
largely disconnected from other processes you'll likely be happier. The
big win would be if you are doing all of your simulation in C with an
event-based update such that the simulation generates changes as a
queued list of updates to the model. You could then have the simulation
yield the interpreter lock and seldom block the rest of the process
(save when it touches a Python object (i.e. the updates coming from or
going to the queue)).
Mike C. Fletcher
Designer, VR Plumber, Coder