Using python for a CAD program

D

David Cuthbert

Terry said:
I disagree that transactions are bad for CAD -- they may have
a different semantic role and the needed granularity may be
different, but the need to roll data back to an earlier revision
is just as present in drawings as it is for code or financial
transactions.

Sure, but that's called source control. So you do need transactions,
but for the entire database state, not on database operations. (This
was certainly true at Cadence -- there was a thriving third-party market
for handling source control on CDBA databases, and I never encountered
any code which operated on the database in a transactional manner.
OpenAccess did have some basic support for transactions, but I never
heard of anyone using them.)

Some kind of transactionality is needed for undo/redo, but this is
usually done in a different (some might say "more efficient", others
might say "hackier") method than how transactions are implemented for
RDBMS (that I've dealt with, anyway). I suspect this can be attributed
to two major factors: concurrent access is never an issue in such
systems, and the number of database objects representing a given state
is far larger than in, say, e-commerce.
 
M

Mike C. Fletcher

Thanks for this. I'm enjoying this discussion and I'm learning a lot
about people's views and how they differ from mine.

However, I'm still wondering about my original post.

Can the experts please comment on python's usage for the following:

1. Databases. Assuming I roll my own, does python have any
performance issues for this sort of thing?
CAD software has generally proven to be somewhat less than amenable to
relational database models. Relational databases are good for storing
and querying large numbers of same-structure things. CAD models tend to
be large numbers of different-structure things. An object database is
likely to give you less of an impedance mismatch. With either approach,
expect your database operations to be slow compared to running in
memory. Storing the model in memory while working (if you can fit it)
is going to be a lot snappier than keeping things out in a database.

There are OO databases that have decent caching (e.g. ZODB) that can
give you some of the benefits of storing in the database without needing
to have everything loaded up at once. But keep in mind that most OO
databases are *not* optimised for this type of use-case, you may need to
rather heavily customise a standard package if you decide to use one.
You'll have to handle the partition of the data such that you can run
your simulations with partial data-sets if you want to take advantage of
that (possible with architecture, interior design, game design,
etceteras, not sure if it's possible with electrical engineering).
2. GUI. Can Python command the various gui libraries fast enough to
qualify as game-quality, with transparency, anti-aliasing, animations,
etc?
While it might be possible to do this fast enough with Python, you will
almost certainly want to use a retained-mode engine for the graphics
display. There are quite a few retained-mode engines for Python, most
are built on OpenGL. Some of those engines are written in Python, more
in C or C++. Basically what the retained mode engine does for you is it
allows you to simply describe what's supposed to be there and let
something else deal with the problem of how to display it. Retained
mode engines tend to be written by fairly experienced 3D programmers who
are somewhat obsessed by performance issues. You can readily setup the
node-graph in (slower) Python code and then have the engine handle the
heavy per-frame rendering.

The retained mode engine will (should) take care of things such as
caching intermediate rendering data (such as simple data-pointers for
rendering entire swaths of your drawings with a single GL call). They
will only update those structures when you actually change something
that affects the rendering. Their formality of operation allows them to
make quite a few optimisations even though some are quite general
platforms (the more general, the fewer optimisations they can easily
make). (For simple drawings it's easy for you to make your own, better,
optimisations, but with more complex scenes such optimisations become
more and more cumbersome without introducing formal structures).
3. Computational stuff like simulations and rules-checking. I'm just
going to assume this needs to be in C++ or pyrex or SciPy or something.
Numpy will often be sufficient for many scientific simulations, but it's
all about what algorithms you need to implement.
4. Programmability. How do I embed an interpreter which has some
custom CLI commands and knowledge of the graphical conext and which has
exposure to the API used in the main application, such that to the user
all the necessary things are exposed by default, and so that it doesn't
feel like it's just a disconnected console window that he/she needs to
set up manually each time?
Most Python GUI libraries have a console widget into which you can pass
context. Doesn't particularly require a lot of work, just follow along
in the docs for the GUI widget. You can see how it's done with wxPython
and OpenGLContext here:

http://pyopengl.cvs.sourceforge.net/pyopengl/OpenGLContext/bin/visualshell.py?view=markup
5. Threads and parallelism. Should I even bother? I've read that
it's possibly more tricky with python than with normal dev tools. I've
never done it, but I'd like to at least think about it up front so
if/when this goes MT it's not a complete rewrite from the ground up.
Threads aren't any more complex with Python than other tools, if
anything they're easier (Python has a good basic API for threading).
The problem is that they are far less *useful* than in C or the like,
because the global interpreter lock means they don't let you take
advantage of multiple processors in the box in most cases. That said,
threading is always tricky, particularly during debugging. If you can
avoid it for most operations and only put things in threads that are
largely disconnected from other processes you'll likely be happier. The
big win would be if you are doing all of your simulation in C with an
event-based update such that the simulation generates changes as a
queued list of updates to the model. You could then have the simulation
yield the interpreter lock and seldom block the rest of the process
(save when it touches a Python object (i.e. the updates coming from or
going to the queue)).

Have fun,
Mike

--
________________________________________________
Mike C. Fletcher
Designer, VR Plumber, Coder
http://www.vrplumber.com
http://blog.vrplumber.com
 
B

baalbek

David said:
Some kind of transactionality is needed for undo/redo, but this is
usually done in a different (some might say "more efficient", others
might say "hackier") method than how transactions are implemented for
RDBMS (that I've dealt with, anyway). I suspect this can be attributed
to two major factors: concurrent access is never an issue in such
systems, and the number of database objects representing a given state
is far larger than in, say, e-commerce.

No, concurrent access is highly relevant; for example, on a team of
about 50 architects working on design and production drawings for a new
hospital, each floor was one 'drawing' (dwg file), and thus stored on
disk as a separate entity from the other floors.

Now, only one architect could work on one floor at any time! And as info
from the project goes (sums and statistics of data, for example areas),
the only reliable (an efficient) way to gather this was to hack an
in-house application that collected the desired info from the various
binary dwgs stored on disk, and saved this data this to a RDBMS, for
further report processing.

And as far as administering the dwgs (see to it that the mass of
drawings follow one dimensioning convention, similar fonts etc) the CAD
systems today are a pure joke: roughly 10-30% of the time the architects
spend on CAD is pure "hacking" each drawing to conform to standard,
problems with layers being out of sync with the rest, internal blocks
not being uptodate, etc etc etc.

And just to work on one small area of a huge floor, the architect has to
load the whole freaking floor...

This is in 2006, not 1986!

All of these problems (the technical ones, as well as problems of
concurrency) would be neatly solved if the data had been stored in a
RDBMS, instead being spread around on thousands of files on disk.

The Autocad DWG format is already in a database like structure (tables
being joined by keys etc), so the technical difficulty is not the
show-stopper here; I suspect the management at Autodesk will simply not
touch the milk-cow that Autocad has become, and does not understand the
needs of the average CAD user anyway...

I'm myself really fed up with the CAD situation of today, having battled
for 10 years the various problems that comes with todays stone-age CAD
systems. Jeez, 2006, and the state of the art CAD, Autocad, is still
basically a primitive, one-user binary application. What a pathetic joke!

Baalbek
 
D

David Cuthbert

baalbek said:
No, concurrent access is highly relevant; for example, on a team of
about 50 architects working on design and production drawings for a new
hospital, each floor was one 'drawing' (dwg file), and thus stored on
disk as a separate entity from the other floors.

Now, only one architect could work on one floor at any time! And as info
from the project goes (sums and statistics of data, for example areas),
the only reliable (an efficient) way to gather this was to hack an
in-house application that collected the desired info from the various
binary dwgs stored on disk, and saved this data this to a RDBMS, for
further report processing.

You have two orthogonal thoughts going here.

It makes sense to collect statistics from a given drawing into an RDBMS
in such a way that, whenever a design is checked in, the statistics are
synched with the RDBMS. Further, such a system could (and should) make
sure that no previous checkin has been clobbered. This is a classic
source control system. I'd be surprised if there aren't plenty of
add-ons which do this. At Cadence, we provided hooks for such systems
to automatically collect necessary information.

This does not mean the design itself should be stored as an RDBMS. As
I've stated previously, CAD data (both electrical and, it appears,
mechanical) does not lend itself to RDBMS relationship modeling.

What you want is an easy way to manage the information, not dictate the
storage format of the information.
And just to work on one small area of a huge floor, the architect has to
load the whole freaking floor...

I don't have any architectural experience, so I can't say whether this
makes sense or not. However, I think of this as being akin to
complaining, "Just to edit a single function, I have to load the entire
source file/check out the entire module/rebuild and rerun all the tests!"

Not that this is necessarily an invalid idea. However, my experience
with Cadence's tools makes me believe the existing behavior might just
be the lesser of two evils. CDBA has a strict library/cell/view
hierarchy; the checkout granularity here is (usually) at the view level
(that is, when you lock a design, you're locking the view). Two
designers could edit two different cells or views in the same library.
This leads to all kinds of data inconsistency problems -- cell borders
aren't lining up in a layout, for example, or the cached data isn't
valid, etc. It's a nightmare.

The newer OpenAccess system has an option to manage this at the library
level. Unfortunately, this is rarely enabled due to backwards
compatibility.
 
B

baalbek

David said:
This does not mean the design itself should be stored as an RDBMS. As
I've stated previously, CAD data (both electrical and, it appears,
mechanical) does not lend itself to RDBMS relationship modeling.

I simply do not agree with this.

A CAD program (like Autocad) is nothing
but an advanced database editor: the program loads data from a binary
file and creates its object (in memory) from the tables that it reads
from the file on the disk.

The point is that this could as well have been stored on tables in a
RDBMS; the process of loading the object once the data has been fetched
(either from a binary file, or a RDBMS) the process is similar.

The advantage of having the data stored in a RDBMS is many, amongst them
the ability to check-out just the data one needs, better administration
of the drawings (consistent plotting, dimensioning styles, fonts, etc)
and a much better tool for gathering statistics of the project (sums of
areas, rooms, doors, etc etc).

What happens in the CAD program (once loaded in memory) is simply
irrelevant to how the data are stored.

Regards,
Baalbek
 
D

David Cuthbert

baalbek said:
I simply do not agree with this.

A CAD program (like Autocad) is nothing
but an advanced database editor: the program loads data from a binary
file and creates its object (in memory) from the tables that it reads
from the file on the disk.

Well, then, good luck with this. Let us know when you've rediscovered
that simply structuring data in a table does not make an RDBMS.
 
P

Paddy

baalbek said:
I simply do not agree with this.

A CAD program (like Autocad) is nothing
but an advanced database editor: the program loads data from a binary
file and creates its object (in memory) from the tables that it reads

Hi Baalbek,
Athouh they are database editors, they are not relational database
editors. Relational database engines fit some type of problems but
others have found that RDBMS don't fit CAD data. They are just too slow
and add complexity in mapping the 'natural CAD data formats' to the
RDBMS table format.

- Pad.
 
B

baalbek

David said:
Well, then, good luck with this. Let us know when you've rediscovered
that simply structuring data in a table does not make an RDBMS.

Remember, a CAD binary file (at least most of them) stores its data in
tables that are already in a (first?) normal form, similar to that of a
RDBMS.

The CAD program parses the data, and the CAD objects are created by
database joins between the different tables (one table for layers, one
table for colors, one table for geometrical data of the CAD entities,
etc etc).

I never said it would be easy, but remember, the O/R mapping of today is
far ahead of what one had only 10 years ago.

Of course, I would not have the CAD client talk directly to the RDBMS,
with constantly updating the database, but through a O/R layer
(Hibernate is a good one for this) that the CAD client connects to
(either through CORBA, ICE or similar), and have the application server
do batch update/fetching of the data.

I think that those that shy away from a RDBMS CAD solution exaggerates
the complexity of the CAD structure, and underestimates the technologies
we have available today.

Regards,
Baalbek
 
B

baalbek

Paddy said:
Hi Baalbek,
Athouh they are database editors, they are not relational database
editors. Relational database engines fit some type of problems but
others have found that RDBMS don't fit CAD data. They are just too slow
and add complexity in mapping the 'natural CAD data formats' to the
RDBMS table format.

Well, considering that most CAD programs (at least those I'm familiar
with) structures its data in a very similar fashion as a RDBMS, AND
given the advanced object/relational mapping tools we have today, how
hard could it be?

The CAD system does not even have to be a straight client-server, but a
three-tiered system where the application server do batch
update/fetching of the data for efficiency.


Regards,
Baalbek
 
P

Paddy

I guess the 'advanced O/R mapping tools' make it easier to map the data
to an RDBMS, but their is still the performance issue. Since this has
degenerated into a an issue of performance then I suggest the original
poster create a clear interface between his data and its persistance
method. This should make it easier to attach another, if his first
choice is not up to it.

- Cheers, Pad.
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Similar Threads

3D Cad Viewer in website 0
Using python for a CAD program 0
Using for loops in Python? 5
python for EE CAD program 8
CAD file format specifications? 0
Tips for using Github??? 3
OT: CAD 0
Simple Program 0

Members online

Forum statistics

Threads
473,744
Messages
2,569,483
Members
44,902
Latest member
Elena68X5

Latest Threads

Top