Design question: use of OO for scientific simulation...

[Egg]

Hi all,

I have a slightly embarassingly naive question or set of questions.(I
am not posting the code because this iteration is in Python--but Ruby
is next and you all are better OO folks too.)

I am writing a class to do Lee Carter demographic projection of
mortality (write me if you care about the details): basically involves
a bunch of 30 x 100 x 18 float matrices, a little linear algebra, and
some simple monte carlo simulation.

Here are parts of the problem:

* We need to organize storage of a bunch of 3-d matrices and derived
parameters (which take lots of time to calculate, so want to retain
them for later). Various dimensions of the matrices correspond to
various things, like years of step ahead, age, simulation run, etc, so
keeping track of that would be good too. I think this stuff belongs in
an instance class.

* We have several parameter-finding and simulation-running algorithms,
taking big matrices and returning big matrices. I think these should
be written as functions which don't retain state across calls--then
they are more testable since they don't require an instantiation of
anything.

* Each instance needs to display itself in a number of different views
including graphics and text, often for webservers (so in html), but
also text for command line use.

* Long term storage, both for command line playing and to put into part
of a web service (probably run as part of a queue and stored for later
viewing).

* The basic sequence of interaction is: 1, store some death rates. 2,
infer some parameters from them. 3, run some simulations. 4, infer
some more parameters from those simulations. 5, display it all in a
couple of different ways.

Anyway, I was wondering if anyone has done anything that they liked
with a situation like this, and could share a pattern or two. The
closest design problem might be analysing and simulating genetic
sequences, if that matters.

 

[Phil Tomson]

Hi all,

I have a slightly embarassingly naive question or set of questions.(I
am not posting the code because this iteration is in Python--but Ruby
is next and you all are better OO folks too.)

I am writing a class to do Lee Carter demographic projection of
mortality (write me if you care about the details): basically involves
a bunch of 30 x 100 x 18 float matrices, a little linear algebra, and
some simple monte carlo simulation.

Here are parts of the problem:

* We need to organize storage of a bunch of 3-d matrices and derived
parameters (which take lots of time to calculate, so want to retain
them for later). Various dimensions of the matrices correspond to
various things, like years of step ahead, age, simulation run, etc, so
keeping track of that would be good too. I think this stuff belongs in
an instance class.

* We have several parameter-finding and simulation-running algorithms,
taking big matrices and returning big matrices. I think these should
be written as functions which don't retain state across calls--then
they are more testable since they don't require an instantiation of
anything.

* Each instance needs to display itself in a number of different views
including graphics and text, often for webservers (so in html), but
also text for command line use.

* Long term storage, both for command line playing and to put into part
of a web service (probably run as part of a queue and stored for later
viewing).

* The basic sequence of interaction is: 1, store some death rates. 2,
infer some parameters from them. 3, run some simulations. 4, infer
some more parameters from those simulations. 5, display it all in a
couple of different ways.

Anyway, I was wondering if anyone has done anything that they liked
with a situation like this, and could share a pattern or two. The
closest design problem might be analysing and simulating genetic
sequences, if that matters.

Sorry, my advice isn't pattern related. You mention matrices, so I wanted
to recommend Narray if you haven't already looked at it. It'll speed
things up.

Phil

 

[jason r tibbetts]

I have a slightly embarassingly naive question or set of questions.(I
am not posting the code because this iteration is in Python--but Ruby
is next and you all are better OO folks too.)

I am writing a class to do Lee Carter demographic projection of
mortality (write me if you care about the details): basically involves
a bunch of 30 x 100 x 18 float matrices, a little linear algebra, and
some simple monte carlo simulation.

Here are parts of the problem:

* We need to organize storage of a bunch of 3-d matrices and derived
parameters (which take lots of time to calculate, so want to retain
them for later). Various dimensions of the matrices correspond to
various things, like years of step ahead, age, simulation run, etc, so
keeping track of that would be good too. I think this stuff belongs in
an instance class.
Correct.

* We have several parameter-finding and simulation-running algorithms,
taking big matrices and returning big matrices. I think these should
be written as functions which don't retain state across calls--then
they are more testable since they don't require an instantiation of
anything.

Sounds reasonable. But since you'll still need instances of the matrices
to be converted, you could put these functions into a Module and have
the matrices transform themselves.

* Each instance needs to display itself in a number of different views
including graphics and text, often for webservers (so in html), but
also text for command line use.

I'd write an abstract Displayer class, with subclasses to produce each
type of output. Having a class display itself with a to_s() method is
fair enough, but for the number and complexity of display types that you
propose, I think that a separate class is justified.

* Long term storage, both for command line playing and to put into part
of a web service (probably run as part of a queue and stored for later
viewing).

Ruby's serialization (Marshal module) should work fine here.

 

[Alan Chen]

* The basic sequence of interaction is: 1, store some death rates. 2,
infer some parameters from them. 3, run some simulations. 4, infer
some more parameters from those simulations. 5, display it all in a
couple of different ways.

Anyway, I was wondering if anyone has done anything that they liked
with a situation like this, and could share a pattern or two. The
closest design problem might be analysing and simulating genetic
sequences, if that matters.

I don't know how long it takes for your simulation to execute, but I
run simulations (aerospace) which can take anywhere from under a minute
to days on end. In my case the simulation itself is C++, but we've
implemented a lot of utilities in ruby -- input parameter permutation,
queue execution, post-process, and automated post process, plot
generation and packaging). Even if your core sim is in ruby, I think it
is often worth splitting up the actual simulation core from your pre
and post analysis tools.

If your organization doesn't mind you opening ports and running
webservers, a wiki is nice to post simulation data to, and it's easy to
annotate the output pages with notes. I'll mention RuWiki as a blatant
plug for the wiki that I started and Austin Zeigler has really picked
up and carried along (Project page at
http://rubyforge.org/projects/ruwiki/ and demo at
http://ruwiki.rubyforge.org/). RuWiki may actually be nice for you as
it has a fairly simple file system backend for which you could directly
write simulation output pages. I'm sure many other wiki's both in Ruby
and other languages would give you a nice ranges of choices.

Depending on how simple the inputs for your simulation are, you could
even try a fitnesse style operation, where the inputs are entered into
a wiki page and the server will come along and execute the scenario. If
your simulation model is anything like mine in complexity though, this
isn't really a feasable option though...

HTH

- alan