Choosing a scripting language for scientific programming

[deltaquattro]

Hi,

I'm a Fortran scientific programmer, and recently I have had to write
a lot of shell scripts which let different codes communicate by means
of I/O files. Right now I'm using awk and (to a lesser extent) perl to
do this. However, I'd like to learn a newer scripting language and at
the same time learn OOP, so I'm choosing between Python and Ruby. The
applications I have in mind are:

1) (easy) automatic I/O files manipulation. Calculations, comparisons,
etc. among values contained in tables, and correspondent modfication
of the table entries;
2) (less easy) semi-automatic generation of graph, images, report,
etc., based on data in huge binary files (O(Gb));
3) (hard) create GUIs for scientifc codes, which could enable
unexperienced users to run the codes;

Which are the main differences between Ruby and Python? Which is
easier to learn? I've seen a lot of numerical libraries/extensions for
Python: does Ruby also support scientific programming? Thanks,

Best regards,

deltaquattro

 

[Bil Kleb]

Hi,
Hi.

I'm a Fortran scientific programmer,

Me too, although not so much anymore. Ruby's ruined it for me.

and recently I have had to write
a lot of shell scripts which let different codes communicate by means
of I/O files. Right now I'm using awk and (to a lesser extent) perl to
do this.

Sounds like me in 1999.

However, I'd like to learn a newer scripting language and at
the same time learn OOP, so I'm choosing between Python and Ruby.

That was we did underneath our Exploring XP for Scientific Research
IEEE article: ieeexplore.ieee.org/iel5/52/26915/01196317.pdf

1) (easy) automatic I/O files manipulation. Calculations, comparisons,
etc. among values contained in tables, and correspondent modfication
of the table entries;

All the time; just cooked another one up a couple hours ago to check input
parameter distributions for a Monte Carlos sensitivity analysis of
flowfield radiation of a capsule re-entering Earth at 10.5 km/s.

2) (less easy) semi-automatic generation of graph, images, report,
etc., based on data in huge binary files (O(Gb));

I've done at least 4 scientific papers with Ruby generated graphs,
images, reports, and tables based on data in huge binary files.
I've helped several other colleagues do the same.

3) (hard) create GUIs for scientifc codes, which could enable
unexperienced users to run the codes;

I'm embarking on this journey by wrapping simulation codes
in a RESTful web service, currently with Rails. So one gains
both a "GUI" via a web browser, but there is also a RESTful back
door to enable one to glue together simulation tools from
disparate disciplines.

Which are the main differences between Ruby and Python?

Ruby derives power from diversity; Python derives power from
the one true way. My brain best fit the former; YMMV.

I feel that creating a domain specific language is easier
in Ruby than Python, but my Python skills are /extremely/ weak.

Which is easier to learn?
Yes.

I've seen a lot of numerical libraries/extensions for
Python: does Ruby also support scientific programming?

Yes; see for instance http://sciruby.codeforpeople.com

Regards,

 

[Axel Etzold]

Dear deltaquattro,


-------- Original-Nachricht --------

Datum: Tue, 20 Nov 2007 22:10:07 +0900
Von: deltaquattro <[email protected]>
An: (e-mail address removed)
Betreff: Choosing a scripting language for scientific programming

Hi,

I'm a Fortran scientific programmer, and recently I have had to write
a lot of shell scripts which let different codes communicate by means
of I/O files. Right now I'm using awk and (to a lesser extent) perl to
do this. However, I'd like to learn a newer scripting language and at
the same time learn OOP, so I'm choosing between Python and Ruby. The
applications I have in mind are:

I don't know about Python, so I can't give you comparative information.
There is a guide to Ruby for programmers with hardly any prerequisites,
explaining the main features of the language, which you could go
through for the Ruby part of your comparison, to get a first impression
of that language and OOP:

http://pine.fm/LearnToProgram/

THE online reference to Ruby is the "Pickaxe book":

http://www.rubycentral.com/pickaxe/ .

This mailing list is of course a little biased towards Ruby when
it comes to comparing it to other languages .

1) (easy) automatic I/O files manipulation. Calculations, comparisons,
etc. among values contained in tables, and correspondent modfication
of the table entries;

To do file I/O manipulation, look at rio:

http://rio.rubyforge.org/ ,

if you want to read/write CSV data,

http://www.ruby-doc.org/stdlib/libdoc/csv/rdoc/index.html, may be of help.

You could also automate, say, Excel:

http://rubyonwindows.blogspot.com/2007/03/automating-excel-with-ruby-workbook.html

or connect to databases, such as MySQl, Postgresql etc ...

2) (less easy) semi-automatic generation of graph, images, report,
etc., based on data in huge binary files (O(Gb));

gd-graph (just do a search for library names on http://raa.ruby-lang.org/, if you want to know what exists)

can generate graphs,
ruport can generate reports,
rmagick can manipulate images,

3) (hard) create GUIs for scientifc codes, which could enable
unexperienced users to run the codes;

there are several GUI creation frameworks,
such as wxruby, fxruby, and others - there's just recently been
a discussion thread about pros and cons of those.

http://blade.nagaokaut.ac.jp/cgi-bin/vframe.rb/ruby/ruby-talk/279564?279371-279878

Which are the main differences between Ruby and Python? Which is
easier to learn? I've seen a lot of numerical libraries/extensions for
Python: does Ruby also support scientific programming? Thanks,

There is a website http://sciruby.codeforpeople.com/sr.cgi/InterestingProjects collecting
some scientific projects.

Then, there are bindings to GSL

http://rb-gsl.rubyforge.org/

and to the R statistical language,

http://raa.ruby-lang.org/project/rsruby/,

and others ...

An argument in favour of Ruby is its elegance and conceptual
versatility - I am much faster coding in Ruby than in other
languages I have used.
Yet, generally, Ruby tends to be slower when it comes to execution
times of algorithms than other scripting languages - if scripting
for you essentially means crunching huge amounts of numbers, it might
be worth considering extending Ruby right away with C via rubyinline

http://www.zenspider.com/ZSS/Products/RubyInline/

or with fortran - as you know it already . This one could use
some of your help ;-)

http://blog.zenspider.com/archives/2006/10/inlinefortran_100_released.html

Best regards,

Axel

 

[coreyhaines]

have you looked at F#?

Hi,

I'm a Fortran scientific programmer, and recently I have had to write
a lot of shell scripts which let different codes communicate by means
of I/O files. Right now I'm using awk and (to a lesser extent) perl to
do this. However, I'd like to learn a newer scripting language and at
the same time learn OOP, so I'm choosing between Python and Ruby. The
applications I have in mind are:

1) (easy) automatic I/O files manipulation. Calculations, comparisons,
etc. among values contained in tables, and correspondent modfication
of the table entries;
2) (less easy) semi-automatic generation of graph, images, report,
etc., based on data in huge binary files (O(Gb));
3) (hard) create GUIs for scientifc codes, which could enable
unexperienced users to run the codes;

Which are the main differences between Ruby and Python? Which is
easier to learn? I've seen a lot of numerical libraries/extensions for
Python: does Ruby also support scientific programming? Thanks,

Best regards,

deltaquattro

 

[Bil Kleb]

have you looked at F#?

Or http://fortress.sunsource.net ?

Regards,

 

[Chris Lowis]

I've done at least 4 scientific papers with Ruby generated graphs,
images, reports, and tables based on data in huge binary files.
I've helped several other colleagues do the same.

I'd love to have some more information, or a little write-up off how you
do some of these things, Bil. I'm currently trying to ween myself off
Matlab for these tasks.

Chris

 

[ara.t.howard]

I'd love to have some more information, or a little write-up off
how you
do some of these things, Bil. I'm currently trying to ween myself off
Matlab for these tasks.

Chris

for plotting

http://tioga.rubyforge.org/

for huge binary files

http://narray.rubyforge.org/

for general purpose computing

http://rubyforge.org/projects/rb-gsl/

note that the last is the second largest, closely behind rails,
project on rubyforge. it's very complete.

a @ http://codeforpeople.com/

 

[Paul van Delst]

Hi,

I'm a Fortran scientific programmer, and recently I have had to write
a lot of shell scripts which let different codes communicate by means
of I/O files. Right now I'm using awk and (to a lesser extent) perl to
do this. However, I'd like to learn a newer scripting language and at
the same time learn OOP, so I'm choosing between Python and Ruby. The
applications I have in mind are:

1) (easy) automatic I/O files manipulation. Calculations, comparisons,
etc. among values contained in tables, and correspondent modfication
of the table entries;
2) (less easy) semi-automatic generation of graph, images, report,
etc., based on data in huge binary files (O(Gb));
3) (hard) create GUIs for scientifc codes, which could enable
unexperienced users to run the codes;

Which are the main differences between Ruby and Python? Which is
easier to learn? I've seen a lot of numerical libraries/extensions for
Python: does Ruby also support scientific programming? Thanks,

Hello,

I use ruby to glue together a great deal of my Fortran executables. More importantly for
me, I also use ruby to generate a lot of boilerplate Fortran code. I've never used ruby
for graphing or GUIs since I have a large IDL library for the former (although the tioga
link is intriguing), and I'm not too interested in the latter (for my work stuff at least.
"Users? What users?" ).

I started stone cold comparing Ruby and Python a couple years ago to teach myself about
OOP (in preparation for Fortran2003 compilers [still waiting!] that have OO-capabilities)
and settled on ruby because its basic technical premise is easy for me to grok:
"everything is an object". And Yukihiro Matsumoto's words in the Foreword to the "Pickaxe"
book made up my mind: "I believe that the purpose of life is, at least in part, to be
happy. Based on this belief, Ruby is designed to make programming not only easy but also fun."

Crikey. How can you argue with that?! )

cheers,

paulv

p.s. I'm sure this is true of other popular scripting languages but, being primarily a
Fortran95 coder, I am constantly amazed at the speed and quality of innovation in the ruby
community. It may be because I'm used to the glacial pace of Fortran language standard
updates, but it's amazing nonetheless.

 

[ara.t.howard]

I use ruby to glue together a great deal of my Fortran executables.
More importantly for me, I also use ruby to generate a lot of
boilerplate Fortran code. I've never used ruby for graphing or GUIs
since I have a large IDL library for the former (although the tioga
link is intriguing), and I'm not too interested in the latter (for
my work stuff at least. "Users? What users?" ).

you might interesting in a few peices of code we use then:

session.rb : allows one to drive idl from ruby

idlrun: runs any idl routine from the command line using a special
syntax or yaml config file

yaml2idl: turns a yaml file into an idl declaration, we use this to
configure idl programs via yaml

regards.

a @ http://codeforpeople.com/

 

[Eric Lavigne]

p.s. I'm sure this is true of other popular scripting languages but, being primarily a

Fortran95 coder, I am constantly amazed at the speed and quality of innovation in the ruby
community. It may be because I'm used to the glacial pace of Fortran language standard
updates, but it's amazing nonetheless.

Ruby is one of the better ones, but yes this difference has more to do
with the glacial pace of Fortran development. I recommend trying OCaml
as a Fortran replacement. It fills the same niche as Fortran (high
performance computation and simulation) but is also well designed and
continues to improve at a brisk pace. OCaml and Ruby make an excellent
combination for scientific programming.

--
There are two ways of constructing a software design: One way is to
make it so simple that there are obviously no deficiencies, and the
other way is to make it so complicated that there are no obvious
deficiencies. The first method is far more difficult.

- C.A.R. Hoare -

 

[deltaquattro]

deltaquattrowrote:

Hi.

Hi, Bil,

sorry for not replying before to your very interesting post: I'm very
busy doing by hand all these marvelous things you do automatically
with Ruby I'm astonished by the degree of automatization that you
reached with Ruby. Now I understand how it was possible that a U.S.
engineer at Boeing was able to 1) set up, 2) run, 3) postprocess a
complex CFD simulation and 4) write a detailed report, before I could
even get point 3) done for the same test case. Clearly he uses Ruby
(or Python

I've done at least 4 scientific papers with Ruby generated graphs,
images, reports, and tables based on data in huge binary files.
I've helped several other colleagues do the same.

This is great: your colleagues are very lucky to have you in their
team. May I ask you some more information on how you do this?
[..]

Yes.

Pardon, I made a typo: I meant "Which one is easier to learn?". Do you
mean Ruby is easier to learn than Python?

Regards,

Thanks for all the information: as soon as I have some more time I'd
like to contact you privately on the subject, if that's ok for you.
BTW, will you attend AIAA conference in Reno, in January?

Best regards,

deltaquattro

 

[Eric Lavigne]

Which is easier to learn?

Pardon, I made a typo: I meant "Which one is easier to learn?". Do you
mean Ruby is easier to learn than Python?

I think that Python is easier to learn, but Ruby provides more
flexibility. Both of them are so easy to learn that it doesn't matter
which is easier (thus the answer is "yes"). You could even learn both
and decide which you like best.

I would recommend purchasing the second edition of "Programming Ruby:
The Pragmatic Programmer's Guide", but you can also read the first
edition online.
http://www.ruby-doc.org/docs/ProgrammingRuby/

I would recommend this tutorial for Python:
http://docs.python.org/tut/tut.html
Then this document as a reference:
http://docs.python.org/lib/lib.html

Also, once you have learned how to automate all those things that you
once did by hand, I strongly recommend learning OCaml next. Fortran is
no longer one of the better languages available, and it shouldn't be
used for new programs. You will find that programs written in OCaml
are shorter, simpler, and easier to write than equivalent Fortran
programs. When the process of programming becomes easier, one of the
results is that it becomes easier to experiment with ways to improve
your program's speed. The following book focuses on using OCaml for
the types of programs that are traditionally written in Fortran.

http://www.ffconsultancy.com/products/ocaml_for_scientists/

--
There are two ways of constructing a software design: One way is to
make it so simple that there are obviously no deficiencies, and the
other way is to make it so complicated that there are no obvious
deficiencies. The first method is far more difficult.

- C.A.R. Hoare -

 

[Bil Kleb]

Hi, Bil,
Hi.

Now I understand how it was possible that a U.S.
engineer at Boeing was able to 1) set up, 2) run, 3) postprocess a
complex CFD simulation and 4) write a detailed report, before I could
even get point 3) done for the same test case.

Curious: who and which CFD code?

This is great: your colleagues are very lucky to have you in their
team. May I ask you some more information on how you do this?
Certainly.


Pardon, I made a typo: I meant "Which one is easier to learn?". Do you
mean Ruby is easier to learn than Python?

I think they are both easy compared to Fortran. I don't think there
is any significant difference in ease of learning between the two.

Thanks for all the information: as soon as I have some more time I'd
like to contact you privately on the subject, if that's ok for you.

That's fine.

BTW, will you attend AIAA conference in Reno, in January?

No, I'm conferenced out for the time being: I have a NATO RTO in
Athens, Greece in the beginning of December, then I'm done for a while..

Regards,

 

[Marc Heiler]

"Right now I'm using awk and (to a lesser extent) perl to
do this."

One advantage with python or ruby would be that you can reuse
a lot of that, still have the functionality (of like awk or
near to, at least a nicer syntax), while still being able to
maintain it nicely (different to perl, maintaining perl
scripts STINKS in my opinion)

"However, I'd like to learn a newer scripting language and at
the same time learn OOP, so I'm choosing between Python and Ruby."

My Python is not that good...
I think python as a language is acceptable, more than perl in
practise. But Ruby's OOP model does not feel second class, different
to the python differences about length(object) or object.foo().
For example, python feels the need to pass def foo(self).
This is MANDATORY (although you can use any other name than self (!).

When you invoke a method, such as cat.meow, in python you
MUST use the meow(). Python says this is to your advantage, but to
me it is a fundamental difference in philosophy ...

It basically tells you, python way, or the highway.

Python forces, ruby allows ... ;-)


"2) (less easy) semi-automatic generation of graph, images, report,
etc., based on data in huge binary files (O(Gb));"

I think this can be a bit difficult in either language, but
whatever applies to python, will apply to ruby too. (Or to
perl in this regard, at least perl-ruby ... ruby is really
how perl should have been from the start, perl's OOP is
a shame, and perl offers 10000 ugly ways to do something
you write once, and then never look at it again ...)

Each language has some other advantages here and there,
for example it was my impression that python has a little bit
more bindings in general than ruby (i.e. blender plugin, ogre3d
wrapper and so on). Ruby catches up to this, but it still will
take a bit more time ...

"3) (hard) create GUIs for scientifc codes, which could enable
unexperienced users to run the codes"

I dare claim that ruby-gtk is (almost) as good as python-gtk.
So at least for gtk, there really is no difference IMHO to the
python variant (there is a tiny catch if you need to use a lot of
threads and stuff... but just have a look at recent mailing
list, in my opinion this is really no "showstopper" at all)

"Which are the main differences between Ruby and Python?"
I really think the attitude towards OOP is a huge difference.
In essence, ruby is a lot cleaner in this philosophical regard.
I can forgive python its indent-obsession, but i can not forgive
it the implicit self and forced () ... by the way, the default
interactive python is also annoying in its mindset, "exit" or
"quit" does not work... I am just glad I can alias that for
irb without a fuzz. Again a little philosophical difference ...
(But in my opinion, ruby is actually a lot more fun than python
as ruby doesn't superimpose as much as python does on you)

"Which is easier to learn?"
To be honest, I think BOTH are easy to learn but BOTH are
hard to master. I recommend you to use only a subset of any
language you choose, and slowly build up on it.

"I've seen a lot of numerical libraries/extensions for
Python: does Ruby also support scientific programming?"

Well there is gsl, also some stuff is in the Math module, then
we have some matrices and more stuff... I guess the hardcore
maths users go and use Haskell anyway

 

[Eric Lavigne]

I hope that the original poster will just start learning one of Ruby
or Python and not spend too much time thinking about the difference
for now. Learning the basics will only take a weekend, so it really
does make sense to learn both and decide for yourself. However, I
enjoy discussing the features and merits of programming languages, so
I will reply anyway. :-D

When you invoke a method, such as cat.meow, in python you
MUST use the meow(). Python says this is to your advantage, but to
me it is a fundamental difference in philosophy ...

It basically tells you, python way, or the highway.

Python forces, ruby allows ... ;-)

I believe that those parentheses are an advantage, and they actually
allow me greater flexibility in my code by making the method
invocation explicit. I have done a lot of functional programming, and
in Python if I leave the parentheses off of a method then I can pass
the method itself, uninvoked, as a parameter to another method. Typing
two parentheses for each method call is a small price to pay for this
convenience, especially since I end up typing as many parentheses in
either case (but in different places) to control order of operations.
It is strange to me that, in Ruby, if I try to pass around a method
that takes several arguments, it complains about not receiving the
arguments instead of just allowing itself to be passed around.

Another Python advantage is that idiomatic Python code is extremely
easy to read, even by mediocre programmers with no experience in the
language. The indent-obsession that you criticize helps a great deal
with that.

On the other hand, here I am in a Ruby newsgroup. I like the blocks
and iterators, the monkey patching, the greater support for concise
code. Yes, I also like the consistency of length being a method (one
less thing to remember), though I can't imagine why such a trivial
issue comes up so often in these conversations.

"I've seen a lot of numerical libraries/extensions for
Python: does Ruby also support scientific programming?"

Well there is gsl, also some stuff is in the Math module, then
we have some matrices and more stuff... I guess the hardcore
maths users go and use Haskell anyway

Haskell draws in programmer mathematicians, but most scientific
programmers aren't quite so hardcore in either programming or
mathematics. I wish that Haskell had better documentation for
beginners (a problem that Haskellers are certainly working hard at)
but for now the entrance barrier seems too high.

--
There are two ways of constructing a software design: One way is to
make it so simple that there are obviously no deficiencies, and the
other way is to make it so complicated that there are no obvious
deficiencies. The first method is far more difficult.

- C.A.R. Hoare -