the new great computer language shootout

[Phil Tomson]

FYI: The great computer language shootout is now being hosted by one of
the Debian groups. It's been updated to use the latest
compilers/interpreters (Ruby 1.8.1 is in there now).

You can find it here:
http://shootout.alioth.debian.org/index.php

It's interesting that Python has now moved ahead of Perl in performance -
in the older shootout Python was still slower than Perl. Ruby is
somewhere near the middle of the scale ( slower than both Perl and
Python).

Phil

 

[Lennon Day-Reynolds]

Part of the speedup appears to be from their use of the Psyco JIT
compiler for the Python code -- the links to source code for Python
end in '.psyco'.

I would guess that some of the Ruby code needs to be optimized, as
well; for example, in the Array Access test, Ruby places dead last.
That's rediculous, and I don't think it's indicative of the array
performance for general cases.

Lennon

 

[Eric Hodel]

--4C6bbPZ6c/S1npyF
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

I would guess that some of the Ruby code needs to be optimized, as
well; for example, in the Array Access test, Ruby places dead last.
That's rediculous, and I don't think it's indicative of the array
performance for general cases.

This is because of the use of Numeric#step. The implementation is
flawed.

--=20
Eric Hodel - (e-mail address removed) - http://segment7.net
All messages signed with fingerprint:
FEC2 57F1 D465 EB15 5D6E 7C11 332A 551C 796C 9F04


--4C6bbPZ6c/S1npyF
Content-Type: application/pgp-signature
Content-Disposition: inline

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.2 (FreeBSD)

iD8DBQFA0hysMypVHHlsnwQRAn7lAKC7pTzxm5piJAGGMyE8LeBDRtsSzQCeNAe0
kpYB9vsrPkXq6ggbsRhEjyw=
=FSDa
-----END PGP SIGNATURE-----

--4C6bbPZ6c/S1npyF--

 

[Eric Hodel]

--9Q2l3mYpK16UQ/iv
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

This is because of the use of Numeric#step. The implementation is
flawed.

Never mind, I am smoking crack.

The real problem appears to be iteration in general.

--=20
Eric Hodel - (e-mail address removed) - http://segment7.net
All messages signed with fingerprint:
FEC2 57F1 D465 EB15 5D6E 7C11 332A 551C 796C 9F04


--9Q2l3mYpK16UQ/iv
Content-Type: application/pgp-signature
Content-Disposition: inline

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.2 (FreeBSD)

iD8DBQFA0iMnMypVHHlsnwQRAmZ2AKDKWgyuQqXnAlx+dGAo1m2wUPuiKwCgqto+
E0Ov0cb46p3ABluneTelINM=
=L/2K
-----END PGP SIGNATURE-----

--9Q2l3mYpK16UQ/iv--

 

[Lennon Day-Reynolds]

To be honest, some of the numbers seem highly suspect to me. For
example, on my P4/Fedora Core 1 box at least, the Sieve of
Eratosthenes example runs 20-30% *faster* in Ruby than it does in
Python, while his dataset shows is taking almost twice as long.

Lennon

 

[Mark Hubbart]

Never mind, I am smoking crack.

The real problem appears to be iteration in general.

surprisingly, using += on line 18 give a 12% speedup on my box.

cheers,
Mark

 

[Michael Campbell]

To be honest, some of the numbers seem highly suspect to me. For
example, on my P4/Fedora Core 1 box at least, the Sieve of
Eratosthenes example runs 20-30% *faster* in Ruby than it does in
Python, while his dataset shows is taking almost twice as long.

Are you using psyco? As has been mentioned, it looks as if the python
stuff has been done using that.

 

[Lennon Day-Reynolds]

The Sieve results I got weren't using Psyco; using it, the difference
in performance is about what the shootout lists, proportionally.
However, it requires source-level changes to the files, and the source
code on the shootout site doesn't included those changes. It's also
not mentioned in the language information page for Python, or listed
as a seperate implementation.

Psyco does some great things for the performance of certain
algorithms, but it's not part of the core Python distribution for any
number of reasons: only works on x86, breaks several standard
introspection and debugging libraries, and perhaps most importantly,
prevents redefinition of methods at runtime on optimized objects.

In short, it's almost more like a specialized reimplementation of
Python (ala Stackless) than an add-on library. I don't think it's
significantly more appropriate to use for this kind of benchmarking
than, say, Ruby or Perl scripts rewritten to use one of the Inline
libraries to do the heavy listing in C...especially when it's not
explicitly mentioned on the benchmark site.

Lennon

 

[Michael Campbell]

The Sieve results I got weren't using Psyco; using it, the difference
in performance is about what the shootout lists, proportionally.
However, it requires source-level changes to the files, and the source
code on the shootout site doesn't included those changes. It's also
not mentioned in the language information page for Python, or listed
as a seperate implementation.

...

No disputes from me, I was just mentioning I think that's what they
did, and that it might explain the discrepancy.

 

[Phil Tomson]

Are you using psyco? As has been mentioned, it looks as if the python
stuff has been done using that.

Isn't psycho kind of a special varient of Python? Is it fair to compare
psycho Python in the language shootout (they should probalby use the more
widely used PythonC).

Phil

 

[Carl Youngblood]

I agree that they really should stick strictly to Python code if they
are comparing languages. We don't want to know how well C performs
with a Python outer shell, we want to know how well Python performs.

Carl

 

[GGarramuno]

This is because of the use of Numeric#step. The implementation is
flawed.

Huh? In what way is it flawed? While there you can use slicing for
some operations, other array operations need iteration thru the array
as showed.
I do find the shootout is pretty accurate in terms of real ruby
performance overall.
The new web page is pretty bad in that most links are broken due to
typos and what not, so it is not clear to me the python tests are
based on psyco. There does appear to be a separate psyco benchmark.

 

[David Ross]

Huh? In what way is it flawed? While there you can use slicing for

In what way is it flawed? The whole thing is flawed. It is not fair to judge
the languages like this. Especially when they are compiled with GNU gcc, the
optimizations are really poor. It would be fair if someone used an optimized
compiler to compile ruby and the others then compete. As you can see Java
apears as first on some of the lists. I will state the fact that Java is not
compiled by GCC. They are probably using a optimized comiler. which makes
everything faster. I would like to see someone compiler ruby by Intel's
Compiler ICC. I'm sure Ruby would then be better than java in tests. Java has
optimized code and assembly. theres no possible way much of anything can be
better when they are the ones using optimized comiler and tossing out the
binaries. ---David Ross

 

[Michael Geary]

Isn't psycho kind of a special varient of Python? Is it fair
to compare psycho Python in the language shootout (they
should probalby use the more widely used PythonC).

You may be thinking of Pyrex, a Python dialect that compiles to C and lets
you mix C and Python data types:

http://www.cosc.canterbury.ac.nz/~greg/python/Pyrex/

Pyrex is really quite nifty, especially for writing C extensions to Python
and for wrapping existing C libraries. You can still code in Python or close
to it, and let Pyrex write all the boilerplate C code for you. For
performance critical algorithms, if you're using C types the generated code
should be as fast as any other C code.

Psyco, OTOH, works with standard Python code. It's essentially a Just In
Time compiler for Python:

http://psyco.sourceforge.net/introduction.html

Psyco isn't part of the standard Python distribution--you need to install
the Psyco module and add this to the top of your source file:

import psyco; psyco.full()

But that's all there is to it. There's nothing special about the
language--it's standard Python.

Psyco is only for x86 at the moment, but it can really pep up Python code on
the x86.

What's fair for performance comparisons? Well, a thorough language benchmark
would measure Psyco, standard cPython, and Jython times for x86, plus
cPython and Jython for other popular processors.

These are all Open Source, so don't let anybody stop you from porting them
to Ruby...

-Mike

 

[Joao Pedrosa]

Hi,

What's fair for performance comparisons? Well, a
thorough language benchmark
would measure Psyco, standard cPython, and Jython
times for x86, plus
cPython and Jython for other popular processors.

These are all Open Source, so don't let anybody stop
you from porting them
to Ruby...

That's a lot of pythons.

I'll keep my precious Ruby, and worry about
performance later, when my programs and libraries are
written.

Perhaps one of the main reasons to use a script
language, is to use it as a glue language, and I'm
fine with that.

Cheers,
Joao



__________________________________
Do you Yahoo!?
New and Improved Yahoo! Mail - Send 10MB messages!
http://promotions.yahoo.com/new_mail

 

[Michael Neumann]

The Sieve results I got weren't using Psyco; using it, the difference
in performance is about what the shootout lists, proportionally.
However, it requires source-level changes to the files, and the source
code on the shootout site doesn't included those changes. It's also
not mentioned in the language information page for Python, or listed
as a seperate implementation.

Psyco does some great things for the performance of certain
algorithms, but it's not part of the core Python distribution for any
number of reasons: only works on x86, breaks several standard
introspection and debugging libraries, and perhaps most importantly,
prevents redefinition of methods at runtime on optimized objects.

Hmm, if we'd had namespace selectors in Ruby, such optimizations (e.g.
optimizing integer/floating point operations) would become possible, at
least theoretically, without breaking code or loosing introspection at
the overall level. I might be wrong, of course

Regards,

Michael

 

[Mauricio Fernández]

Hmm, if we'd had namespace selectors in Ruby, such optimizations (e.g.
optimizing integer/floating point operations) would become possible, at
least theoretically, without breaking code or loosing introspection at

They are actually possible with the current language; you'd "only"
need a code specializer + careful code cache invalidation (take a look
at the Self papers).

the overall level. I might be wrong, of course

--
Running Debian GNU/Linux Sid (unstable)
batsman dot geo at yahoo dot com

Q: What's the big deal about rm, I have been deleting stuff for years? And
never lost anything.. oops!
A: ...
-- From the Frequently Unasked Questions

 

[Michael Neumann]

They are actually possible with the current language; you'd "only"
need a code specializer + careful code cache invalidation (take a look
at the Self papers).

Hm, but isn't that much harder to implement? And you need all these
tests at runtime (well, should be no performance penalty).

For example:

a = 1
a += 2

This piece of code would become invalid, when the Integer#+ method is
redefined.

Hm, indeed, doesn't look imposible for me. What are the advantages (of
both)?

Regards,

Michael

 

[Mauricio Fernández]

Hm, but isn't that much harder to implement? And you need all these

========================

Indeed. The Self team needed many man-years and they knew what they were
doing. It took Sun quite a long time to create an acceptable VM/JIT
for a much less dynamic language like Java, and they had most of the
original Self crew working for them

tests at runtime (well, should be no performance penalty).

For example:

a = 1
a += 2

This piece of code would become invalid, when the Integer#+ method is
redefined.

That's what I referred to with 'code cache invalidation'; on
redefinition of Integer#+, all specialized code segments that used the
old one would be flushed out. This is performed on method redefinition,
which should be quite rare.

Hm, indeed, doesn't look imposible for me. What are the advantages (of
both)?

"Both"? If you're referring to selector namespaces, they don't simplify
the runtime needs that much, unless you forbid method redefinitions.
There could be some small speed increase, since code invalidation would
be a bit less frequent (would of course depend on the performance of
the code specializer).

--
Running Debian GNU/Linux Sid (unstable)
batsman dot geo at yahoo dot com

Linux: Where Don't We Want To Go Today?
-- Submitted by Pancrazio De Mauro, paraphrasing some well-known sales talk

 

[denis]

surprisingly, using += on line 18 give a 12% speedup on my box.

cheers,
Mark

It is not so surprising += is always faster in Ruby (from my
experience).This change would place us between mzscheme and mawk.
Note that many other languages (perl, icon lua...) uses += so it
should be allowed.

Ciao

Denis