Py 2.5 on Language Shootout

[bearophileHUGS]

The The Computer Language Shootout has just published results for
Python 2.5 and Psyco 1.5.2. Comparing the old (Python 2.4) Gentoo
Pentium 4 results (now not visible anymore) with the new results, I
have seen that all the tests with Python 2.5 are faster than the ones
with Python 2.4 (some results can't be compared because N is changed):

Gentoo Pentium 4, Python 2.4.3 measurements:
Program & Logs CPU Time Memory KB GZip N
binary-trees 99.26 15,816 402 16
chameneos Timout 5.000.000
cheap-concurrency 23,13 5.252 160 15.000
fannkuch 66,38 2.200 395 10
fasta 81,62 9.884 861 2.500.000
k-nucleotide 15,52 15.580 459 250.000
mandelbrot 363,86 2.412 472 3.000
n-body Timout 20.000.000
nsieve 9,79 34.416 269 9
nsieve-bits 164,72 42.412 320 11
partial-sums 38,64 2.300 410 2.500.000
pidigits 9,22 2.388 391 2.500
recursive 701,64 14.360 344 11
regex-dna 6,21 24.160 326 500.000
reverse-complement 2,7 46.032 272 2.500.000
spectral-norm 696,76 2.456 266 2.500
startup 6,38 29 200
sum-file 8,08 2.364 61 8.000


Regarding Psyco, two tests only are worse (the sourcecode, CPU and SO
are the same):

Old (Python 2.4.3, older Psyco):
nsieve 4.22 22,680 211 9
reverse-complement 1.66 49,336 330 2,500,000

New (Python 2.5, Psyco 1.5.2):
nsieve 4.26 22,904 211 9
reverse-complement 1.75 52,056 330 2,500,000

Bye,
bearophile

 

[pgarrone]

Alioth is a great site for selecting the language in which to implement
primitives. Usually it's C.

Two of the alioth benchmarks, Partial-sums and Spectral-norm, could be
done using Numarray, or would be done with Numarray if most of the
program was in Python and there was a need to implement a similar
numerical procedure. The speed would be up near the compiled language
benchmarks. However the specific wording of these benchmarks prohibits
this approach. Spectral-norm must pretend the dataset is infinite, and
Partial-sums has to be implemented in a simple dumb loop.

Looking over the benchmarks, one gains the impression that Python is a
slow language.
My first serious Python programming exercise involved converting a 900
line Bash Shell program to a 500 line Python program, with a speedup
factor of 17. Using Python allowed an OO structure and advanced
containers, meaning the program was more maintainable and portable,
which were the main aims of the exercise. The speedup was a surprising
and welcome side benefit. I think it was mosly because the Python
byte-code interpreter is probably an order of magnitude faster than
Bash's direct interpretation, and because in Python system calls to
recurse directories and create symbolic links were not forked to
separate processes. In fact I would guess that the overall speed of the
Python program would be little less than a C program, given that most
of the time would be spent in system calls.

Its almost possible to make a large Python program arbitrarily fast by
profiling it and implementing slow bits as primitives. Size is probably
of greater concern.

 

[Isaac Gouy]

Alioth is a great site for selecting the language in which to implement
primitives. Usually it's C.

And for selecting a language for which you might need to implement
primitives in C

Two of the alioth benchmarks, Partial-sums and Spectral-norm, could be
done using Numarray, or would be done with Numarray if most of the
program was in Python and there was a need to implement a similar
numerical procedure. The speed would be up near the compiled language
benchmarks. However the specific wording of these benchmarks prohibits
this approach. Spectral-norm must pretend the dataset is infinite, and
Partial-sums has to be implemented in a simple dumb loop.

And we wouldn't use a naïve recursive algorithm to find fibonnaci
numbers ... unless we were interested in recursion for its own sake.

Maybe the author of spectral-norm was interested in function calls.
Maybe the author of partial-sums was interested in simple dumb loops
and simple dumb Math.

Looking over the benchmarks, one gains the impression that Python is a
slow language.

What does that even mean - a slow language?

My first serious Python programming exercise involved converting a 900
line Bash Shell program to a 500 line Python program, with a speedup
factor of 17. Using Python allowed an OO structure and advanced
containers, meaning the program was more maintainable and portable,
which were the main aims of the exercise. The speedup was a surprising
and welcome side benefit. I think it was mosly because the Python
byte-code interpreter is probably an order of magnitude faster than
Bash's direct interpretation, and because in Python system calls to
recurse directories and create symbolic links were not forked to
separate processes. In fact I would guess that the overall speed of the
Python program would be little less than a C program, given that most
of the time would be spent in system calls.

/I would guess/

Its almost possible to make a large Python program arbitrarily fast by
profiling it and implementing slow bits as primitives. Size is probably
of greater concern.

We could read that simply as - /it's not possible/ to make a large
Python program arbitrarily fast. Is that what you meant?

 

[pgarrone]

And for selecting a language for which you might need to implement
primitives in C

Well if you like C so much, just do it in C. ""

And we wouldn't use a naïve recursive algorithm to find fibonnaci
numbers ... unless we were interested in recursion for its own sake.

Maybe the author of spectral-norm was interested in function calls.
Maybe the author of partial-sums was interested in simple dumb loops
and simple dumb Math.

I am not disputing this. I think you take my point though.

What does that even mean - a slow language?

The alioth benchmarks provide a set of numbers by which
languages may be compared.

/I would guess/

I don't have the time, or interest, to recode it in C to find out.
In reality the choice would be C++ because of OO and STL.
Perhaps traversing and linking a tree containing about 1000 files would
not
take a full second. I might be wrong. All i know is, its a lot faster
than Bash.

We could read that simply as - /it's not possible/ to make a large
Python program arbitrarily fast. Is that what you meant?

No. I meant that if my Python program is too big in terms of its
execution memory
requirements, then that would be a difficult issue to deal with. Rather
than
optimizing execution hotspots, I might have to use another language.

Cheers,
P.S. Alioth is a great site.

 

[bearophileHUGS]

(e-mail address removed):

In reality the choice would be C++ because of OO and STL.

I have seen that when Python+Psyco are too much slow, D language is a
good sweet half point between Python and C++ Fast as C++ and with a
simpler syntax and semantics (Pyrex isn't bad, but D gives high-level
things at higher speed). And in the future ShedSkin and RPython may
become options too.

Bye,
bearophile

 

[Carl Friedrich Bolz]

>
> The alioth benchmarks provide a set of numbers by which
> languages may be compared.

Wrong. The benchmarks provide a set of numbers by which
_implementations_ of languages can be compared. After all, it is
possible that someone implements a magic-pixie-dust-interpreter that
executes Python programs several orders of magnitude fastes than
CPython. Or you could say that C is slow because if you use CINT, a C
interpreter ( http://root.cern.ch/root/Cint.html ) to execute it, it is
slow.

Cheers,

Carl Friedrich Bolz

 

[Ramon Diaz-Uriarte]

Wrong. The benchmarks provide a set of numbers by which
_implementations_ of languages can be compared. After all, it is
possible that someone implements a magic-pixie-dust-interpreter that
executes Python programs several orders of magnitude fastes than
CPython. Or you could say that C is slow because if you use CINT, a C
interpreter ( http://root.cern.ch/root/Cint.html ) to execute it, it is
slow.

Yeah, but this is hair-splitting. Except for Jython, IronPython, and
Stackless, I think when we say "Python is slow/fast" we think CPython
(otherwise, we qualify the implementation). For that matter it is
often said "the GIL ..."; oh, but wait, Stackless ...

With other languages (e.g., Common Lisp) the separation between the
language and the implementation is key because, to begin with, there
is something external from, and independent of, any particular
implementation. That is not the case with Python.

And the example of CINT is hair-splitting to the nth power.To begin
with, I do not think CINT implements the full standard C. But even if
it were, when people think of C they rarely think of CINT.

I think readers understood the previous poster.

Best,

R.

Cheers,

Carl Friedrich Bolz

--
Ramon Diaz-Uriarte
Statistical Computing Team
Structural Biology and Biocomputing Programme
Spanish National Cancer Centre (CNIO)
http://ligarto.org/rdiaz

 

[Isaac Gouy]

Well if you like C so much, just do it in C. ""

iirc I've never written a program in C or C++ (although I may have
fixed one or two that someone else wrote).

I am not disputing this. I think you take my point though.

I'm confident in my ability to misunderstood the point someone intended
to make - I think your point is some variation on 'we wouldn't write
the program like that' '"real-world" programs aren't like that'.

The alioth benchmarks provide a set of numbers by which
languages may be compared.

Unless we notice that sometimes there's more than one language
implementation for some languages; unless we notice that sometimes
there's more than one program for the same language implementation.
(Unless we read the bold text on the homepage.)

 

[Isaac Gouy]

Yeah, but this is hair-splitting. Except for Jython, IronPython, and
Stackless, I think when we say "Python is slow/fast" we think CPython
(otherwise, we qualify the implementation). For that matter it is
often said "the GIL ..."; oh, but wait, Stackless ...

When we say "Python is slow/fast" what does "slow/fast" mean?

With other languages (e.g., Common Lisp) the separation between the
language and the implementation is key because, to begin with, there
is something external from, and independent of, any particular
implementation. That is not the case with Python.

And the example of CINT is hair-splitting to the nth power.To begin
with, I do not think CINT implements the full standard C. But even if
it were, when people think of C they rarely think of CINT.

And that's why the existence of CINT is such a stark reminder of the
separation between the language and the implementation. When people
think of C what do they think of - gcc? tiny-c? intel c? microsoft c?
some mythical C implementation?

 

[Ramon Diaz-Uriarte]

When we say "Python is slow/fast" what does "slow/fast" mean?

Oh, well, I have no idea. I guess you'd have to define what "speed"
and, if that is a single number, then we can just rank languages. We'd
probably never agree on how to obtain a single number. But I think
most of us, when looking at the shootout, can see that there are some
languages that, for most of the programs, are consistently faster than
the rest, and others that are consistently slower. But really, I think
the shootout page has large, detailed and <b></b> statements about the
perils and pitfalls of measuring these things. (And I do not go to the
shootout to hear the oracle tell me which language I should use in my
next project).

And that's why the existence of CINT is such a stark reminder of the
separation between the language and the implementation. When people
think of C what do they think of - gcc? tiny-c? intel c? microsoft c?
some mythical C implementation?

Really, this ain't my war. Sure, there are two things: the language
and the implementation. But, for practical purposes, when most people
today say Python they mean CPython, whereas if the say Scheme, they
certainly need to say _which_ Scheme (I think only PLT is in the
official shootout page; there are others in the beta tests). If people
want to mean Jython or Stackless, they just say that. As for C, I
think people will need to qualify what exactly they mean.

I think all these issues do not really lead to confusion for most of
us; certainly not if you go to the shootout page. But as I said, this
ain't my war. I was simply pointing out that correcting one poster for
talking about languages when referring to python was hair splitting.

And I think we are all running in circles, because I guess we all
agree. This is turning into what in Spain (a country of catholic
tradition) we call a discussion about "the sex of the angels" (el sexo
de los angeles), i.e., whether angels are male or female or something
else. Since sexing angels is not my area of expertise, I'll just shut
up (I actually don't really know why I even said anything about this
issue; please, forgive my chatiness).

Best,

R.



--
Ramon Diaz-Uriarte
Statistical Computing Team
Structural Biology and Biocomputing Programme
Spanish National Cancer Centre (CNIO)
http://ligarto.org/rdiaz

 

[Isaac Gouy]

Oh, well, I have no idea. I guess you'd have to define what "speed"
and, if that is a single number, then we can just rank languages. We'd
probably never agree on how to obtain a single number. But I think
most of us, when looking at the shootout, can see that there are some
languages that, for most of the programs, are consistently faster than
the rest, and others that are consistently slower. But really, I think
the shootout page has large, detailed and <b></b> statements about the
perils and pitfalls of measuring these things. (And I do not go to the
shootout to hear the oracle tell me which language I should use in my
next project).

That's wise ;-)

Yes the shootout page has statements about the perils and pitfalls of
measuring these things - it's encouraging that at least one person has
noticed

Really, this ain't my war. Sure, there are two things: the language
and the implementation. But, for practical purposes, when most people
today say Python they mean CPython, whereas if the say Scheme, they
certainly need to say _which_ Scheme (I think only PLT is in the
official shootout page; there are others in the beta tests). If people
want to mean Jython or Stackless, they just say that. As for C, I
think people will need to qualify what exactly they mean.

I think all these issues do not really lead to confusion for most of
us; certainly not if you go to the shootout page. But as I said, this
ain't my war. I was simply pointing out that correcting one poster for
talking about languages when referring to python was hair splitting.

And I think we are all running in circles, because I guess we all
agree. This is turning into what in Spain (a country of catholic
tradition) we call a discussion about "the sex of the angels" (el sexo
de los angeles), i.e., whether angels are male or female or something
else. Since sexing angels is not my area of expertise, I'll just shut
up (I actually don't really know why I even said anything about this
issue; please, forgive my chatiness).

In England the corresponding expression is "Counting Angels on a
Pinhead"
http://dannyayers.com/2001/misc/angels.htm

 

[Fuzzyman]

Wrong. The benchmarks provide a set of numbers by which
_implementations_ of languages can be compared. After all, it is
possible that someone implements a magic-pixie-dust-interpreter that
executes Python programs several orders of magnitude fastes than
CPython. Or you could say that C is slow because if you use CINT, a C
interpreter ( http://root.cern.ch/root/Cint.html ) to execute it, it is
slow.

If you rephrase your argument to read 'it is never useful to discuss
the comparative speed of langauges' then hopefully the error should be
obvious. But then to discuss the speed of languages rather than
implementations would be a generalisations, and generalisations are
always wrong... ;-)

Fuzzyman
http://www.voidspace.org.uk/python/articles.shtml

 

[Ramon Diaz-Uriarte]

(...)



In England the corresponding expression is "Counting Angels on a
Pinhead"
http://dannyayers.com/2001/misc/angels.htm

Thanks, that is neat. I find the discussion on the sex of the angels,
well, sexier. But we are probably a few hundred years late to start a
catholic-protestant religious war here .


R.


--
Ramon Diaz-Uriarte
Statistical Computing Team
Structural Biology and Biocomputing Programme
Spanish National Cancer Centre (CNIO)
http://ligarto.org/rdiaz

 

[Carl Friedrich Bolz]

>
> Thanks, that is neat. I find the discussion on the sex of the angels,
> well, sexier. But we are probably a few hundred years late to start a
> catholic-protestant religious war here .
>

NOBODY expects the Spanish Inquisition! Our chief weapon is
surprise...surprise and fear...fear and surprise.

 

[Carl Friedrich Bolz]

>
> Thanks, that is neat. I find the discussion on the sex of the angels,
> well, sexier. But we are probably a few hundred years late to start a
> catholic-protestant religious war here .
>

NOBODY expects the Spanish Inquisition! Our chief weapon is
surprise...surprise and fear...fear and surprise.