Python future performance and speed

[Neuruss]

It seems there are quite a few projects aimed to improve Python's
speed and, therefore, eliminate its main limitation for mainstream
acceptance.
I just wonder what do you all think?
Will Python (and dynamic languages in general) be someday close to
compiled languages speed?
What will be the future of Psyco, Pypy, Starkiller, Ironpython and all
the other projects currently on development?

 

[Paul Rubin]

Will Python (and dynamic languages in general) be someday close to
compiled languages speed?

They already are (Lisp, etc.)

What will be the future of Psyco, Pypy, Starkiller, Ironpython and all
the other projects currently on development?

Hopefully they will speed up Python.

 

[Aahz]

It seems there are quite a few projects aimed to improve Python's
speed and, therefore, eliminate its main limitation for mainstream
acceptance.

What makes you think speed is Python's primary limitation for mainstream
acceptance?
--
Aahz ([email protected]) <*> http://www.pythoncraft.com/

"To me vi is Zen. To use vi is to practice zen. Every command is a
koan. Profound to the user, unintelligible to the uninitiated. You
discover truth everytime you use it." (e-mail address removed)

 

[Paul Rubin]

What makes you think speed is Python's primary limitation for mainstream
acceptance?

Well, whenever I want to write a mainstream application and think of
using Python, my first reaction is that Python is too slow...

 

[Tim Churches]

Well, whenever I want to write a mainstream application and think of
using Python, my first reaction is that Python is too slow...

As a population health epidemiologist whose stock-in-trade is
manipulation and analysis of large health data sets, I have to say that
Python's run-time speed is almost always much faster than I would have
expected from such a dynamic language, but also almost always much
slower than I would like, especially when compared to widely-used (but
vastly expensive and sprawling) data manipulation environments such as
SAS system (see http://www.sas.com) which is the "industry standard" in
my particular field. However the speed of programme creation with Python
is, in general, outstandingly swift, making it the environment of choice
for one-off tasks. But for routinely-executed jobs which process large
amounts of data, especially for jobs which can't be scheduled to run
overnight, Python is much slower than I would like. Numpy is a godsend
in situations in which it can be used, Psyco helps a lot, I am rapidly
warming to Pyrex (if you'll pardon the pun), and I can't wait to try out
Starkiller. But don't get me wrong: I love Python (which my better half
refers to as "ang babaeng ahas" - the "snake woman" - who has firmly
stolen my affections, or punningly, as "ang asawang sawa" - literally
the "python spouse" but with connotations of "the clinging consort" -
and I was recently much amused when a colleague revealed that she
seriously, albeit momentarily, thought that her object-oriented,
dynamic-language-phile programmer husband was having a torrid
extra-marital affair when she discovered hundreds of email in his in-box
referring to someone called Ruby).
--

Tim C

PGP/GnuPG Key 1024D/EAF993D0 available from keyservers everywhere
or at http://members.optushome.com.au/tchur/pubkey.asc
Key fingerprint = 8C22 BF76 33BA B3B5 1D5B EB37 7891 46A9 EAF9 93D0

 

[Sam Holden]

Well, whenever I want to write a mainstream application and think of
using Python, my first reaction is that Python is too slow...

That you have have an irrational first reaction is hardly the "main limitation"
for python.

After all python has been used as the scripting engine for a number of
commercial games - a domain that certainly cares about speed.

<http://www.pygame.org/interview/stevemoret.shtml> implies that a python script
runs on every frame for at least one commercial game.

 

[Anthony Baxter]

Well, whenever I want to write a mainstream application and think of
using Python, my first reaction is that Python is too slow...

Too slow for what?

I'm not being facetious here - what exactly are the bits of Python that
are too slow? I used to think this as well, but see my PyCon paper from
earlier this year.

 

[Dominic]

It seems there are quite a few projects aimed to improve Python's
speed and, therefore, eliminate its main limitation for mainstream
acceptance.
I just wonder what do you all think?

Well, I have written a (simple) graphical real time simulator
which does 1024x768x16 with at least 20 frames per
second on a PIII 800MHz notebook.
I have wrapped a game graphics library with Pyrex
which was not very difficult.
However the differential equation solver had been
slow in Python. So I rewrote in C++ and wrapped
it with Boost until I learned that using Pysco
would give me the same boost without C++!
So apart from the small graphics library wrapper
everything has been done in Python.
If JIT for Python gets better Pyrex and the likes
will "only" be useful for wrapping existing C/C++ code.

Ciao,
Dominic

 

[Ville Vainio]

Paul> Well, whenever I want to write a mainstream application and
Paul> think of using Python, my first reaction is that Python is
Paul> too slow...

What kind of mainstream application would be too slow if implemented
in Python? Well, perhaps image / video editing...

 

[Anthony Baxter]

What kind of mainstream application would be too slow if implemented
in Python? Well, perhaps image / video editing...

Maybe. You could, however, do the computationally expensive parts in
numarray.

(As a random-anecdote on the topic - experiments with using shtoom to
transmit and receive video over IP as well as audio over IP show that
Python is fast enough to do this...)

 

[Roy Smith]

What makes you think speed is Python's primary limitation for mainstream
acceptance?

Speed is the primary limitation for mainstream acceptance of high level
(aka scripting) languages in general. I can think of no reason why so
much software is still written in languages like C++ other than
performance (or at least, the perception of it).

 

[Roy Smith]

As a population health epidemiologist whose stock-in-trade is
manipulation and analysis of large health data sets, I have to say that
Python's run-time speed is almost always much faster than I would have
expected from such a dynamic language, but also almost always much
slower than I would like, especially when compared to widely-used (but
vastly expensive and sprawling) data manipulation environments such as
SAS system (see http://www.sas.com) which is the "industry standard" in
my particular field.

It should not be surprising at all that a general-purpose tool will be
beaten by a domain-specific tool in that domain. SAS, as you say, is
designed to munch huge numeric data sets. That is its only reason for
existing, and it's got 20 or 30 years of development effort behind it to
make it do that one task as fast as possible.

 

[Neuruss]

What makes you think speed is Python's primary limitation for mainstream

acceptance?

Why would someone reject to use a language that is clean, very well
designed, intuitive, expresive, flexible and a pleasure to use?
Perhaps the fact that i is 100 times slower than c...

 

[Neuruss]

They already are (Lisp, etc.)

That's a surprise! I didn't know about Lisp, so I did a quick search
in Google and I found this quote from CMUCL website:

"a sophisticated native-code compiler which is capable of powerful
type inferences, and generates code competitive in speed with C
compilers."

Has anyone tried to learn from this experience? Could this technology
be used with Python?

Regarding you Paul, I also found this:

"There's another guy I'd also like to invite, a Lisp expert, if that's
ok with you. He's been interested in writing a Python compiler for a
while. I'll ask him if he wants to join, but he might not."

What happened?

 

[Peter Hansen]

Speed is the primary limitation for mainstream acceptance of high level
(aka scripting) languages in general. I can think of no reason why so
much software is still written in languages like C++ other than
performance (or at least, the perception of it).

I'm glad you added that parenthetical comment, because I'm nearly
certain that it is *not* actual experience with Python, and
disappointment with its speed, that is the cause.

Far more likely, it seems to me, is that the primary limitation
for mainstream acceptance of scripting languages(*) in general
is ignorance. More specifically, a lack of knowledge of their
capabilities or ability to handle the job. I'm quite sure there
are far more people out there doing C++ or Java work who have
never *heard* of Python, than people who have heard of it and
have the perception it is too slow for their work, though
appropriate in other ways.

-Peter

* I limit this to scripting languages since it's quite certain
that the mainstream *has* accepted high level languages quite
fully. The last time I checked, Java and C++ (even C) were
widely considered to be high level languages. Has someone been
raising the bar while I wasn't looking?

 

[Peter Hansen]

Why would someone reject to use a language that is clean, very well
designed, intuitive, expresive, flexible and a pleasure to use?
Perhaps the fact that i is 100 times slower than c...

Statements like that serve only to demonstrate the narrowness of
thought of the writer.

In some cases, often contrived examples or with poorly written code,
Python can be about 100 times slower than C.

Much more often it lies somewhere between 10 and 30 times slower.

In a surprising number of cases, it is faster than 10 times
slower and in a few cases it can even achieve rough parity.

And if we have to rehash the old arguments again for those who haven't
taken the time to search the archives, we can point out once
more things like how a Python program will be completed much
sooner than the C program, resulting in lots of extra time to
run the program (and many programs are not run very often).

We can also point to the fact that most performance issues are
more heavily affected by poor algorithms than by anything else,
and the Python programmer has both the tools and the time to
improve the algorithms whereas the C programmer will not...

-Peter

 

[Kirk Job-Sluder]

Far more likely, it seems to me, is that the primary limitation
for mainstream acceptance of scripting languages(*) in general
is ignorance.

I think there are a few other reasons as well. I think that the general
python policy of making things clear and obvious is incompatible with
software business models that consider compilation and distribution of
executables as a way to hide trade secrets.

Another problem with interpreted languages is that the interpreter and
libraries for might be moving targets. I've not specifically had any
problems with python, but I have had problems with an update to R
killing functionality for a library I was using. I've also have had
problems with some Java applications depending on which JRE I might be
running on a system.

 

[Francesco Bochicchio]

* I limit this to scripting languages since it's quite certain
that the mainstream *has* accepted high level languages quite
fully. The last time I checked, Java and C++ (even C) were
widely considered to be high level languages. Has someone been
raising the bar while I wasn't looking?

I heard sometime Python referred as a 'Very High Level Language', that is
a notch on the top of Java, and more than that on top of C++. I concur
with this definition, since Python is for me closer to the developers
mind (and thus more distant from the machine) than any other language I
use (including Java, C++, C and ADA).
So, the bar has been raised, in a way.
And VHLLs will become more and more used as computer power grows and
becomes cheaper than developer time (it is already so in several
application areas).

 

[Roy Smith]

I'm glad you added that parenthetical comment, because I'm nearly
certain that it is *not* actual experience with Python, and
disappointment with its speed, that is the cause.

I suspect that most decisions to use or not use any given technology are
made long before any objective tests have been run. Perception is often
more important than measurement.

it's quite certain
that the mainstream *has* accepted high level languages quite
fully. The last time I checked, Java and C++ (even C) were
widely considered to be high level languages. Has someone been
raising the bar while I wasn't looking?

Yes, I think they have. I don't think a language that deals with memory
management at the level that C and C++ do (explicit allocation and
deallocation, pointers, etc) can be called high-level today. Even in
the 1970's, when C first appeared, it wasn't considered particularly
high level.

There's a lot that I don't like about Java too, but at least it doesn't
expose raw memory to the application logic, so I'll go along with Java
being high level. At least for now. It's a moving target, and if you
ask me again in 5 years, I'll probably give you a different answer.

 

[Ville Vainio]

Roy> (aka scripting) languages in general. I can think of no
Roy> reason why so much software is still written in languages
Roy> like C++ other than performance (or at least, the perception
Roy> of it).

Inertia, ignorance.

No need for virtual machine.

Real time requirements and predictabilify (e.g. fine grained control
of memory management).

Large amount of legacy code.

But most often it's still inertia and ignorance. Never underestimate
the power of incompetence ;-).