Python too slow?

[Jaimy Azle]

Wow, this is pretty misleading.

Java is, indeed, compiled to bytecode; however, modern JVMs typically
compile the bytecode to native code and then execute the native code.

CPython strictly interprets bytecode; it does not compile the
bytecode to native code.

By the existence of psyco which has equal functionality, i think he is right
in term of terminology.

Salam,

-Jaimy.

 

[Bruno Desthuilliers]

Ed Jensen a écrit :

If you have something substantive to add to the discussion, by all
means, do so. But please, keep your childish and insulting comments
to yourself.

I don't think you're going to make you some friends here insulting
Fredrik. I don't know who Ed Jensen is, but we are quite a lot here to
know and respect Mr Lundh for his contributions to Python as both a
language and a community.

 

[Bruno Desthuilliers]

George Sakkis a écrit :

Because these "some Java VMs" with JIT compilers are the de facto
standard used by millions;

Repeating an argument doesn't make it more true nor more relevant. Once
again, this doesn't change anything to the fact exposed above.

the spec is pretty much irrelevant

I mentionned this because this kind of choice is usually not part of the
language spec but of a specific implementation. Java is AFAIK the only
language where this implementation stuff is part of the spec.

(unless
you're a compiler writer or language theorist).

I thought it was quite clear and obvious that I was talking about points
relating to these fields.

 

[Bruno Desthuilliers]

Ross Ridge a écrit :

Ed Jensen a écrit :



Python's byte-code interpreter is not "just like" Java's virtual machine.

of course it's not "just like" - different languages, different
byte-codes, different implementations. What is "just like" is the
byte-code/VM scheme. Thought this was obvious to anyone able to parse a
simple sentence.

You're deliberately trying to mislead people into thinking Python performs
similarily to Java.

I don't know what you're smoking, but you should perhaps stop - because
this seems to drive you into paranoïd delirium.

 

[Bruno Desthuilliers]

Fredrik Lundh a écrit :

so you only run your programs once?

Lol !-)

 

[George Sakkis]

George Sakkis a écrit :



Repeating an argument doesn't make it more true nor more relevant. Once
again, this doesn't change anything to the fact exposed above.


I mentionned this because this kind of choice is usually not part of the
language spec but of a specific implementation. Java is AFAIK the only
language where this implementation stuff is part of the spec.


I thought it was quite clear and obvious that I was talking about points
relating to these fields.

No it wasn't, and besides the OP is most likely interested in these as
a simple user so the distinction between a spec and a de facto
standard implementation (such as JDK for Java and CPython for Python)
are almost pedantic if not misleading. We're not Lisp (yet ;-)), with
five major implementations and a dozen of minor ones.

George

 

[Ross Ridge]

And the reference implementation of Python (CPython) is not
interpreted, it's compiled to byte-code, which is then executed by a VM
(just like Java).

Ross Ridge a écrit :

Python's byte-code interpreter is not "just like" Java's virtual machine.

of course it's not "just like" - different languages, different
byte-codes, different implementations. What is "just like" is the
byte-code/VM scheme.

No the schemes aren't "just like" each other. They perform much
differently.

Thought this was obvious to anyone able to parse a simple sentence.

What's obvious is that you're lying.

I don't know what you're smoking, but you should perhaps stop - because
this seems to drive you into paranoïd delirium.

This isn't the first time you've tried to mislead people into
thinking Python's byte-code interpreter works just like Java's VM.
Your over-zealous Python advocacy is benefiting no one.

Ross Ridge

 

[Bruno Desthuilliers]

George Sakkis a écrit :

No it wasn't,
"""
> or is Python just too slow
> as an interpreted language

Being "interpreted" is a quality of an implementation, not of a language.
"""

If that isn't clear enough what I'm talking about, then sorry but I
can't help.

and besides the OP is most likely interested in these as
a simple user so the distinction between a spec and a de facto
standard implementation (such as JDK for Java and CPython for Python)
are almost pedantic if not misleading.

I can live with being called "pedantic" - even I'm not sure whether
correcting a wrong statement about CPython's execution model is pedantic
or not. But I *still* fail to see how it could be "misleading", and
*you* still fail to explain in which way it could be misleading.

If your point is that saying that CPython uses a byte-code/VM scheme
"just like Java" necessarily implies JIT compilation just because some
JVM support this feature, then it would be time you pay more attention
to what is effectively written.

We're not Lisp (yet ;-)), with
five major implementations and a dozen of minor ones.

And ? In which way does it make the distinction between a language and a
language implementation less true ?

 

[Bruno Desthuilliers]

Ross Ridge a écrit :

Ross Ridge a écrit :



No the schemes aren't "just like" each other. They perform much
differently.

fact 1: CPython compiles source code to byte-code.
fact 2: CPython executes this byte-code.
fact 3: Sun's JDK compiles source code to byte-code.
fact 4: Sun's JDK executes this byte-code.

Care to prove me wrong on any of these points ? Don't bother: you can't.
So my first assertion that "CPython is compiled to byte-code, which is
then executed by a VM" is true, and since the same assertion also stands
for Java (ie: sun's JDK), then the "just like" qualifier is true too.
Period.

What's obvious is that you're lying.

Man, you suffer from a bad case of paranoia.

This isn't the first time you've tried to mislead people into
thinking Python's byte-code interpreter works just like Java's VM.

I challenge you to come with any evidence to support this claim. I never
wrote anything about CPython's byte-code interpreter inner working.

Your over-zealous Python advocacy is benefiting no one.

Is there any cure for paranoia ?

 

[Ross Ridge]

fact 1: CPython compiles source code to byte-code.
fact 2: CPython executes this byte-code.
fact 3: Sun's JDK compiles source code to byte-code.
fact 4: Sun's JDK executes this byte-code.

Care to prove me wrong on any of these points ? Don't bother: you can't.
So my first assertion that "CPython is compiled to byte-code, which is
then executed by a VM" is true, and since the same assertion also stands
for Java (ie: sun's JDK), then the "just like" qualifier is true too.
Period.

No, the "just like" qualifier is false. Python doesn't compile "just
like" Java, nor does it execute "just like" Java. The "byte-code"
langauges are very different and they perform much differently.
Java compiles slower but executes faster. Python's byte-code is ment to
quickly generated on the fly to save having to reparse the source code.
Java code is compiled using optimizations into a virtual machine languague
ment to be executed as fast as possible on a wide range of processors.
The similarities between the two are superficial, Python doesn't compile
and execute code "just like" Java.

Try all you want to try to reparse what you wrote in to a different
meaning, it doesn't change the fact your intent was to mislead.

Ross Ridge

 

[Hrvoje Niksic]

fact 1: CPython compiles source code to byte-code.
fact 2: CPython executes this byte-code.
fact 3: Sun's JDK compiles source code to byte-code.
fact 4: Sun's JDK executes this byte-code.

Care to prove me wrong on any of these points ? Don't bother: you
can't.

Fact 4 is misleading because it is only one option available to Sun's
JDK. Sun's JDK is also capable of transforming the byte-code to
native code and letting the processor execute that instead of the
original byte code, and that is where the most significant speed
increase comes from. Most importantly, it does so automatically, by
default, with no programmer intervention or configuration, and with
100% compatibility, so it doesn't compare well to Python accelerators
like psyco.

 

[George Sakkis]

George Sakkis a écrit :





Being "interpreted" is a quality of an implementation, not of a language.
"""
If that isn't clear enough what I'm talking about, then sorry but I
can't help.

Pedantic once again. For languages with a single (or practically
single) implementation such as Python, the average user couldn't care
less about the distinction. Your point might have more merit if
PyPy or IronPython or Jython enter the same league with CPython in
terms of usage.

I can live with being called "pedantic" - even I'm not sure whether
correcting a wrong statement about CPython's execution model is pedantic
or not. But I *still* fail to see how it could be "misleading", and
*you* still fail to explain in which way it could be misleading.

If your point is that saying that CPython uses a byte-code/VM scheme
"just like Java" necessarily implies JIT compilation just because some
JVM support this feature, then it would be time you pay more attention
to what is effectively written.

What three different people in this thread have been trying to tell
you but you seem to miss is that claiming CPython's VM "is just like
Java" is comparable to saying "a Yugo's car engine is just like a
BMW's" (or "humans are just like chimpanzees"), which for some value
of "just like" is technically correct but it's not what most people
would call an accurate statement.

And ? In which way does it make the distinction between a language and a
language implementation less true ?

In the way that most plain users care (or not) about.

George

 

[George Sakkis]

Fact 4 is misleading because it is only one option available to Sun's
JDK. Sun's JDK is also capable of transforming the byte-code to
native code and letting the processor execute that instead of the
original byte code, and that is where the most significant speed
increase comes from. Most importantly, it does so automatically, by
default, with no programmer intervention or configuration, and with
100% compatibility, so it doesn't compare well to Python accelerators
like psyco.

Plus, IIRC Java's JIT is not limited to optimizing special cases,
while psyco helps primarily with number-crunching code (admittedly an
important special case) and can have zero or even (small) negative
effect on arbitrary Python programs.

George

 

[Chris Mellon]

Pedantic once again. For languages with a single (or practically
single) implementation such as Python, the average user couldn't care
less about the distinction. Your point might have more merit if
PyPy or IronPython or Jython enter the same league with CPython in
terms of usage.


What three different people in this thread have been trying to tell
you but you seem to miss is that claiming CPython's VM "is just like
Java" is comparable to saying "a Yugo's car engine is just like a
BMW's" (or "humans are just like chimpanzees"), which for some value
of "just like" is technically correct but it's not what most people
would call an accurate statement.

The statement was in response to a claim that Python was slow because
it is interpreted. This is a little like correcting someone who says
that a Yugo is slow because it has a steam engine by telling that no,
it's internal combustion, just like the BMW has.

It's possible for this a claim like this to lead to a clarifying and
informative discussion about JIT technology and how it improves Javas
performance, and the use of corresponding techniques in Python. What
we got instead was someone who felt some sort of juvenile urge to jump
all over a what he thought of as a claim that Python is as fast as
Java (which, of course, it sometimes is - the issue is more
complicated than a sound bite).

In the way that most plain users care (or not) about.

Not that I think any of you care about anything except your e-penis at
this point, but there is no reason to proscribe discussion to only
what "plain users" want, even if the OP was such a person.

 

[Ed Jensen]

fact 1: CPython compiles source code to byte-code.
fact 2: CPython executes this byte-code.
fact 3: Sun's JDK compiles source code to byte-code.
fact 4: Sun's JDK executes this byte-code.

Care to prove me wrong on any of these points ? Don't bother: you can't.
So my first assertion that "CPython is compiled to byte-code, which is
then executed by a VM" is true, and since the same assertion also stands
for Java (ie: sun's JDK), then the "just like" qualifier is true too.
Period.

#2 and #4 are wrong (or, at best, misleading). Here, I'll fix them
for you:

Fact 2: CPython interprets the bytecode.

Fact 4: Sun's JVM does some interpretation of the bytecode, but also
compiles some of the bytecode to native code and executes the
resulting native code.

These distinctions can be important and it's intellectually dishonest
to gloss over them.

 

[Ed Jensen]

I don't think you're going to make you some friends here insulting
Fredrik. I don't know who Ed Jensen is, but we are quite a lot here to
know and respect Mr Lundh for his contributions to Python as both a
language and a community.

I'll keep in mind that some people around these parts are untouchable
and have a free pass to be insulting and childish.

Thanks for the tip.

 

[sturlamolden]

Is there a better way to do color tracking, or is Python just too slow
as an interpreted language to do any effective color tracking?

You should code numerically intensive tasks using NumPy arrays. If
things are getting slow, chances are you are using Python for loops
instead of vectorized NumPy expressions. This is the same trick you
would use in e.g. Matlab for boosting performance. If things are
running slow despite of having properly vectorized your code, chances
are that porting to C will not make a big difference.

If you need to resort to C or Fortran, it is easy to interface Python
with these languages (e.g. ctypes, Pyrex, Swig, f2py, or weave). Put
the offending bottleneck in a lower level language (detect it using
the profiler module) and leave everything else in Python.

 

[Paul Boddie]

While your answer was technically correct, by omitting pertinent
information, your answer lead readers to the wrong conclusion.

Indeed. A perusal of the Java VM architecture documents plus any
discussions around dynamic language support (and there's a discussion
group about that very topic) reveal a number of differences between
the CPython VM and Sun's Java VM. Moreover, despite claims about the
Sun VM only being one of many, it holds a fairly similar position to
that of the CPython VM in its own part of the computing world, with
adoption only likely to increase now that it's open source. (And many
new adopters were quite possibly using stuff like gcj previously
which, as many will already know, is a native code compiler for Java.)

These days, I seriously doubt that anyone uses the term "interpreted"
to mean "parses the source text and works out what to do, over and
over again as the program runs". Instead, I imagine that the term
typically suggests the presence of a virtual processor doing what all
processors do: interpret instructions. It's just that when native code
is produced, we look away from the process of interpretation done by
the hardware and consider it as the plain "on the metal" execution of
the code. Sure, Sun's Java VM may not remove such a virtual processor
entirely from the overall execution of a program, and the output of
gcj may include the invocation of lots of moderately expensive library
calls, but in contrast with CPython, some sequences of instructions
will not ultimately be interpreted by the virtual machine at run-time.

All this said, there are options other than CPython, and I imagine
that they will become even more interesting to mainstream Python users
over the next months and years.

Paul

 

[sturlamolden]

You are correct that optimizing Python is hard. However, in many cases,
the problem is not that Python is too slow, but the specific algorithm
chosen by the programmer is slow, e.g. no compiler optimization is going
to turn an O(n**2) algorithm into an O(1) algorithm.

This is possibly the number one cause of 'slowness' complained about
on mailing lists.

The Original Poster says it takes one or two seconds to process an
800x600 GIF. That sounds believable: on my PC, it takes about five
seconds to loop over range(800*600) and do a tiny bit of processing.
Something like Psycho might speed that up a lot, possibly by an order of
magnitude or two.

It seems the code he is referring to is doing k-means clustering on
each frame. The clustering is done by from SciPy's cluster module,
which is doing 'vector quantitization'. The algorithm is written in
plain C. It is not Python that is slow. It is the amount of processing
done on each frame. As the bottleneck is already in C, it cannot be
done any faster without radically changing the algorithm and/or the
hardware.

 

[sturlamolden]

Is there a better way to do color tracking, or is Python just too slow
as an interpreted language to do any effective color tracking?

The slowness stems from the use of k-means clustering on each frame.
SciPy's clustering module used for the purpose is written in plain C.
It is not Python that is slow.