Python too slow?

S

Steven D'Aprano

It seems the code he is referring to is doing k-means clustering on each
frame. The clustering is done by from SciPy's cluster module, which is
doing 'vector quantitization'. The algorithm is written in plain C. It
is not Python that is slow. It is the amount of processing done on each
frame. As the bottleneck is already in C, it cannot be done any faster
without radically changing the algorithm and/or the hardware.

Thank you for taking the effort to actually investigate the OP's *actual*
problem.

Such a pity that the chances are he's probably gone away in disgust at
the pedantry and insults in this thread. He'll probably become another
one of those people convinced that Python is "slow" because it's
interpreted when in fact this is an example of C being slow in spite of
being compiled.
 
S

Steven D'Aprano

These days, I seriously doubt that anyone uses the term "interpreted" to
mean "parses the source text and works out what to do, over and over
again as the program runs".

Given the way that people seem to use "interpreted" as a pejorative and a
synonym for "slow", I don't doubt it one bit. Especially in management,
where they might be making technical judgments on the basis of half-
remembered Comp Sci 101 lessons from fifteen years earlier and vague
buzzword-laden articles in trade magazines.
 
B

bruno.desthuilliers

On Jan 11, 8:59 am, Bruno Desthuilliers <bruno. (snip)


What three different people in this thread have been trying to tell
you but you seem to miss is that claiming CPython's VM "is just like
Java"

George, *please*, re-read what I wrote. I *never* claimed that
CPython's VM is just like a JVM, I just stated that both CPython and
Java use a byte-code/VM execution model. Can't you understand that ?
 
P

Paul Boddie

Given the way that people seem to use "interpreted" as a pejorative and a
synonym for "slow", I don't doubt it one bit. Especially in management,
where they might be making technical judgments on the basis of half-
remembered Comp Sci 101 lessons from fifteen years earlier and vague
buzzword-laden articles in trade magazines.

Indeed. Still, there's reason to be upbeat about Python's potential
here. The big question is this: what is everyone with any degree of
concern about Python's performance doing to improve the situation?
Sure, C (or actually C++) seems to win the shootout [1], but there are
plenty of language implementations between g++ and CPython to suggest
that the old choice of extension modules written in C vs. other code
written in Python doesn't provide a complete map of the opportunities.

Paul

[1] http://shootout.alioth.debian.org/gp4sandbox/benchmark.php?test=all&lang=all
 
B

bruno.desthuilliers

Fact 4 is misleading because it is only one option available to Sun's
JDK. Sun's JDK is also capable of transforming the byte-code to
native code and letting the processor execute that instead of the
original byte code, and that is where the most significant speed
increase comes from. Most importantly, it does so automatically, by
default, with no programmer intervention or configuration, and with
100% compatibility, so it doesn't compare well to Python accelerators
like psyco.

Then fact 1 is misleading too since Python handles the compilation
automatically without programmer's intervention while Java requires
someone to explicitely invoke the byte-code compiler.

I just don't understand what's all that fuss with this simple and
quite comparison of Java and Python. You can google this ng archives,
you'll find hundreds of posts saying the same thing, and everyone so
far seemed to be smart enough to understand that this had nothing to
do with the byte-code specifications, VM implementation and presence
or absence of a JIT compiler.

Anyway, I do maintain that what I said is 100% correct, 100% accurate
given the context, and 0% misleading unless you're clueless enough to
not be able to parse a single english sentence (in which case I just
can't help). As a matter of fact, this didn't seem to "mislead" the OP
into thinking such a thing.

Regards.
 
B

bruno.desthuilliers

#2 and #4 are wrong (or, at best, misleading). Here, I'll fix them
for you:

Fact 2: CPython interprets the bytecode.

Fact 4: Sun's JVM does some interpretation of the bytecode, but also
compiles some of the bytecode to native code and executes the
resulting native code.

These distinctions can be important and it's intellectually dishonest
to gloss over them.

These distinctions are important if the context is the VM
implementation, which is *obviously* not the case here. FWIW, I you
had spent a bit more time reading the remaining of the discussion with
the OP, I do clearly mention this distinction. So please get a life
and keep your comment about "intellectual dishonesty" where they
belong.
 
B

bruno.desthuilliers

While your answer was technically correct, by omitting pertinent
information, your answer lead readers to the wrong conclusion.

That's pure non-sense.
The only question that remains is if you were being accidentally
misleading or purposefully misleading.

The only question that remains for me is if you are being just stupid
or totally paranoid.

fu2 /dev/null
 
D

dongie.agnir

Oh.. it seems my naiveness has stirred quite a discussion >_<...

I must admit though, I've learned a bit about Python reading through
this topic. Thanks to everyone who pointed out the flaws in the
code. I'll see if I can come up with my own color tracking solution
in a few weeks and post back here. Thanks!
 
J

Jaimy Azle

Then fact 1 is misleading too since Python handles the compilation
automatically without programmer's intervention while Java requires
someone to explicitely invoke the byte-code compiler.

Sadly it is true also, I read somewhere this silly point was used also to
make distinction between java and python, then claiming python is just like
another interpreter, while java has it's own (what they call as) 'compiler'.
:)

perhaps in the future another sillly point could be added also, Java has
Jython, while Python doesn't have some thing like PyJava or... perhaps Py-va
(Python based Java Language).

Salam,

-Jaimy.
 
B

bearophileHUGS

Paul Boddie:
what is everyone with any degree of
concern about Python's performance doing to improve the situation?

They are probably developing systems like Cython and ShedSkin, and
hoping to see Psyco improved again to manage itertools better (and
maybe 64 bit CPUs too).

Sure, C (or actually C++) seems to win the shootout [1],

Beside C++ being the faster (not C anymore), there are other ways to
"win" the Shootout, like using less RAM, writing shorter code, etc. If
you change the way you measure you can see that FreePascal, Ruby and D
too "win" the Shootout :) (the first as the one using less memory,
the second one as the one with shorter programs, and the third one as
"faster && with shorter programs").

but there are
plenty of language implementations between g++ and CPython to suggest
that the old choice of extension modules written in C vs. other code
written in Python doesn't provide a complete map of the opportunities.

I think in few years it may become feasible to use Pyd to write
extensions in D for CPython :)

Bye,
bearophile
 
B

Bruno Desthuilliers

Jaimy Azle a écrit :
Sadly it is true also, I read somewhere this silly point was used also to
make distinction between java and python, then claiming python is just like
another interpreter, while java has it's own (what they call as) 'compiler'.
:)

perhaps in the future another sillly point could be added also, Java has
Jython, while Python doesn't have some thing like PyJava or... perhaps Py-va
(Python based Java Language).

Lol. At least some common sens in this stupid thread. Thanks Jaimy, you
made my day !-)

Peace.

PS : and BTW : my apologies to the community - I should have made my
point only once and then shut up. Sorry.
 
C

cokofreedom

A lecturer gave me the perfect answer to the question of speed.

"You have two choices when it comes to programming. Fast code, or fast
coders."
 
P

Paul Boddie

perhaps in the future another sillly point could be added also, Java has
Jython, while Python doesn't have some thing like PyJava or... perhaps Py-va
(Python based Java Language).

You could compile Java to CPython bytecode or, in the case of a little
experiment I did some time ago, translate Java bytecode to CPython
bytecode: the CPython virtual machine operates at a higher level and
can support the Java instructions fairly easily, whereas a fair amount
of work is required to support CPython instructions on the Java
virtual machine. I found that the biggest obstacle was probably
treating Java packages like Python packages - something which
admittedly isn't completely necessary, but which would make the thing
more usable at the prompt. Ultimately, I had little need for Java-
based software and thus wasn't motivated into continuing the work,
although Python 2.5 and beyond do provide some conveniences which
might make some aspects of the implementation more bearable.

Given that other languages (eg. Logix [2], various Lisp dialects) have
been implemented for CPython, I think your point could be better
formulated.

Paul

[1] http://www.boddie.org.uk/python/javaclass.html
[2] http://www.livelogix.net/logix/
 
P

Paul Rudin

A lecturer gave me the perfect answer to the question of speed.

"You have two choices when it comes to programming. Fast code, or fast
coders."

Yes, although it's more a continuum than that suggests. The tricky bit
is deciding in each situation where you should be on the continuum.
 
C

cokofreedom

Yes, although it's more a continuum than that suggests. The tricky bit
is deciding in each situation where you should be on the continuum.

Ah, it all comes from hindsight! You always realise in the last week
of the deadline you were on the wrong end of the continuum! Problem
solved, panic created! :)
 
B

bearophileHUGS

(e-mail address removed):
A lecturer gave me the perfect answer to the question of speed.
"You have two choices when it comes to programming. Fast code, or fast
coders."

I don't believe that anymore, ShedSkin compiles slowly and it has
limitations still, but it shows that it's possible to create a
language with simple short syntax and high running speed at the same
time.

Bye,
bearophile
 
J

Jaimy Azle

Paul Boddie said:
You could compile Java to CPython bytecode or, in the case of a little
experiment I did some time ago, translate Java bytecode to CPython
bytecode: the CPython virtual machine operates at a higher level and
can support the Java instructions fairly easily, whereas a fair amount
of work is required to support CPython instructions on the Java
virtual machine. I found that the biggest obstacle was probably
treating Java packages like Python packages - something which
admittedly isn't completely necessary, but which would make the thing
more usable at the prompt. Ultimately, I had little need for Java-
based software and thus wasn't motivated into continuing the work,
although Python 2.5 and beyond do provide some conveniences which
might make some aspects of the implementation more bearable.

Wow, serious... what you've done was really, really cool... :)

I was expect there are nobody willing to do to have python runs Java
Language (such as PyPy) over CPython. Perhaps your javaclass does not work
just like as PyPy, but really... it is damned cool to get CPython execute
java byte-code, congratulations...

Salam,

-Jaimy.
 
P

Paul Boddie

Wow, serious... what you've done was really, really cool... :)

In practice, not that cool. ;-)
I was expect there are nobody willing to do to have python runs Java
Language (such as PyPy) over CPython. Perhaps your javaclass does not work
just like as PyPy, but really... it is damned cool to get CPython execute
java byte-code, congratulations...

Well, the limitations that stopped me working on it are listed on the
page I referenced, so it wasn't that cool in the end. In fact, the
project grew out of just wanting to inspect .class files and pull out
method signatures, but it was so tempting to see whether Java
bytecodes could be rewritten and run in a CPython environment.

I think the benefits of running Java on CPython are significantly less
than those had by running Python on the Java VM (or another VM).
Firstly, who wants to write statically typed code which then runs on a
virtual machine that can't take advantage of the type declarations?
Secondly, isn't it just better to use a virtual machine with just-in-
time compilation and all sorts of security mechanisms if you're
wanting to write the Java code that, when compiled, can take advantage
of all that stuff? In other words: what makes CPython a compelling VM
for the Java programmer?

My perspective now is that it's a lot more interesting to target
Python for virtual machines other than the CPython one because that's
where the performance and functionality benefits are most likely to be
found. And the most important motivation for this: I prefer writing
Python, not Java. ;-)

Paul
 
E

Ed Jensen

A lecturer gave me the perfect answer to the question of speed.

"You have two choices when it comes to programming. Fast code, or fast
coders."

"You're either with us, or against us."

George W. Bush




My understanding is that while CPython performance won't be winning
any awards anytime soon, Jython and IronPython are pretty impressive
performers.

Disclaimer: I haven't personally used Jython or IronPython.
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

No members online now.

Forum statistics

Threads
473,774
Messages
2,569,598
Members
45,149
Latest member
Vinay Kumar Nevatia0
Top