Python too slow?

[dongie.agnir]

I'm pretty new to Python, and even newer to Image/Video processing,
and trying to get started on a project similar to GRL Vienna's laser
marker. I found some sample code here http://janto.blogspot.com/2006/01/motion-capture-in-python.html,
but after running the code with the included sample input file, it
seems quite slow (1-2 seconds from start to finish to do the 800 by
600 gif image).

Is there a better way to do color tracking, or is Python just too slow
as an interpreted language to do any effective color tracking?

 

[Bruno Desthuilliers]

(e-mail address removed) a écrit :

I'm pretty new to Python, and even newer to Image/Video processing,
and trying to get started on a project similar to GRL Vienna's laser
marker. I found some sample code here http://janto.blogspot.com/2006/01/motion-capture-in-python.html,
but after running the code with the included sample input file, it
seems quite slow (1-2 seconds from start to finish to do the 800 by
600 gif image).

Is there a better way to do color tracking,

Not having any experience with this domain, I can't comment on this.

or is Python just too slow
as an interpreted language

Being "interpreted" is a quality of an implementation, not of a
language. And the reference implementation of Python (CPython) is not
interpreted, it's compiled to byte-code, which is then executed by a VM
(just like Java). So while CPython may possibly be too slow for your
application (it can indeed be somewhat slow for some tasks), the reasons
are elsewhere (hint: how can a compiler safely optimize anything in a
language so dynamic that even the class of an object can be changed at
runtime ?) ...

 

[dongie.agnir]

Thanks for the clarification.

Though I was hoping someone could give me a definitive answer. I was
quite excited about this project initially, but seeing the actual
execute times was a big downer. Seeing that the original laser marker
by GRL Vienna is done in Processing which from what I know is built on
Java, I was hoping that Python would be up to a similar task.

 

[Christian Heimes]

I'm pretty new to Python, and even newer to Image/Video processing,
and trying to get started on a project similar to GRL Vienna's laser
marker. I found some sample code here http://janto.blogspot.com/2006/01/motion-capture-in-python.html,
but after running the code with the included sample input file, it
seems quite slow (1-2 seconds from start to finish to do the 800 by
600 gif image).

Have you profiled your application? Do you know the bottle necks and the
reason for the bottle necks? Is your application IO, memory or CPU
bound? Have you considered rewriting the bottle necks in C?

Christian

 

[Ben Finney]

[...] after running the code with the included sample input file,
it seems quite slow (1-2 seconds from start to finish to do the
800 by 600 gif image).

Have you profiled your application? Do you know the bottle necks and
the reason for the bottle necks? Is your application IO, memory or
CPU bound? Have you considered rewriting the bottle necks in C?

All good questions. Another important one: Is the startup time of the
Python process (i.e. before executing any of the actual statements in
the program) a significant part of the total time in this case?

 

[bruno.desthuilliers]

Thanks for the clarification.

Though I was hoping someone could give me a definitive answer.

Sorry - but I'm definitively not the right person on this...

I was
quite excited about this project initially, but seeing the actual
execute times was a big downer. Seeing that the original laser marker
by GRL Vienna is done in Processing which from what I know is built on
Java, I was hoping that Python would be up to a similar task.

Java's declarative static typing allow agressive just-in-time
optimizations - which is not the case in Python due to it's higly
dynamic nature[1]. You may want to have a look at psyco (a JIT
compiler) or to look for other angles (if there's any - IIRC the
example you're talking about already uses at least PIL for image
processing and something like scipy or numpy for intensive math ops).

If none of the two above answers fits your needs, and no Python Guru
comes with a better answer, then I'm afraid you'll have to go for a
language with a faster implementation. Python is quite faster nowadays
than it used to be, and is usually fast enough for most day-to-day
programming tasks (a, but it's still not as highly optimized as some
Lisp implementations (to compare to another highly





[1] you can read more on this on the pypy website, and specially here
IIRC: http://codespeak.net/pypy/dist/pypy/doc/dynamic-language-translation.html)

 

[Ed Jensen]

And the reference implementation of Python (CPython) is not
interpreted, it's compiled to byte-code, which is then executed by a VM
(just like Java).

Wow, this is pretty misleading.

Java is, indeed, compiled to bytecode; however, modern JVMs typically
compile the bytecode to native code and then execute the native code.

CPython strictly interprets bytecode; it does not compile the
bytecode to native code.

 

[Steven D'Aprano]

hint: how can a compiler safely optimize anything in a language so
dynamic that even the class of an object can be changed at runtime ?

Is that a trick question?

By finding things to optimize that *can't* change at runtime.

E.g. a keyhole optimizer that detects when you write something like this:

def foo():
x = 1 + 2
return x

and optimizes it to this:
2 0 LOAD_CONST 3 (3)
3 STORE_FAST 0 (x)

3 6 LOAD_FAST 0 (x)
9 RETURN_VALUE

(Only in CPython version 2.5 or better.)

Okay, okay, so constant folding isn't going to save you a lot of time
(unless your constant is being re-calculated millions of times) but it's
still an optimization. There are others, some have been done (e.g.
optimize away the quadratic behaviour for string concatenation in some
limited cases), some are possible but rejected by the BDFL (e.g. tail-
call optimization), presumably there are some possible but not yet
implemented or even discovered (which is the point of the PyPy project).

You are correct that optimizing Python is hard. However, in many cases,
the problem is not that Python is too slow, but the specific algorithm
chosen by the programmer is slow, e.g. no compiler optimization is going
to turn an O(n**2) algorithm into an O(1) algorithm.

The Original Poster says it takes one or two seconds to process an
800x600 GIF. That sounds believable: on my PC, it takes about five
seconds to loop over range(800*600) and do a tiny bit of processing.
Something like Psycho might speed that up a lot, possibly by an order of
magnitude or two.

 

[Bruno Desthuilliers]

Ed Jensen a écrit :

Wow, this is pretty misleading.

Ho yes ??? Why so, please ? Care to point to anything *wrong* in the
above statement ?

Java is, indeed, compiled to bytecode; however, modern JVMs typically
compile the bytecode to native code and then execute the native code.

Which is known as JIT compilation - and there are a couple attempts at
it in Python too.

Anyway, the JIT compiler is not part of the Java spec (while the
byte-code/VM is), and its not garanteed to be there on each an every
Java VM.

CPython strictly interprets bytecode;

And ?

it does not compile the
bytecode to native code.

And ?

I fail to see how the existence of JIT compilers in some Java VM changes
anything to the fact that both Java (by language specification) and
CPython use the byte-code/VM scheme.

 

[A.T.Hofkamp]

I'm pretty new to Python, and even newer to Image/Video processing,
and trying to get started on a project similar to GRL Vienna's laser
marker. I found some sample code here http://janto.blogspot.com/2006/01/motion-capture-in-python.html,
but after running the code with the included sample input file, it
seems quite slow (1-2 seconds from start to finish to do the 800 by
600 gif image).

Is there a better way to do color tracking, or is Python just too slow
as an interpreted language to do any effective color tracking?

People seem quite obsessed with execution speed of CPU's.

I have come to believe that it is only one part of the equation, development
time and ease of maintenance or modification is another one, which I believe is
much more interesting to perform as fast as possible.

1-2 seconds may be slower than some alternative solution (you could in
principle develop custom hardware for it, and do the trick in a few
nano-seconds). The question that arises (for me at least) is, how fast can you
write, maintain, and modify what you want in that other solution.

Suppose you can do color-tracking in 0.5 seconds. So, you gained 1.5
seconds CPU time.
Now the question you need to answer for yourself, is how much more worth is
your own time compared to the gain in CPU time. If you think they are equal (ie
the problem as a whole should be solved as fast as possible, thus the sum of
development time + execution time should be as short as possible), you can
spend an additional 1.5 seconds development in the alternative solution.

Of course, if you do more color-tracking, the differences between alternative
solutions become larger, and your custom hardware solution may become a
feasible alternative....

In my experience however, differences in CPU execution time are usually
meaningless compared to differences in development time.


Albert

 

[Jeroen Ruigrok van der Werven]

In my experience however, differences in CPU execution time are usually
meaningless compared to differences in development time.

I have to disagree with you to a point.

Yes, maintenance of code is important, no denying that. However, if I can
calculate N variations of a certain material's properties in an hour and using
Python will cut that in half, the number of calculated variations per hour, I
will be concerned.

This is especially so for people designing, say, optics. If you look at the
amount of optimizing designs and calculating the resulting properties and
doing this for X iterations in a day it becomes quite painfully obvious that
raw CPU execution time *does* matter.

't All varies with what you want, of course...

--
Jeroen Ruigrok van der Werven <asmodai(-at-)in-nomine.org> / asmodai
イェルーン ラウフロック ヴァン デル ウェルヴェン
http://www.in-nomine.org/ | http://www.rangaku.org/
When you have eliminated the impossible, whatever remains, however
improbable, must be the truth...

 

[dongie.agnir]

For the most part, I agree with you, which is why I chose Python in
the first place. I like quick development. Unfortunately, this is
one of those cases where execution time is a factor. Who knows, maybe
someone who's done camera vision with Python will come in and say it's
just the algorithm in the example that needs work (I'm crossing my
fingers that that's the case).

 

[Ant]

For the most part, I agree with you, which is why I chose Python in
the first place. I like quick development. Unfortunately, this is
one of those cases where execution time is a factor. Who knows, maybe
someone who's done camera vision with Python will come in and say it's
just the algorithm in the example that needs work (I'm crossing my
fingers that that's the case).

It would probably be worth reposting your question with a different
subject line. Something along the lines of "Image/Video processing
performance in Python", and explaining what you actually want to
achieve. "Python too slow?" doesn't give enough information - clearly
Python is fast enough for many many tasks, since it is so widely used.
It also sounds like a troll - so many people who would actually be
able to assist you (such as the PIL, pygame and numpy/scipy
communities) may have ignored this thread.

Cheers,

 

[dongie.agnir]

It would probably be worth reposting your question with a different
subject line. Something along the lines of "Image/Video processing
performance in Python", and explaining what you actually want to
achieve. "Python too slow?" doesn't give enough information - clearly
Python is fast enough for many many tasks, since it is so widely used.
It also sounds like a troll - so many people who would actually be
able to assist you (such as the PIL, pygame and numpy/scipy
communities) may have ignored this thread.

Cheers,

Thanks for the advice. I'll do that now.

 

[Fredrik Lundh]

Now the question you need to answer for yourself, is how much more worth is
your own time compared to the gain in CPU time. If you think they are equal (ie
the problem as a whole should be solved as fast as possible, thus the sum of
development time + execution time should be as short as possible), you can
spend an additional 1.5 seconds development in the alternative solution.

so you only run your programs once?

</F>

 

[George Sakkis]

I fail to see how the existence of JIT compilers in some Java VM changes
anything to the fact that both Java (by language specification) and
CPython use the byte-code/VM scheme.

Because these "some Java VMs" with JIT compilers are the de facto
standard used by millions; the spec is pretty much irrelevant (unless
you're a compiler writer or language theorist).

George

 

[Ross Ridge]

And the reference implementation of Python (CPython) is not
interpreted, it's compiled to byte-code, which is then executed by a VM
(just like Java).

Ed Jensen a écrit :

Wow, this is pretty misleading.

Ho yes ??? Why so, please ? Care to point to anything *wrong* in the
above statement ?

Python's byte-code interpreter is not "just like" Java's virtual machine.
You're deliberately trying to mislead people into thinking Python performs
similarily to Java.

Ross Ridge

 

[Ed Jensen]

I fail to see how the existence of JIT compilers in some Java VM changes
anything to the fact that both Java (by language specification) and
CPython use the byte-code/VM scheme.

While your answer was technically correct, by omitting pertinent
information, your answer lead readers to the wrong conclusion.

The only question that remains is if you were being accidentally
misleading or purposefully misleading.

 

[Fredrik Lundh]

The only question that remains is if you were being accidentally
misleading or purposefully misleading.

oh, please. it was perfectly clear for anyone with the slightest clue
what Bruno was talking about (especially if they'd read the post he was
replying to), so the only question that remains is why you didn't
understand it.

</F>

 

[Ed Jensen]

oh, please. it was perfectly clear for anyone with the slightest clue
what Bruno was talking about (especially if they'd read the post he was
replying to), so the only question that remains is why you didn't
understand it.

If you have something substantive to add to the discussion, by all
means, do so. But please, keep your childish and insulting comments
to yourself.