How much does Python optimize?

[Blackbird]

Hello!

I think

for i in range(10):
<some code that depends on i>


is more readable than a while loop with explicit incrementation of i.
However, range(10) in the command interpreter obviously returns a list. Is
this list optimized away in the code above, or is it actually constructed
internally? (With, say, CPython in one of the recent versions.)

 

[Felipe Almeida Lessa]

Em Sex, 2006-03-03 às 10:26 +0100, Blackbird escreveu:

However, range(10) in the command interpreter obviously returns a list. Is
this list optimized away in the code above, or is it actually constructed
internally? (With, say, CPython in one of the recent versions.)

It's constructed. That's why you should use xrange.

--
"Quem excele em empregar a força militar subjulga os exércitos dos
outros povos sem travar batalha, toma cidades fortificadas dos outros
povos sem as atacar e destrói os estados dos outros povos sem lutas
prolongadas. Deve lutar sob o Céu com o propósito primordial da
'preservação'. Desse modo suas armas não se embotarão, e os ganhos
poderão ser preservados. Essa é a estratégia para planejar ofensivas."

-- Sun Tzu, em "A arte da guerra"

 

[Scott David Daniels]

I think

for i in range(10):
<some code that depends on i>

is more readable than a while loop with explicit incrementation of i.

However, range(10) in the command interpreter obviously returns a list. Is
this list optimized away in the code above, or is it actually constructed
internally? (With, say, CPython in one of the recent versions.)

Yup, and if you are tuning a piece of code to the wall, you should time
it and possibly care. Likely you are not, and the timing makes no
difference. Someday, range will behave like xrange automagically,
and in the meantime your code will read just fine. If you are in
trouble now, your code reads much more like:
for i in range(100000):
...

The general rule is make the code clear, measure if its too slow,
and "don't worry, be happy (yagni)."

--Scott David Daniels
(e-mail address removed)

 

[Blackbird]

Yup, and if you are tuning a piece of code to the wall, you should
time it and possibly care. Likely you are not, and the timing makes
no difference. Someday, range will behave like xrange automagically,
and in the meantime your code will read just fine. If you are in
trouble now, your code reads much more like:
for i in range(100000):
...

Thanks. Yes, this will be a problem for iterations in the order of 10**5 and
upwards only, and those are rare in most applications. So my question was
more motivated by a general curiosity about the inner workings of the Python
interpreter, and I sort of understand why advanced optimization on an
instruction set like bytecode would be difficult. And in the mean time, I
found this insightful piece:

http://www.python.org/doc/essays/list2str.html

 

[Alex Martelli]

...
Thanks. Yes, this will be a problem for iterations in the order of 10**5 and
upwards only, and those are rare in most applications. So my question was

And not crucial when they happen, mostly:

helen:~ alex$ python -mtimeit -s'x=10**5' 'for i in range(x): j=i*i'
10 loops, best of 3: 102 msec per loop

helen:~ alex$ python -mtimeit -s'x=10**5' 'for i in xrange(x): j=i*i'
10 loops, best of 3: 80.9 msec per loop

a 20% difference (for a very lightweight loop body), while measurable,
will matter at all only in the hottest of hotspots, the narrowest of
bottlenecks. Anyway, do learn about -mtimeit, it's cool indeed for such
performance measurements.

more motivated by a general curiosity about the inner workings of the Python
interpreter, and I sort of understand why advanced optimization on an
instruction set like bytecode would be difficult. And in the mean time, I

Python deliberately avoids all sorts of optimizations, including obvious
ones such as constant hoisting -- the idea being that when you need
something hoisted, you can hoist it yourself, and meanwhile by knowing
that the compiler is dirt-simple and literal-minded you avoid all kinds
of risks connected to optimizer bugs or misunderstandings thereof.


Alex