why I don't like range/xrange

S

stdazi

Hello!

Many times I was suggested to use xrange and range instead of the
while constructs, and indeed, they are quite more elegant - but, after
calculating the overhead (and losen flexibility) when working with
range/xrange, and while loops, you get to the conclusion that it isn't
really worth using range/xrange loops.

I'd like to show some examples and I'll be glad if someone can suggest
some other fixes than while a loop :)

a) range overfllow :


for i in range(0, 1 << len(S)) :
......
OverflowError: range() result has too many items

ok, so we fix this one with xrange !

b) xrange long int overflow :

for i in xrange(0, 1 << len(S)) :
.........
OverflowError: long int too large to convert to int

Next thing I miss is the flexibility as in C for loops :

for (i = 0; some_function() /* or other condition */ ; i++)

or,

for (i = 0 ; i < 10 ; i++)
i = 10;


I don't think range/xrange sucks, but I really think there should be
some other constructs to improve the looping flexibility. Other thing
may be, that I just miss an equally elegant alternative that's why I'd
like to hear some suggestions on how to fix the above issues.. (btw,
I've already browsed the archives related to my issue,but i don't see
any good solution)

Thanks

Jernej.
 
C

Chris Mellon

Hello!

Many times I was suggested to use xrange and range instead of the
while constructs, and indeed, they are quite more elegant - but, after
calculating the overhead (and losen flexibility) when working with
range/xrange, and while loops, you get to the conclusion that it isn't
really worth using range/xrange loops.

I'd like to show some examples and I'll be glad if someone can suggest
some other fixes than while a loop :)

a) range overfllow :


for i in range(0, 1 << len(S)) :
.....
OverflowError: range() result has too many items

ok, so we fix this one with xrange !

b) xrange long int overflow :

for i in xrange(0, 1 << len(S)) :
........
OverflowError: long int too large to convert to int

xrange should be able to handle this. It's probably worth a bug report.
Next thing I miss is the flexibility as in C for loops :

for (i = 0; some_function() /* or other condition */ ; i++)

You'd use a regular for or a while loop here, without the loop index.
or,

for (i = 0 ; i < 10 ; i++)
i = 10;

This doesn't do anything as written. For all reasonable uses of it,
you can do it the same way in Python. Writing C code in Python is a
waste of time and an exercise in frustration.
I don't think range/xrange sucks, but I really think there should be
some other constructs to improve the looping flexibility. Other thing
may be, that I just miss an equally elegant alternative that's why I'd
like to hear some suggestions on how to fix the above issues.. (btw,
I've already browsed the archives related to my issue,but i don't see
any good solution)

Enumeration over range() and xrange() is rare in Python. loops like:

data = [...]
for ii in xrange(len(data)):
datum = data[ii]

are a wart. Enumerate over whatever you're going to work with.

In some cases, you want the index and the item. Use enumerate for that:

for index, datum in enumerate(data):
....
 
P

Paul McGuire

Hello!

Many times I was suggested to use xrange and range instead of the
while constructs, and indeed, they are quite more elegant - but, after
calculating the overhead (and losen flexibility) when working with
range/xrange, and while loops, you get to the conclusion that it isn't
really worth using range/xrange loops.

I'd like to show some examples and I'll be glad if someone can suggest
some other fixes than while a loop :)

a) range overfllow :

for i in range(0, 1 << len(S)) :
.....
OverflowError: range() result has too many items

ok, so we fix this one with xrange !

b) xrange long int overflow :

for i in xrange(0, 1 << len(S)) :
........
OverflowError: long int too large to convert to int

Next thing I miss is the flexibility as in C for loops :

for (i = 0; some_function() /* or other condition */ ; i++)

or,

for (i = 0 ; i < 10 ; i++)
i = 10;

I don't think range/xrange sucks, but I really think there should be
some other constructs to improve the looping flexibility. Other thing
may be, that I just miss an equally elegant alternative that's why I'd
like to hear some suggestions on how to fix the above issues.. (btw,
I've already browsed the archives related to my issue,but i don't see
any good solution)

Thanks

Jernej.

Very little of my own code uses range or xrange, most of my for loops
iterate over a sequence or generator, as in "for item in blahList: do
something with item". I think range/xrange are common beginner's
constructs, since they reflect a more C-like looping method ("for i in
range(len(blahList)): do something with blahList").

I really would *not* encourage use of range/xrange, but feel that
iteration over sequences and generators is the more elegant/Pythonic
way to go - who is suggesting you use range/xrange?

For that matter, just what is it you plan to do 2**len(S) times? If S
is of any significant length, the sun may be a lump of coal before you
are finished, regardless of what loop mechanism you use (although it
would make sense to avoid range's implementation of creating a list of
2**len(S) items - fortunately the implementation already resolves this
problem by raising a "that's too many items" exception, and thankfully
so).

Maybe instead of working around range/xrange, you should think whether
a 2**len(S) approach to your problem is feasible in the first place.

-- Paul
 
L

Larry Bates

stdazi said:
Hello!

Many times I was suggested to use xrange and range instead of the
while constructs, and indeed, they are quite more elegant - but, after
calculating the overhead (and losen flexibility) when working with
range/xrange, and while loops, you get to the conclusion that it isn't
really worth using range/xrange loops.

I'd like to show some examples and I'll be glad if someone can suggest
some other fixes than while a loop :)

a) range overfllow :


for i in range(0, 1 << len(S)) :
.....
OverflowError: range() result has too many items

ok, so we fix this one with xrange !

b) xrange long int overflow :

for i in xrange(0, 1 << len(S)) :
........
OverflowError: long int too large to convert to int

Next thing I miss is the flexibility as in C for loops :

for (i = 0; some_function() /* or other condition */ ; i++)

or,

for (i = 0 ; i < 10 ; i++)
i = 10;


I don't think range/xrange sucks, but I really think there should be
some other constructs to improve the looping flexibility. Other thing
may be, that I just miss an equally elegant alternative that's why I'd
like to hear some suggestions on how to fix the above issues.. (btw,
I've already browsed the archives related to my issue,but i don't see
any good solution)

Thanks

Jernej.

Your example of for i in xrange(0, 1<<len(s)): must have resulted in
a number greater than 1 billion. Are you really doing this much or
are you just pointing out an edge case here?

You can always use while loop:

i=0
while i < (1<<len(s)):
i+=1


or

i=1
while 1:
if i == 10: break
i+=1

Personally I use a lot of for loops in my code where I iterate
over lists or with a generator as my target that returns instances
of my data until exhausted and find the code easy to read:

for line in myfile:
# do something with the line

No need to even "think" about numbers here unless I want to keep
track of the line numbers in which case I would do:

for linenumber, line in enumerate(myfile):
# do something with each linenumber, line

I find that I use i, j, k pointers less and less as I write more code in
Python. I guess it is just what you get accustomed to using.

-Larry
 
S

Steven D'Aprano

Hello!

Many times I was suggested to use xrange and range instead of the
while constructs, and indeed, they are quite more elegant - but, after
calculating the overhead (and losen flexibility) when working with
range/xrange, and while loops, you get to the conclusion that it isn't
really worth using range/xrange loops.

I prefer to _measure_ the overhead instead of guessing.

import timeit
whileloop = """i = 0
while i < N:
i += 1
pass
"""
forloop = """for i in xrange(N):
pass
"""

Now let's see how fast the loops are.
timeit.Timer(whileloop, "N = 10000").repeat(3, 1000) [3.5716907978057861, 3.5263650417327881, 3.5975079536437988]
timeit.Timer(forloop, "N = 10000").repeat(3, 1000)
[1.3608510494232178, 1.341961145401001, 1.3180010318756104]

Looks to me that a for loop using xrange is more than twice as fast as a
while loop. The advantage is about the same for small N:
timeit.Timer(whileloop, "N = 100").repeat(3, 1000) [0.052264213562011719, 0.049374103546142578, 0.041945934295654297]
timeit.Timer(forloop, "N = 100").repeat(3, 1000)
[0.012259006500244141, 0.013512134552001953, 0.015196800231933594]


What makes you think that a while loop has less overhead?


I'd like to show some examples and I'll be glad if someone can suggest
some other fixes than while a loop :)

a) range overfllow :


for i in range(0, 1 << len(S)) :
.....
OverflowError: range() result has too many items

ok, so we fix this one with xrange !


By the way, you don't need to write range(0, N). You can just write
range(N).

Yes, you're correct, range(some_enormous_number) will fail if
some_enormous_number is too big.


b) xrange long int overflow :

for i in xrange(0, 1 << len(S)) :
........
OverflowError: long int too large to convert to int


Are you really doing something at least 2147483647 times? I'm guessing
that you would be better off rethinking your algorithm.


Next thing I miss is the flexibility as in C for loops :

for (i = 0; some_function() /* or other condition */ ; i++)

This would be written in Python as:

i = 0
while some_function():
i += 1

A more flexible way would be to re-write some_function() as an iterator,
then use it directly:

for item in some_function():
# do something with item


or,

for (i = 0 ; i < 10 ; i++)
i = 10;


This would be written in Python as:

for i in xrange(10):
i = 10


I don't think range/xrange sucks, but I really think there should be
some other constructs to improve the looping flexibility.

Every loop can be turned into a while loop. If you have while, you don't
_need_ anything else.

But for elegance and ease of use, a small number of looping constructs is
good. Python has:

while condition:
block
else:
# runs if while exits *without* hitting break statement
block


for item in any_sequence:
block
else:
# runs if for exits *without* hitting break statement
block


Nice, clean syntax.

any_sequence isn't limited to mere arithmetic sequences -- it can be any
sequence of any objects. But if you need a C-style for loop, using
integers, range/xrange([start, ] stop [, step]) is provided.
 
B

Bart Ogryczak

for (i = 0; some_function() /* or other condition */ ; i++)

C's "for(pre,cond,post) code" is nothing more, then shorthand form of
"pre; while(cond) {code; post;}"
Which translated to Python would be:

pre
while cond:
code
post
 
M

Marc 'BlackJack' Rintsch

Steven said:
This would be written in Python as:

for i in xrange(10):
i = 10

Nope, in Python it's:

for i in xrange(10):
break

I think his example should demonstrate that assigning to the loop variable
has no effect on the "loop test". Which IMHO is a good thing.

Ciao,
Marc 'BlackJack' Rintsch
 
B

Bjoern Schliessmann

stdazi said:
Many times I was suggested to use xrange and range instead of the
while constructs, and indeed, they are quite more elegant - but,
after calculating the overhead (and losen flexibility) when
working with range/xrange, and while loops, you get to the
conclusion that it isn't really worth using range/xrange loops.

How did you calculate that?
b) xrange long int overflow :

for i in xrange(0, 1 << len(S)) :
........
OverflowError: long int too large to convert to int

Why do you bit shift a len result here? Just to get a huge number?
Try the same with C. You'd get an integer overflow in the first
place.

But this long int => int issue should not exist in a future python
version any more, IIRC int and long int is scheduled to be merged
somehow. (Or isn't it?)

Regards,


Björn
 
N

Neil Cerutti

C's "for(pre,cond,post) code" is nothing more, then shorthand form of
"pre; while(cond) {code; post;}"
Which translated to Python would be:

pre
while cond:
code
post

No, when you consider the continue statement, which which Python
also supports.

for (pre; cond; post) {
continue;
}

That's not an infinite loop in C, but the Python while-loop
version would be.
 
R

Roger Miller

C's "for(pre,cond,post) code" is nothing more, then shorthand form of
"pre; while(cond) {code; post;}"

I don't disagree with your basic point, but technically this is not
quite true.
Try it with
for (i = 0; i < n; i++) {
if (x == 0) continue;

printf("%d\n", x);
}
 
B

Bruno Desthuilliers

stdazi a écrit :
Hello!

Many times I was suggested to use xrange and range instead of the
while constructs, and indeed, they are quite more elegant - but, after
calculating the overhead (and losen flexibility) when working with
range/xrange, and while loops, you get to the conclusion that it isn't
really worth using range/xrange loops.

I've almost never had to use (x)range in a for loop. Usually, one just
iterate over an iterable.

(snip)

Next thing I miss is the flexibility as in C for loops :

for (i = 0; some_function() /* or other condition */ ; i++)

You may want to have a look at generators. The main idea here is to
encapsulate the knowledge about how iteration is implemented, so the
client code just have to iterate.
or,

for (i = 0 ; i < 10 ; i++)
i = 10;

for i in range(10):
i = 10

What's your point, exactly ?
I don't think range/xrange sucks, but I really think there should be
some other constructs to improve the looping flexibility.

Have a look at generators then.
 
R

Roel Schroeven

Bruno Desthuilliers schreef:
stdazi a écrit :

for i in range(10):
i = 10

What's your point, exactly ?

In the first iteration, i is set equal to 10. Then, before starting the
second iteration, i is incremented to 11; then the loop condition is
checked and results in false. So the loop terminates after the first
iteration.

So, the point is that in C you can influence the loop's behavior by
modifying the loop variable, while you cannot do that in Python (at
least not in a for-loop).

In other words, the point is that you can't translate loops literally
from C to Python. Which is nothing new, and I fail to see how that is
supposed to be a disadvantage.
 
S

Samuel Karl Peterson

Roel Schroeven <[email protected]> on Sat, 17 Feb 2007
01:31:13 GMT didst step forth and proclaim thus:

....
So, the point is that in C you can influence the loop's behavior by
modifying the loop variable, while you cannot do that in Python (at
least not in a for-loop).

What's wrong with...

for i in range(10):
if condition: break

....?
 
B

Bruno Desthuilliers

Roel Schroeven a écrit :
Bruno Desthuilliers schreef:



In the first iteration, i is set equal to 10. Then, before starting the
second iteration, i is incremented to 11; then the loop condition is
checked and results in false. So the loop terminates after the first
iteration.

oops - my bad. But why would one do so when a break would do the trick:
for i in range(10):
break
So, the point is that in C you can influence the loop's behavior by
modifying the loop variable, while you cannot do that in Python (at
least not in a for-loop).

A good thing IMHO.
In other words, the point is that you can't translate loops literally
from C to Python. Which is nothing new, and I fail to see how that is
supposed to be a disadvantage.
<aol />
 
S

sjdevnull

Bruno said:
Roel Schroeven a ecrit :

oops - my bad. But why would one do so when a break would do the trick:
for i in range(10):
break

After the C loop finishes, i is 11. After the python loop-with-break
finishes, i is 0. That may effect later code.
 
B

Bruno Desthuilliers

(e-mail address removed) a écrit :
After the C loop finishes, i is 11. After the python loop-with-break
finishes, i is 0. That may effect later code.
Lord saves me from having to maintain programs based on such
constructions...
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

Forum statistics

Threads
473,744
Messages
2,569,484
Members
44,906
Latest member
SkinfixSkintag

Latest Threads

Top