using range() in for loops

Discussion in 'Python' started by John Salerno, Apr 5, 2006.

  1. John Salerno

    John Salerno Guest

    I'm reading Text Processing in Python right now and I came across a
    comment that is helping me to see for loops in a new light. I think
    because I'm used to the C-style for loop where you create a counter
    within the loop declaration, for loops have always seemed to me to be
    about doing something a certain number of times, and not about iterating
    over an object.

    The reason for this distinction comes from the fact that I read a lot
    how using range and for is somewhat discouraged, because it doesn't
    really use a for loop for it's true purpose. So my question is, is this
    just a Python-oriented opinion about for loops, or is it a general idea?

    Also, what if you *do* need to just do something a set number of times.
    Is this okay, or does it mean you are approaching the problem
    incorrectly? Using for and range together seems to be a common idiom,
    yet at the same time discouraged, so I'm wondering what is a good balance.

    Thanks.
    John Salerno, Apr 5, 2006
    #1
    1. Advertising

  2. John Salerno

    Paul Rubin Guest

    John Salerno <> writes:
    > The reason for this distinction comes from the fact that I read a lot
    > how using range and for is somewhat discouraged, because it doesn't
    > really use a for loop for it's true purpose. So my question is, is
    > this just a Python-oriented opinion about for loops, or is it a
    > general idea?


    Normally you'd use range or xrange. range builds a complete list in
    memory so can be expensive if the number is large. xrange just counts
    up to that number.
    Paul Rubin, Apr 5, 2006
    #2
    1. Advertising

  3. John Salerno

    Guest

    hi John,
    Python doesn't provide for loop like C / C++ but using Range() or
    Xrange() you can achive all the functionalities of the C for loop.If
    you wants distributed for loop You can use Xrange.
    John Salerno wrote:
    > I'm reading Text Processing in Python right now and I came across a
    > comment that is helping me to see for loops in a new light. I think
    > because I'm used to the C-style for loop where you create a counter
    > within the loop declaration, for loops have always seemed to me to be
    > about doing something a certain number of times, and not about iterating
    > over an object.
    >
    > The reason for this distinction comes from the fact that I read a lot
    > how using range and for is somewhat discouraged, because it doesn't
    > really use a for loop for it's true purpose. So my question is, is this
    > just a Python-oriented opinion about for loops, or is it a general idea?
    >
    > Also, what if you *do* need to just do something a set number of times.
    > Is this okay, or does it mean you are approaching the problem
    > incorrectly? Using for and range together seems to be a common idiom,
    > yet at the same time discouraged, so I'm wondering what is a good balance.
    >
    > Thanks.
    , Apr 5, 2006
    #3
  4. John Salerno

    Ben Sizer Guest

    John Salerno wrote:
    > The reason for this distinction comes from the fact that I read a lot
    > how using range and for is somewhat discouraged, because it doesn't
    > really use a for loop for it's true purpose. So my question is, is this
    > just a Python-oriented opinion about for loops, or is it a general idea?


    The use of range in for loops is not discouraged, just the unnecessary
    use of it. In many languages, you were forced to use range - or size,
    or length, or count, etc - because the only practical method of
    iteration was to use successive indices into the array or other
    collection, stopping when the indexes run out. Most loops operate on
    some sort of structure in this way.

    In Python, you can iterate directly over the structure instead, so you
    no longer have to worry about keeping track of the current index,
    comparing it to the maximum index, or collecting the right object from
    the structure according to its index. Python just gives you the next
    object until there are none left. As a result, for most looping tasks,
    you have no need to use range or an equivalent.

    > Also, what if you *do* need to just do something a set number of times.
    > Is this okay, or does it mean you are approaching the problem
    > incorrectly?


    Do you really need to do something "a number of times"? If so, range
    (or xrange) is perfect for the job. More often though, you need to do
    something "once for every <whatever>", so put your 'whatevers' in a
    list and iterate over that.

    --
    Ben Sizer
    Ben Sizer, Apr 5, 2006
    #4
  5. John Salerno

    Ant Guest

    It's not just a Python thing, Java for example generally uses the
    idiom:

    for (Iterator it = list.iterator(); it.hasNext(); ) {
    Object next = it.next();
    //Do stuff to next
    }

    Horrible compared to the python idiom of course (though the latest
    version supports for (x : list){})

    Ruby has something similar in:

    list.each do |item|
    print item
    end

    Again, not as nice as the python idiom IMHO (Though there may be better
    ways - I don't know Ruby very well).

    So really, all modern programming languages seem to prefer the
    loop-over-iterator idiom in preference to looping over a numerical
    range.
    Ant, Apr 5, 2006
    #5
  6. John Salerno schreef:
    > I'm reading Text Processing in Python right now and I came across a
    > comment that is helping me to see for loops in a new light. I think
    > because I'm used to the C-style for loop where you create a counter
    > within the loop declaration, for loops have always seemed to me to be
    > about doing something a certain number of times, and not about iterating
    > over an object.
    >
    > The reason for this distinction comes from the fact that I read a lot
    > how using range and for is somewhat discouraged, because it doesn't
    > really use a for loop for it's true purpose. So my question is, is this
    > just a Python-oriented opinion about for loops, or is it a general idea?
    >
    > Also, what if you *do* need to just do something a set number of times.
    > Is this okay, or does it mean you are approaching the problem
    > incorrectly? Using for and range together seems to be a common idiom,
    > yet at the same time discouraged, so I'm wondering what is a good balance.


    I felt more or less the same when I first learned Python; I was also
    used to C-style loops, coming from a C/C++ background. In the end
    though, it turned out to be a non-issue for me.

    In many cases loops really are for iterating over sequences; more so
    than I realized when using for loops in C or C++. In these cases,
    Python's for statement works better than C-style loops. And if you
    really need to do something a certain number of times, there's still
    range() or xrange() to do it.

    It's quite simple, I think:
    - You have a sequence or iterator to loop over? Use for x in sequence.
    - You want something done a set number of times? Use for i in range().
    - You want to loop over a sequence and also need the index? Use for i, x
    in enumerate(sequence).

    --
    If I have been able to see further, it was only because I stood
    on the shoulders of giants. -- Isaac Newton

    Roel Schroeven
    Roel Schroeven, Apr 5, 2006
    #6
  7. John Salerno

    AndyL Guest

    Paul Rubin wrote:
    > Normally you'd use range or xrange. range builds a complete list in
    > memory so can be expensive if the number is large. xrange just counts
    > up to that number.


    so when range would be used instead of xrange. if xrange is more
    efficient, why range was not reimplemented?
    AndyL, Apr 5, 2006
    #7
  8. John Salerno

    John Salerno Guest

    Roel Schroeven wrote:

    > In many cases loops really are for iterating over sequences; more so
    > than I realized when using for loops in C or C++. In these cases,
    > Python's for statement works better than C-style loops. And if you
    > really need to do something a certain number of times, there's still
    > range() or xrange() to do it.


    Yeah, I'm starting to see the distinction now. I think one thing that
    confused me was that C# had two separate loops for these
    functionalities, the for loop and the foreach loop (which is equivalent
    to Python's for). But even when just doing something a number of times
    (C#'s for, Python's for), it looks much cleaner in Python because you
    don't have the long, messy three-part for statement.
    John Salerno, Apr 5, 2006
    #8
  9. John Salerno

    Georg Brandl Guest

    wrote:
    > hi John,
    > Python doesn't provide for loop like C / C++ but using Range() or
    > Xrange() you can achive all the functionalities of the C for loop.


    Not quite.

    Georg
    Georg Brandl, Apr 5, 2006
    #9
  10. John Salerno

    Georg Brandl Guest

    AndyL wrote:
    > Paul Rubin wrote:
    >> Normally you'd use range or xrange. range builds a complete list in
    >> memory so can be expensive if the number is large. xrange just counts
    >> up to that number.

    >
    > so when range would be used instead of xrange. if xrange is more
    > efficient, why range was not reimplemented?


    Because of backwards compatibility. range() returns a list, xrange() an
    iterator: list(xrange(...)) will give the same results as range(...).

    In for loops, using xrange instead of range makes no difference since the
    loop only iterates over the range. But it's a problem when someone just
    does

    l = range(100)

    and assumes that he's got a list, probably doing

    l.remove(5)

    and so on.

    In Python 3000, plans are that range() will be the same as xrange() is now,
    and anyone needing a list can call list(range(...)).

    Georg
    Georg Brandl, Apr 5, 2006
    #10
  11. On Tue, 2006-04-04 at 21:54 -0400, John Salerno wrote:
    > I'm reading Text Processing in Python right now and I came across a
    > comment that is helping me to see for loops in a new light. I think
    > because I'm used to the C-style for loop where you create a counter
    > within the loop declaration, for loops have always seemed to me to be
    > about doing something a certain number of times, and not about iterating
    > over an object.


    This is a normal reaction. Try to keep in mind that when you use any
    "higher level" language your code will closer reflect what you want
    rather than how to accomplish what you want. When you are creating a
    loop over numbers, is your goal really to count, or to use the integer
    that you are counting to do something else, like perhaps dereference
    elements of a string.

    Also, part of the problem is a preconceived restriction that you hold on
    the use of forloops. They arn't just for counting, there are all sorts
    of interesting things that can go in there. Remember, a for loop is
    basically a while loop with a little bit of syntactic sugar. Look at
    this:


    for( a, b, c ) {
    d;
    e;
    }

    is the same as

    a;
    while (b)
    {
    d;
    e;
    c;
    }


    C for and while are the same creature. Python's while loop is Python's
    version of for/while. If you wanted to mimick C you could write

    i = 0
    while( i<10 ):
    print i
    i+=1

    but this is clumbsy and slower. for(i=0;i<10;i++) is a common enough
    programing pattern, you arn't really interested in setting i,
    incrementing, doing all of that housekeeping. You really want to repeat
    10 times with with i set to 0, 1 ...

    for ... xrange does this well and is somewhat the motivation for the
    creation of xrange.



    > The reason for this distinction comes from the fact that I read a lot
    > how using range and for is somewhat discouraged, because it doesn't
    > really use a for loop for it's true purpose. So my question is, is this


    Nothing in a well defined language has a true purpose. The true purpose
    of an int in C isn't to be for loop fodder. One of the hallmarks of a
    well designed language is orthogonality; most anything works with
    anything else. A feature that has one specific use doesn't provide much
    utility to justify the effort used to create it.


    > just a Python-oriented opinion about for loops, or is it a general idea?


    Programming languages borrow from heavily from natural languages; yes,
    for is a loaned word from English that exists in C, python ... lots of
    languages. But just as when one human language borrows from another,
    the resulting semantics are not always the same.

    Each as a formal semantic. C and Python are somewhat different, and
    yes, you could describe the philosophical difference as a matter of
    opinions.

    >
    > Also, what if you *do* need to just do something a set number of times.
    > Is this okay, or does it mean you are approaching the problem
    > incorrectly? Using for and range together seems to be a common idiom,
    > yet at the same time discouraged, so I'm wondering what is a good balance.


    The correct idiom is for( xrange( foo )). This is preferred over range
    for efficiency.

    Historically there were no iters. If you wanted to do loop you would
    say

    for x in range( 10 ):
    foo
    bar

    range created a list of 10 items and x marched over them. Creating this
    list in advance doesn't need to take any more time; you have to create
    the number objects anyway to assign to x at some point, so you might as
    well get that done with upfront.

    The problem was memory consumption. The memory requirements of the
    list put a bound on large your iteration could be.

    xrange was created, and soon followed general iters. Now, the range
    object generates the numbers on the fly as the loop runs .. the loop
    says "hey, whats next" and the next item is returned.

    When people say "don't say for x in range" they are really saying "use
    xrange instead of range."
    Adam DePrince, Apr 5, 2006
    #11
  12. AndyL <ask@me> wrote:
    >Paul Rubin wrote:
    >> Normally you'd use range or xrange. range builds a complete list in
    >> memory so can be expensive if the number is large. xrange just counts
    >> up to that number.

    >so when range would be used instead of xrange. if xrange is more
    >efficient, why range was not reimplemented?


    If you actually want the list for some reason:

    $ python2.4 -mtimeit 'list(xrange(100))'
    100000 loops, best of 3: 4.54 usec per loop
    $ python2.4 -mtimeit 'range(100)'
    100000 loops, best of 3: 2.61 usec per loop

    --
    \S -- -- http://www.chaos.org.uk/~sion/
    ___ | "Frankly I have no feelings towards penguins one way or the other"
    \X/ | -- Arthur C. Clarke
    her nu becomeþ se bera eadward ofdun hlæddre heafdes bæce bump bump bump
    Sion Arrowsmith, Apr 5, 2006
    #12
  13. On Wed, 05 Apr 2006 09:16:37 -0400, AndyL wrote:

    > Paul Rubin wrote:
    >> Normally you'd use range or xrange. range builds a complete list in
    >> memory so can be expensive if the number is large. xrange just counts
    >> up to that number.

    >
    > so when range would be used instead of xrange. if xrange is more
    > efficient, why range was not reimplemented?


    For historical reasons.

    Don't worry, in Python3000, range() will be an iterator, and xrange() will
    disappear. Until then, I use range() for small loops (by small I mean
    anything up to a few tens of thousands), and xrange() only when I
    absolutely have to optimize my code to save a piddling few tens of
    kilobytes of memory.


    --
    Steven.
    Steven D'Aprano, Apr 5, 2006
    #13
  14. On Wed, 05 Apr 2006 16:15:12 +0200, Georg Brandl wrote:

    > wrote:
    >> hi John,
    >> Python doesn't provide for loop like C / C++ but using Range() or
    >> Xrange() you can achive all the functionalities of the C for loop.

    >
    > Not quite.


    Care to explain what the differences are, or shall we guess?



    --
    Steven.
    Steven D'Aprano, Apr 5, 2006
    #14
  15. John Salerno

    Georg Brandl Guest

    Steven D'Aprano wrote:
    > On Wed, 05 Apr 2006 16:21:02 +0200, Georg Brandl wrote:
    >
    >> Because of backwards compatibility. range() returns a list, xrange() an
    >> iterator: list(xrange(...)) will give the same results as range(...).

    >
    > Georg is pretty much correct in his explanation, but just to dot all the
    > I's and cross all the T's, we should explain that xrange() doesn't return
    > an iterator, it returns a special xrange object:


    Ah yes, the old iterator <-> iterable problem ;)

    Georg
    Georg Brandl, Apr 5, 2006
    #15
  16. On Wed, 05 Apr 2006 16:21:02 +0200, Georg Brandl wrote:

    > Because of backwards compatibility. range() returns a list, xrange() an
    > iterator: list(xrange(...)) will give the same results as range(...).


    Georg is pretty much correct in his explanation, but just to dot all the
    I's and cross all the T's, we should explain that xrange() doesn't return
    an iterator, it returns a special xrange object:

    >>> x = xrange(1000)
    >>> type(x)

    <type 'xrange'>

    xrange existed as a special bit of magic before Python supported
    iterators. While xrange objects behave (sort of) like iterators, they
    aren't quite the same. For instance, they don't have a next attribute:

    >>> x.next

    Traceback (most recent call last):
    File "<stdin>", line 1, in ?
    AttributeError: 'xrange' object has no attribute 'next'

    whereas iterators do:

    >>> i = iter(range(1000))
    >>> i.next

    <method-wrapper object at 0xf7054d2c>

    Likewise, you can get random access to the items in an xrange object:

    >>> x[500]

    500

    but not in iterators:

    >>> i[500]

    Traceback (most recent call last):
    File "<stdin>", line 1, in ?
    TypeError: unsubscriptable object



    --
    Steven.
    Steven D'Aprano, Apr 5, 2006
    #16
  17. John Salerno

    Georg Brandl Guest

    Steven D'Aprano wrote:
    > On Wed, 05 Apr 2006 16:15:12 +0200, Georg Brandl wrote:
    >
    >> wrote:
    >>> hi John,
    >>> Python doesn't provide for loop like C / C++ but using Range() or
    >>> Xrange() you can achive all the functionalities of the C for loop.

    >>
    >> Not quite.

    >
    > Care to explain what the differences are, or shall we guess?


    C's for is much more powerful.

    for(a; b; c) { d } translates to

    a
    while b
    d
    c

    which can't be replaced by a simple

    for i in range(...)

    Georg
    Georg Brandl, Apr 5, 2006
    #17
  18. Georg Brandl <> wrote:

    > Steven D'Aprano wrote:
    > > On Wed, 05 Apr 2006 16:15:12 +0200, Georg Brandl wrote:
    > >
    > >> wrote:
    > >>> hi John,
    > >>> Python doesn't provide for loop like C / C++ but using Range() or
    > >>> Xrange() you can achive all the functionalities of the C for loop.
    > >>
    > >> Not quite.

    > >
    > > Care to explain what the differences are, or shall we guess?

    >
    > C's for is much more powerful.
    >
    > for(a; b; c) { d } translates to
    >
    > a
    > while b
    > d
    > c
    >
    > which can't be replaced by a simple
    >
    > for i in range(...)


    No, but it can be easily replaced by:

    # factoring out the loopstructure...
    def gotcha():
    a
    while b:
    yield c

    # ...from the loopbody
    for i in gotcha():
    d

    or several variations thereof. In C++, I almost half-heartedly replace
    most of Python's generators' power with C++'s templated iterators, but
    when I program in C I find myself really straightjacketed these days in
    NOT being able to pull out the "loopstructure" as I can in Python (and,
    about halfway, in C++).


    Alex
    Alex Martelli, Apr 6, 2006
    #18
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Steve R. Hastings

    efficiency of range() and xrange() in for loops

    Steve R. Hastings, Apr 5, 2006, in forum: Python
    Replies:
    29
    Views:
    767
    Fredrik Lundh
    Apr 9, 2006
  2. John Salerno
    Replies:
    38
    Views:
    696
    Dave Parker
    May 20, 2008
  3. Ebenezer
    Replies:
    0
    Views:
    218
    Ebenezer
    Jul 24, 2011
  4. Me
    Replies:
    2
    Views:
    226
  5. it_says_BALLS_on_your forehead

    extract range of lines using range op bug?

    it_says_BALLS_on_your forehead, Mar 3, 2006, in forum: Perl Misc
    Replies:
    3
    Views:
    171
    it_says_BALLS_on_your forehead
    Mar 3, 2006
Loading...

Share This Page