modifying mutable list elements in a for loop

Discussion in 'Python' started by Peter Ballard, May 26, 2004.

  1. Whew. I hope that title is descriptive!

    Hi all,

    The python tutorial tells me "It is not safe to modify the sequence
    being iterated over in the loop". But what if my list elements are
    mutable, such as lists or objects, e.g.

    a = [[1,2], [3,4], [5,6], [7,8]]
    for coord in a:
    print str(a)

    When I tried it, it gave the "expected" answer, i.e.

    [[1, 2, 10], [3, 4, 10], [5, 6, 10], [7, 8, 10]]

    It worked, but is it safe? I can't see why it wouldn't be, but
    technically I have broken the rules because I have modified the
    sequence which is being looped over.

    [It seems to me that the list elements are pointers (in C-speak), so I
    can safely modify the data they are pointing to, because I am not
    modifying the list elements themselves. Or is that an implementation
    detail (i.e. not safe)?]

    Actually the more I think about it the more I think it must be OK,
    because how else can one perform an operation on a list of objects?
    But that phrase "It is not safe to modify the sequence being iterated
    over in the loop" in the tutorial has me slightly worried.


    Peter Ballard
    Adelaide, AUSTRALIA
    Peter Ballard, May 26, 2004
    1. Advertisements

  2. You're fine. You didn't modify the sequence a, but other objects which
    happened to be in the sequence. What the tutorial warns you against is
    doing something like a.append([]) or a.pop() in your loop. Then you'd be
    modifying the sequence itself.
    Shalabh Chaturvedi, May 26, 2004
    1. Advertisements

  3. Peter Ballard

    Peter Hansen Guest

    Treat that as "not safe if you don't know what you are doing".
    David's answer is on the mark, but furthermore you *can* modify
    the sequence being iterated over, if that's really what you
    want to do. Some algorithms can probably even benefit from the
    technique, though I can't think of anything off-hand.

    As with many things in Python, consenting adults can do what they
    want, and the tutorial is just giving a necessary warning since
    many beginners, and even non-beginners from time to time, will be
    caught by this problem otherwise.
    It's safe. And around here they're usually called "references" instead
    of "pointers".
    Good. ;-) Then you'll have to pause and think about it from time to
    time as you learn, until you've integrated the knowledge so deeply
    that you automatically do the right thing when iterating over lists.
    That's the goal of that warning in the tutorial (if I may channel the
    author for a moment :).

    Peter Hansen, May 26, 2004
  4. Peter Ballard

    Larry Bates Guest

    If you write this as a list comprehension it "seems"
    more safe (and clear to me at least):

    a = [[1,2], [3,4], [5,6], [7,8]]
    a = [x+[10] for x in a]

    Larry Bates
    Syscon, Inc.
    Larry Bates, May 26, 2004
  5. Peter Ballard

    Terry Reedy Guest

    What is not safe *in general* is adding and subtracting items to and from
    the list while traversing it, either directly or indirectly (via an index
    variable). There are exceptions, but they may or may not be
    implementation dependent (I would have to read the standard more closely to
    say more), and should only be used by people who understand them.

    What is safe is modifying or replacing the 'current' item. The former is
    easy. The latter requires the item index, as in 'for i in range(l): l =
    f(l)' (in-place map) -- but here, the iteration list isnt the list being
    modified! (nor is it with enumerate()!)

    Terry J. Reedy
    Terry Reedy, May 26, 2004
  6. Peter Ballard

    Arthur Guest


    for k in d:

    results in:

    RuntimeError: dictionary changed size during iteration

    I don't recall why I ran into this. But was mildly surprised.

    Arthur, May 27, 2004
  7. Peter Ballard

    Peter Hansen Guest

    Interesting, but on second thought quite logical I think.
    After all, dictionaries are not *sequences*, so there is
    no defined order in which to iterate over their keys, so
    changes would have undefined results unless a copy was made
    of the keys at the start of the loop. I suspect that
    wasn't done because it would waste memory and time and
    the only benefit would be allowing in-loop modifications.

    Iterating over a sequences, on the other hand, is handled
    in a clearly defined fashion and therefore modifications
    to the sequence can be done during the loop if that makes
    sense (though mostly it doesn't).

    Peter Hansen, May 27, 2004
    1. Advertisements

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments (here). After that, you can post your question and our members will help you out.