modifying mutable list elements in a for loop

Discussion in 'Python' started by Peter Ballard, May 26, 2004.

  1. Whew. I hope that title is descriptive!

    Hi all,

    The python tutorial tells me "It is not safe to modify the sequence
    being iterated over in the loop". But what if my list elements are
    mutable, such as lists or objects, e.g.

    a = [[1,2], [3,4], [5,6], [7,8]]
    for coord in a:
    coord.append(10)
    print str(a)

    When I tried it, it gave the "expected" answer, i.e.

    [[1, 2, 10], [3, 4, 10], [5, 6, 10], [7, 8, 10]]

    It worked, but is it safe? I can't see why it wouldn't be, but
    technically I have broken the rules because I have modified the
    sequence which is being looped over.

    [It seems to me that the list elements are pointers (in C-speak), so I
    can safely modify the data they are pointing to, because I am not
    modifying the list elements themselves. Or is that an implementation
    detail (i.e. not safe)?]

    Actually the more I think about it the more I think it must be OK,
    because how else can one perform an operation on a list of objects?
    But that phrase "It is not safe to modify the sequence being iterated
    over in the loop" in the tutorial has me slightly worried.

    --
    Regards,

    Peter Ballard
    Adelaide, AUSTRALIA

    http://members.ozemail.com.au/~pballard/
     
    Peter Ballard, May 26, 2004
    #1
    1. Advertising

  2. Peter Ballard wrote:
    > Whew. I hope that title is descriptive!
    >
    > Hi all,
    >
    > The python tutorial tells me "It is not safe to modify the sequence
    > being iterated over in the loop". But what if my list elements are
    > mutable, such as lists or objects, e.g.
    >
    > a = [[1,2], [3,4], [5,6], [7,8]]
    > for coord in a:
    > coord.append(10)
    > print str(a)
    >
    > When I tried it, it gave the "expected" answer, i.e.
    >
    > [[1, 2, 10], [3, 4, 10], [5, 6, 10], [7, 8, 10]]
    >
    > It worked, but is it safe? I can't see why it wouldn't be, but
    > technically I have broken the rules because I have modified the
    > sequence which is being looped over.


    You're fine. You didn't modify the sequence a, but other objects which
    happened to be in the sequence. What the tutorial warns you against is
    doing something like a.append([]) or a.pop() in your loop. Then you'd be
    modifying the sequence itself.

    --
    Shalabh
     
    Shalabh Chaturvedi, May 26, 2004
    #2
    1. Advertising

  3. Peter Ballard

    Peter Hansen Guest

    Peter Ballard wrote:

    > The python tutorial tells me "It is not safe to modify the sequence
    > being iterated over in the loop". But what if my list elements are
    > mutable, such as lists or objects, e.g.


    Treat that as "not safe if you don't know what you are doing".
    David's answer is on the mark, but furthermore you *can* modify
    the sequence being iterated over, if that's really what you
    want to do. Some algorithms can probably even benefit from the
    technique, though I can't think of anything off-hand.

    As with many things in Python, consenting adults can do what they
    want, and the tutorial is just giving a necessary warning since
    many beginners, and even non-beginners from time to time, will be
    caught by this problem otherwise.

    > [It seems to me that the list elements are pointers (in C-speak), so I
    > can safely modify the data they are pointing to, because I am not
    > modifying the list elements themselves. Or is that an implementation
    > detail (i.e. not safe)?]


    It's safe. And around here they're usually called "references" instead
    of "pointers".

    > But that phrase "It is not safe to modify the sequence being iterated
    > over in the loop" in the tutorial has me slightly worried.


    Good. ;-) Then you'll have to pause and think about it from time to
    time as you learn, until you've integrated the knowledge so deeply
    that you automatically do the right thing when iterating over lists.
    That's the goal of that warning in the tutorial (if I may channel the
    author for a moment :).

    -Peter
     
    Peter Hansen, May 26, 2004
    #3
  4. Peter Ballard

    Larry Bates Guest

    If you write this as a list comprehension it "seems"
    more safe (and clear to me at least):

    a = [[1,2], [3,4], [5,6], [7,8]]
    a = [x+[10] for x in a]

    Larry Bates
    Syscon, Inc.

    "Peter Ballard" <> wrote in message
    news:...
    > Whew. I hope that title is descriptive!
    >
    > Hi all,
    >
    > The python tutorial tells me "It is not safe to modify the sequence
    > being iterated over in the loop". But what if my list elements are
    > mutable, such as lists or objects, e.g.
    >
    > a = [[1,2], [3,4], [5,6], [7,8]]
    > for coord in a:
    > coord.append(10)
    > print str(a)
    >
    > When I tried it, it gave the "expected" answer, i.e.
    >
    > [[1, 2, 10], [3, 4, 10], [5, 6, 10], [7, 8, 10]]
    >
    > It worked, but is it safe? I can't see why it wouldn't be, but
    > technically I have broken the rules because I have modified the
    > sequence which is being looped over.
    >
    > [It seems to me that the list elements are pointers (in C-speak), so I
    > can safely modify the data they are pointing to, because I am not
    > modifying the list elements themselves. Or is that an implementation
    > detail (i.e. not safe)?]
    >
    > Actually the more I think about it the more I think it must be OK,
    > because how else can one perform an operation on a list of objects?
    > But that phrase "It is not safe to modify the sequence being iterated
    > over in the loop" in the tutorial has me slightly worried.
    >
    > --
    > Regards,
    >
    > Peter Ballard
    > Adelaide, AUSTRALIA
    >
    > http://members.ozemail.com.au/~pballard/
     
    Larry Bates, May 26, 2004
    #4
  5. Peter Ballard

    Terry Reedy Guest

    "Peter Ballard" <> wrote in message
    news:...
    > The python tutorial tells me "It is not safe to modify the sequence
    > being iterated over in the loop".


    What is not safe *in general* is adding and subtracting items to and from
    the list while traversing it, either directly or indirectly (via an index
    variable). There are exceptions, but they may or may not be
    implementation dependent (I would have to read the standard more closely to
    say more), and should only be used by people who understand them.

    What is safe is modifying or replacing the 'current' item. The former is
    easy. The latter requires the item index, as in 'for i in range(l): l =
    f(l)' (in-place map) -- but here, the iteration list isnt the list being
    modified! (nor is it with enumerate()!)

    Terry J. Reedy
     
    Terry Reedy, May 26, 2004
    #5
  6. Peter Ballard

    Arthur Guest

    On Wed, 26 May 2004 09:37:54 -0400, Peter Hansen <>
    wrote:

    >Peter Ballard wrote:
    >
    >> The python tutorial tells me "It is not safe to modify the sequence
    >> being iterated over in the loop". But what if my list elements are
    >> mutable, such as lists or objects, e.g.

    >
    >Treat that as "not safe if you don't know what you are doing".
    >David's answer is on the mark, but furthermore you *can* modify
    >the sequence being iterated over, if that's really what you
    >want to do.


    OTOH:

    d={"a":1,"b":2}
    for k in d:
    d.pop(k)

    results in:

    RuntimeError: dictionary changed size during iteration

    I don't recall why I ran into this. But was mildly surprised.

    Art
     
    Arthur, May 27, 2004
    #6
  7. Peter Ballard

    Peter Hansen Guest

    Arthur wrote:

    > On Wed, 26 May 2004 09:37:54 -0400, Peter Hansen wrote:
    >>Treat that as "not safe if you don't know what you are doing".
    >>David's answer is on the mark, but furthermore you *can* modify
    >>the sequence being iterated over, if that's really what you
    >>want to do.

    >
    > OTOH:
    >
    > d={"a":1,"b":2}
    > for k in d:
    > d.pop(k)
    >
    > results in:
    >
    > RuntimeError: dictionary changed size during iteration
    >
    > I don't recall why I ran into this. But was mildly surprised.


    Interesting, but on second thought quite logical I think.
    After all, dictionaries are not *sequences*, so there is
    no defined order in which to iterate over their keys, so
    changes would have undefined results unless a copy was made
    of the keys at the start of the loop. I suspect that
    wasn't done because it would waste memory and time and
    the only benefit would be allowing in-loop modifications.

    Iterating over a sequences, on the other hand, is handled
    in a clearly defined fashion and therefore modifications
    to the sequence can be done during the loop if that makes
    sense (though mostly it doesn't).

    -Peter
     
    Peter Hansen, May 27, 2004
    #7
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Michael Chermside
    Replies:
    1
    Views:
    340
    Roy Smith
    May 26, 2004
  2. Chris Lasher

    Priority Queue with Mutable Elements

    Chris Lasher, Jun 15, 2007, in forum: Python
    Replies:
    3
    Views:
    551
    Scott David Daniels
    Jun 16, 2007
  3. Replies:
    6
    Views:
    465
    Andrey Tarasevich
    Feb 22, 2008
  4. Isaac Won
    Replies:
    9
    Views:
    397
    Ulrich Eckhardt
    Mar 4, 2013
  5. Matt
    Replies:
    67
    Views:
    443
    glen herrmannsfeldt
    Jan 4, 2014
Loading...

Share This Page