modifying mutable list elements in a for loop

[Peter Ballard]

Whew. I hope that title is descriptive!

Hi all,

The python tutorial tells me "It is not safe to modify the sequence
being iterated over in the loop". But what if my list elements are
mutable, such as lists or objects, e.g.

a = [[1,2], [3,4], [5,6], [7,8]]
for coord in a:
coord.append(10)
print str(a)

When I tried it, it gave the "expected" answer, i.e.

[[1, 2, 10], [3, 4, 10], [5, 6, 10], [7, 8, 10]]

It worked, but is it safe? I can't see why it wouldn't be, but
technically I have broken the rules because I have modified the
sequence which is being looped over.

[It seems to me that the list elements are pointers (in C-speak), so I
can safely modify the data they are pointing to, because I am not
modifying the list elements themselves. Or is that an implementation
detail (i.e. not safe)?]

Actually the more I think about it the more I think it must be OK,
because how else can one perform an operation on a list of objects?
But that phrase "It is not safe to modify the sequence being iterated
over in the loop" in the tutorial has me slightly worried.

--
Regards,

Peter Ballard
Adelaide, AUSTRALIA
(e-mail address removed)
http://members.ozemail.com.au/~pballard/

 

[Shalabh Chaturvedi]

Whew. I hope that title is descriptive!

Hi all,

The python tutorial tells me "It is not safe to modify the sequence
being iterated over in the loop". But what if my list elements are
mutable, such as lists or objects, e.g.

a = [[1,2], [3,4], [5,6], [7,8]]
for coord in a:
coord.append(10)
print str(a)

When I tried it, it gave the "expected" answer, i.e.

[[1, 2, 10], [3, 4, 10], [5, 6, 10], [7, 8, 10]]

It worked, but is it safe? I can't see why it wouldn't be, but
technically I have broken the rules because I have modified the
sequence which is being looped over.

You're fine. You didn't modify the sequence a, but other objects which
happened to be in the sequence. What the tutorial warns you against is
doing something like a.append([]) or a.pop() in your loop. Then you'd be
modifying the sequence itself.

 

[Peter Hansen]

The python tutorial tells me "It is not safe to modify the sequence
being iterated over in the loop". But what if my list elements are
mutable, such as lists or objects, e.g.

Treat that as "not safe if you don't know what you are doing".
David's answer is on the mark, but furthermore you *can* modify
the sequence being iterated over, if that's really what you
want to do. Some algorithms can probably even benefit from the
technique, though I can't think of anything off-hand.

As with many things in Python, consenting adults can do what they
want, and the tutorial is just giving a necessary warning since
many beginners, and even non-beginners from time to time, will be
caught by this problem otherwise.

[It seems to me that the list elements are pointers (in C-speak), so I
can safely modify the data they are pointing to, because I am not
modifying the list elements themselves. Or is that an implementation
detail (i.e. not safe)?]

It's safe. And around here they're usually called "references" instead
of "pointers".

But that phrase "It is not safe to modify the sequence being iterated
over in the loop" in the tutorial has me slightly worried.

Good. ;-) Then you'll have to pause and think about it from time to
time as you learn, until you've integrated the knowledge so deeply
that you automatically do the right thing when iterating over lists.
That's the goal of that warning in the tutorial (if I may channel the
author for a moment .

-Peter

 

[Larry Bates]

If you write this as a list comprehension it "seems"
more safe (and clear to me at least):

a = [[1,2], [3,4], [5,6], [7,8]]
a = [x+[10] for x in a]

Larry Bates
Syscon, Inc.

 

[Terry Reedy]

The python tutorial tells me "It is not safe to modify the sequence
being iterated over in the loop".

What is not safe *in general* is adding and subtracting items to and from
the list while traversing it, either directly or indirectly (via an index
variable). There are exceptions, but they may or may not be
implementation dependent (I would have to read the standard more closely to
say more), and should only be used by people who understand them.

What is safe is modifying or replacing the 'current' item. The former is
easy. The latter requires the item index, as in 'for i in range(l): l =
f(l)' (in-place map) -- but here, the iteration list isnt the list being
modified! (nor is it with enumerate()!)

Terry J. Reedy

 

[Arthur]

Treat that as "not safe if you don't know what you are doing".
David's answer is on the mark, but furthermore you *can* modify
the sequence being iterated over, if that's really what you
want to do.

OTOH:

d={"a":1,"b":2}
for k in d:
d.pop(k)

results in:

RuntimeError: dictionary changed size during iteration

I don't recall why I ran into this. But was mildly surprised.

Art

 

[Peter Hansen]

OTOH:

d={"a":1,"b":2}
for k in d:
d.pop(k)

results in:

RuntimeError: dictionary changed size during iteration

I don't recall why I ran into this. But was mildly surprised.

Interesting, but on second thought quite logical I think.
After all, dictionaries are not *sequences*, so there is
no defined order in which to iterate over their keys, so
changes would have undefined results unless a copy was made
of the keys at the start of the loop. I suspect that
wasn't done because it would waste memory and time and
the only benefit would be allowing in-loop modifications.

Iterating over a sequences, on the other hand, is handled
in a clearly defined fashion and therefore modifications
to the sequence can be done during the loop if that makes
sense (though mostly it doesn't).

-Peter