# Question about idioms for clearing a list

Discussion in 'Python' started by Steven Watanabe, Jan 31, 2006.

1. ### Steven WatanabeGuest

I know that the standard idioms for clearing a list are:

(1) mylist[:] = []
(2) del mylist[:]

I guess I'm not in the "slicing frame of mind", as someone put it, but
can someone explain what the difference is between these and:

(3) mylist = []

Why are (1) and (2) preferred? I think the first two are changing the
list in-place, but why is that better? Isn't the end result the same?

--
Steven.

Steven Watanabe, Jan 31, 2006

2. ### Diez B. RoggischGuest

Steven Watanabe wrote:

> I know that the standard idioms for clearing a list are:
>
> (1) mylist[:] = []
> (2) del mylist[:]
>
> I guess I'm not in the "slicing frame of mind", as someone put it, but
> can someone explain what the difference is between these and:
>
> (3) mylist = []
>
> Why are (1) and (2) preferred? I think the first two are changing the
> list in-place, but why is that better? Isn't the end result the same?

No. Consider this simple example:

class Foo(object):
def __init__(self, all_my_thingies):
self.all_my_thingies = all_my_thingies

things = [1,2,3,4,5]

f = Foo(things)

things = [] # I've been robbed

print f.all_my_thingies # or not?

The reason is that l = [] just rebinds a new object (a list, but it could be
anything) to a name, while l[:] = [] will alter the object _referred_ to by
l. That is a HUGE difference!

Diez

Diez B. Roggisch, Jan 31, 2006

3. ### Xavier MorelGuest

Steven Watanabe wrote:
> I know that the standard idioms for clearing a list are:
>
> (1) mylist[:] = []
> (2) del mylist[:]
>
> I guess I'm not in the "slicing frame of mind", as someone put it, but
> can someone explain what the difference is between these and:
>
> (3) mylist = []
>
> Why are (1) and (2) preferred? I think the first two are changing the
> list in-place, but why is that better? Isn't the end result the same?
>
> --
> Steven.

The solution (1) and (2) clear the content of the list (replace the
content of the list by an empty list for (1), or delete the content of
the list for (2) while the solution (3) creates a new list and binds the
old name to the new list. This means that you don't actually modify
(clear) the initial list.

Just try it out:

>>> a = range(10)
>>> a

[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
>>> b = a
>>> b

[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
>>> c = a
>>> c

[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
>>> a.append(10) # assert that a, b and c point to the same list
>>> a

[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
>>> b

[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
>>> c

[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
>>> b = []
>>> b

[]
>>> a

[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
>>> del c[:]
>>> c

[]
>>> a

[]
>>>

Here we can see that using method (3) on b didn't modify the content of
a (even though we asserted that they were the same list).
Using method (2) on c, the other hand, did modify the content of a.

This is because method (3) on b actually created a new list and bound
"b" (as a name) to that new list, without modifying the list "b" used to
be bound to (which is the one "a" and "c" are still bound to).

Xavier Morel, Jan 31, 2006
4. ### Guest

Diez B. Roggisch:
> The reason is that l = [] just rebinds a new object (a list, but it could be
> anything) to a name, while l[:] = [] will alter the object _referred_ to by
> l. That is a HUGE difference!

In my programs I have seen that there is another practical difference
between version 1 and 3:
(1) mylist[:] = []
(3) mylist = []
If you create a big mylist again and again many times, the version 1
uses the memory more efficiently (probably less work for the garbage
collector), and the program can be (quite) faster (this is true in some
implementations different from CPython too).

Bye,
bearophile

, Jan 31, 2006
5. ### Will McGuganGuest

Steven Watanabe wrote:
> I know that the standard idioms for clearing a list are:
>
> (1) mylist[:] = []
> (2) del mylist[:]
>
> I guess I'm not in the "slicing frame of mind", as someone put it, but
> can someone explain what the difference is between these and:
>
> (3) mylist = []
>
> Why are (1) and (2) preferred? I think the first two are changing the
> list in-place, but why is that better? Isn't the end result the same?

I'm wondering why there is no 'clear' for lists. It feels like a common
operation for mutable containers. :-/

Will McGugan

Will McGugan, Feb 6, 2006
6. ### Steve HoldenGuest

Will McGugan wrote:
> Steven Watanabe wrote:
>
>>I know that the standard idioms for clearing a list are:
>>
>> (1) mylist[:] = []
>> (2) del mylist[:]
>>
>>I guess I'm not in the "slicing frame of mind", as someone put it, but
>>can someone explain what the difference is between these and:
>>
>> (3) mylist = []
>>
>>Why are (1) and (2) preferred? I think the first two are changing the
>>list in-place, but why is that better? Isn't the end result the same?

>
>
> I'm wondering why there is no 'clear' for lists. It feels like a common
> operation for mutable containers. :-/
>

Because it's just as easy to create and assign a new empty list (and
have the old unused one garbage collected).

l = []

is all you need!

regards
Steve
--
Steve Holden +44 150 684 7255 +1 800 494 3119
Holden Web LLC www.holdenweb.com
PyCon TX 2006 www.python.org/pycon/

Steve Holden, Feb 6, 2006
7. ### Steven D'ApranoGuest

On Mon, 06 Feb 2006 13:35:10 +0000, Steve Holden wrote:

>> I'm wondering why there is no 'clear' for lists. It feels like a common
>> operation for mutable containers. :-/
>>

> Because it's just as easy to create and assign a new empty list (and
> have the old unused one garbage collected).
>
> l = []
>
> is all you need!

Not so. If that logic were correct, then dicts wouldn't need a clear
method either, because you could just assign a new empty dict with d = {}.

But your own sentence tells us why this is not sufficient: because you
aren't emptying the list, you are reassigning (rebinding) the name. The
old list still exists, and there is no guarantee that it will be garbage
collected, because there is no guarantee that it isn't in use somewhere
else:

L = [0,1,2]
D = {"key": L}
L = [] # rebinds the name L, but the list instance still exists

Perhaps it is arguable that there is no need for a clear method because
L[:] = [] is so easy to do. Personally, while I agree that it is easy, it
is hardly intuitive or obvious, and I too would prefer an explicit clear
method for mutable sequences.

--
Steven.

Steven D'Aprano, Feb 6, 2006
8. ### Dan SommersGuest

On Tue, 07 Feb 2006 01:01:43 +1100,
Steven D'Aprano <> wrote:

> On Mon, 06 Feb 2006 13:35:10 +0000, Steve Holden wrote:
>>> I'm wondering why there is no 'clear' for lists. It feels like a common
>>> operation for mutable containers. :-/
>>>

>> Because it's just as easy to create and assign a new empty list (and
>> have the old unused one garbage collected).
>>
>> l = []
>>
>> is all you need!

> Not so. If that logic were correct, then dicts wouldn't need a clear
> method either, because you could just assign a new empty dict with d = {}.

> But your own sentence tells us why this is not sufficient: because you
> aren't emptying the list, you are reassigning (rebinding) the name. The
> old list still exists, and there is no guarantee that it will be garbage
> collected, because there is no guarantee that it isn't in use somewhere
> else:

> L = [0,1,2]
> D = {"key": L}
> L = [] # rebinds the name L, but the list instance still exists

That is a red herring. Consider this:

L = [object(), object()]
O = L[1]
L = [] # or insert your favorite list-clearing/emptying statement here

What damage is done now that O is still referring to one of the items
that used to be in L?

The trouble begins when references to "the list to which L refers" end
up somewhere else. Then we have to wonder if rebinding L will leave
some other block of code with an outdated list.

Regards,
Dan

--
Dan Sommers
<http://www.tombstonezero.net/dan/>

Dan Sommers, Feb 6, 2006
9. ### Peter HansenGuest

Steven D'Aprano wrote:
> Perhaps it is arguable that there is no need for a clear method because
> L[:] = [] is so easy to do. Personally, while I agree that it is easy, it
> is hardly intuitive or obvious, and I too would prefer an explicit clear
> method for mutable sequences.

Possibly another case where "patches are welcome"...

Peter Hansen, Feb 6, 2006
10. ### Fredrik LundhGuest

Peter Hansen wrote:

> > Perhaps it is arguable that there is no need for a clear method because
> > L[:] = [] is so easy to do. Personally, while I agree that it is easy, it
> > is hardly intuitive or obvious, and I too would prefer an explicit clear
> > method for mutable sequences.

>
> Possibly another case where "patches are welcome"...

so we can have three ways to do the same thing? the right way to
nuke a sequence is to do "del L[:]". this is explained in Python 101.

(del doesn't work on dictionaries)

</F>

Fredrik Lundh, Feb 6, 2006
11. ### Mel WilsonGuest

Fredrik Lundh wrote:

> (del doesn't work on dictionaries)

.... or rather [:] doesn't work on dictionaries ...

Python 2.4.2 (#1, Jan 23 2006, 21:24:54)
[GCC 3.3.4] on linux2
>>> d={'a':1, 'b':2, 'c':3}
>>> print d

{'a': 1, 'c': 3, 'b': 2}
>>> del d['b']
>>> print d

{'a': 1, 'c': 3}
>>> del d[:]

Traceback (most recent call last):
File "<stdin>", line 1, in ?
TypeError: unhashable type
>>>

Regards. Mel.

Mel Wilson, Feb 6, 2006
12. ### Steven D'ApranoGuest

On Mon, 06 Feb 2006 09:39:32 -0500, Dan Sommers wrote:

> On Tue, 07 Feb 2006 01:01:43 +1100,
> Steven D'Aprano <> wrote:
>
>> On Mon, 06 Feb 2006 13:35:10 +0000, Steve Holden wrote:
>>>> I'm wondering why there is no 'clear' for lists. It feels like a common
>>>> operation for mutable containers. :-/
>>>>
>>> Because it's just as easy to create and assign a new empty list (and
>>> have the old unused one garbage collected).
>>>
>>> l = []
>>>
>>> is all you need!

>
>> Not so. If that logic were correct, then dicts wouldn't need a clear
>> method either, because you could just assign a new empty dict with d = {}.

>
>> But your own sentence tells us why this is not sufficient: because you
>> aren't emptying the list, you are reassigning (rebinding) the name. The
>> old list still exists, and there is no guarantee that it will be garbage
>> collected, because there is no guarantee that it isn't in use somewhere
>> else:

>
>> L = [0,1,2]
>> D = {"key": L}
>> L = [] # rebinds the name L, but the list instance still exists

>
> That is a red herring. Consider this:
>
> L = [object(), object()]
> O = L[1]
> L = [] # or insert your favorite list-clearing/emptying statement here
>
> What damage is done now that O is still referring to one of the items
> that used to be in L?

What relevance is this? If there is one and only one reference to the list
L, then it will be garbage collected when L is rebound. I never denied
that. I pointed out that, in the general case, you may have multiple
references to the list (not all of which are bound to names), and
rebinding the name L will NOT have the side-effect of clearing the list.

> The trouble begins when references to "the list to which L refers" end
> up somewhere else. Then we have to wonder if rebinding L will leave
> some other block of code with an outdated list.

Precisely, just as my example shows.

--
Steven.

Steven D'Aprano, Feb 6, 2006
13. ### Steven D'ApranoGuest

Fredrik Lundh wrote:

> Peter Hansen wrote:
>
>
>>>Perhaps it is arguable that there is no need for a clear method because
>>>L[:] = [] is so easy to do. Personally, while I agree that it is easy, it
>>>is hardly intuitive or obvious, and I too would prefer an explicit clear
>>>method for mutable sequences.

>>
>>Possibly another case where "patches are welcome"...

>
>
> so we can have three ways to do the same thing? the right way to
> nuke a sequence is to do "del L[:]". this is explained in Python 101.

The Zen isn't "only one way to do it". If it were, we
wouldn't need iterators, list comps or for loops,
because they can all be handled with a while loop (at
various costs of efficiency, clarity or obviousness).

del L[:] works, but unless you are Dutch, it fails the
obviousness test. It also fails the introspection test:
neither dir(list) nor help(list) make it easy to
discover how to empty a list. In my opinion, the
primary advantage for a clear() method would be that it
is self-documenting.

--
Steven.

Steven D'Aprano, Feb 7, 2006
14. ### Fredrik LundhGuest

Steven D'Aprano wrote:

> > so we can have three ways to do the same thing? the right way to
> > nuke a sequence is to do "del L[:]". this is explained in Python 101.

>
> The Zen isn't "only one way to do it". If it were, we
> wouldn't need iterators, list comps or for loops,
> because they can all be handled with a while loop (at
> various costs of efficiency, clarity or obviousness).
>
> del L[:] works, but unless you are Dutch, it fails the
> obviousness test.

unless you read some documentation, that is. del on sequences
and mappings is a pretty fundamental part of Python. so are slicings.

both are things that you're likely to need and learn long before you
end up in situation where you need to be able to clear an aliased
sequence.

> It also fails the introspection test:

so does "print L".

</F>

Fredrik Lundh, Feb 7, 2006
15. ### Raymond HettingerGuest

[Steven D'Aprano]
> > The Zen isn't "only one way to do it". If it were, we
> > wouldn't need iterators, list comps or for loops,
> > because they can all be handled with a while loop (at
> > various costs of efficiency, clarity or obviousness).
> >
> > del L[:] works, but unless you are Dutch, it fails the
> > obviousness test.

[Fredrik Lundh]
> unless you read some documentation, that is. del on sequences
> and mappings is a pretty fundamental part of Python. so are slicings.
>
> both are things that you're likely to need and learn long before you
> end up in situation where you need to be able to clear an aliased
> sequence.

Fred is exactly correct. Slicing is absolutely basic to Python.
Accordingly, it gets covered right at the beginning of the tutorial
(section 3.1). Likewise, the del keyword is fundamental -- if you
can't get, set, and del, then you need to go back to collections
school.

Also specious is the suggestion that dir(L) or help(L) is useless. The
entries for the __delitem__ and __delslice__ methods are no more hidden
than for __getitem__ or __add__. The help entry goes a step further
and shows the translation to del x[y] and del x[i:j].

While the sentiment behind the list.clear() suggestion is noble, it is
an exercise in futility to design a language around the presumption
that other programmers are incapable of learning the basics of the
language.

There was a pithy Tim Peters quotation to the effect that he was
unpersuaded by language proposals predicated on some hypothetical
average programmer not being smart enough to understand something that
the rest of us find to be basic.

Raymond Hettinger

Raymond Hettinger, Feb 7, 2006
16. ### Raymond HettingerGuest

[]
> In my programs I have seen that there is another practical difference
> between version 1 and 3:
> (1) mylist[:] = []
> (3) mylist = []
> If you create a big mylist again and again many times, the version 1
> uses the memory more efficiently (probably less work for the garbage
> collector), and the program can be (quite) faster (this is true in some
> implementations different from CPython too).

There should be almost no difference in runtime between the two. The
underlying CPython implementation caches list objects and is usually
able to create the new list without any calls to the memory allocator.
Likewise, garbage collection performs the same for both -- any objects
left with no references are immediately freed and any with only
circular references get freed when the collector runs.

The speed-up you observed likely occured with different code. For
instance, given a large list, you can typically update or replace
individual elements faster than you can build-up a new list:

L = [0] * 1000 # starting list
for i in xrange(len(L)):
L += 1

beats:

L = [0] * 1000 # starting list
L = [L+1 for i in xrange(len(L))]

Raymond

Raymond Hettinger, Feb 7, 2006
17. ### Ben SizerGuest

Raymond Hettinger wrote:
> [Steven D'Aprano]
> > > The Zen isn't "only one way to do it". If it were, we
> > > wouldn't need iterators, list comps or for loops,
> > > because they can all be handled with a while loop (at
> > > various costs of efficiency, clarity or obviousness).
> > >
> > > del L[:] works, but unless you are Dutch, it fails the
> > > obviousness test.

>
> [Fredrik Lundh]
> > unless you read some documentation, that is. del on sequences
> > and mappings is a pretty fundamental part of Python. so are slicings.
> >
> > both are things that you're likely to need and learn long before you
> > end up in situation where you need to be able to clear an aliased
> > sequence.

>
> Fred is exactly correct. Slicing is absolutely basic to Python.
> Accordingly, it gets covered right at the beginning of the tutorial
> (section 3.1).

Yes, right after UTF encoding details, complex numbers, and various
mentions of shell scripts. I don't want to criticise the hard work that
went into making the tutorial but let's not pretend it's the epitome of
documentation or even necessary the most obvious reference for users.

> Likewise, the del keyword is fundamental -- if you
> can't get, set, and del, then you need to go back to collections
> school.

I have hardly used the del keyword in several years of coding in
Python. Why should it magically spring to mind in this occasion?
Similarly I hardly ever find myself using slices, never mind in a
mutable context.

del L[:] is not obvious, especially given the existence of clear() in
dictionaries. I'm not necessarily requesting a clear() method, but I am
requesting a little more understanding towards those who expected one.
The list interface is full of redundant convenience methods, so one
more would hardly be a surprise or an unreasonable thing for people to
expect. Again we unfortunately have a bit of an attitude problem
towards anyone posting here that doesn't know whatever the experts
think is obvious.

--
Ben Sizer

Ben Sizer, Feb 7, 2006
18. ### Steve HoldenGuest

Raymond Hettinger wrote:
> [Steven D'Aprano]

[...]
>
> While the sentiment behind the list.clear() suggestion is noble, it is
> an exercise in futility to design a language around the presumption
> that other programmers are incapable of learning the basics of the
> language.
>

+1 QOTW

> There was a pithy Tim Peters quotation to the effect that he was
> unpersuaded by language proposals predicated on some hypothetical
> average programmer not being smart enough to understand something that
> the rest of us find to be basic.
>

Well, for those who can't find it you just provided a fairly pithy
Raymond Hettinger quote

regards
Steve
--
Steve Holden +44 150 684 7255 +1 800 494 3119
Holden Web LLC www.holdenweb.com
PyCon TX 2006 www.python.org/pycon/

Steve Holden, Feb 7, 2006
19. ### ArthurGuest

Raymond Hettinger wrote:
>
> There was a pithy Tim Peters quotation to the effect that he was
> unpersuaded by language proposals predicated on some hypothetical
> average programmer not being smart enough to understand something that
> the rest of us find to be basic.

Peters pithy

As someone coming to Python as a well less than the average programmer,
I agree wholeheartedly with Mr. Peters.

My ability to grok Python to any extent, from that starting point, was
based on an understanding that it was my responsibility to come to
Python, not it to me. And concerted, almost obsessive effort, And time.

OTOH - because of exactly these factors - I have actively resented and
resisted promulgation of the Python is Easy meme.

Pity that this stance has separated me from a sense of belonging to the
community, which seems to so highly value that meme.

Art

Arthur, Feb 7, 2006
20. ### ArthurGuest

Arthur wrote:
>
> My ability to grok Python to any extent, from that starting point, was
> based on an understanding that it was my responsibility to come to
> Python, not it to me. And concerted, almost obsessive effort, And time.
>
> OTOH - because of exactly these factors - I have actively resented and
> resisted promulgation of the Python is Easy meme.
>
> Pity that this stance has separated me from a sense of belonging to the
> community, which seems to so highly value that meme.

What Python *is*, to me, is a game worth playing. Not something I come
across all that often. And easy games, almost invariably, are not.

Art

Arthur, Feb 7, 2006