Question about idioms for clearing a list

S

Steven Watanabe

I know that the standard idioms for clearing a list are:

(1) mylist[:] = []
(2) del mylist[:]

I guess I'm not in the "slicing frame of mind", as someone put it, but
can someone explain what the difference is between these and:

(3) mylist = []

Why are (1) and (2) preferred? I think the first two are changing the
list in-place, but why is that better? Isn't the end result the same?

Thanks in advance.
 
D

Diez B. Roggisch

Steven said:
I know that the standard idioms for clearing a list are:

(1) mylist[:] = []
(2) del mylist[:]

I guess I'm not in the "slicing frame of mind", as someone put it, but
can someone explain what the difference is between these and:

(3) mylist = []

Why are (1) and (2) preferred? I think the first two are changing the
list in-place, but why is that better? Isn't the end result the same?

No. Consider this simple example:

class Foo(object):
def __init__(self, all_my_thingies):
self.all_my_thingies = all_my_thingies


things = [1,2,3,4,5]

f = Foo(things)

things = [] # I've been robbed

print f.all_my_thingies # or not?


The reason is that l = [] just rebinds a new object (a list, but it could be
anything) to a name, while l[:] = [] will alter the object _referred_ to by
l. That is a HUGE difference!


Diez
 
X

Xavier Morel

Steven said:
I know that the standard idioms for clearing a list are:

(1) mylist[:] = []
(2) del mylist[:]

I guess I'm not in the "slicing frame of mind", as someone put it, but
can someone explain what the difference is between these and:

(3) mylist = []

Why are (1) and (2) preferred? I think the first two are changing the
list in-place, but why is that better? Isn't the end result the same?

Thanks in advance.

The solution (1) and (2) clear the content of the list (replace the
content of the list by an empty list for (1), or delete the content of
the list for (2) while the solution (3) creates a new list and binds the
old name to the new list. This means that you don't actually modify
(clear) the initial list.

Just try it out:
>>> a = range(10)
>>> a [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
>>> b = a
>>> b [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
>>> c = a
>>> c [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
>>> a.append(10) # assert that a, b and c point to the same list
>>> a [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
>>> b [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
>>> c [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
>>> b = []
>>> b []
>>> a [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
>>> del c[:]
>>> c []
>>> a []
>>>

Here we can see that using method (3) on b didn't modify the content of
a (even though we asserted that they were the same list).
Using method (2) on c, the other hand, did modify the content of a.

This is because method (3) on b actually created a new list and bound
"b" (as a name) to that new list, without modifying the list "b" used to
be bound to (which is the one "a" and "c" are still bound to).
 
B

bearophileHUGS

Diez B. Roggisch:
The reason is that l = [] just rebinds a new object (a list, but it could be
anything) to a name, while l[:] = [] will alter the object _referred_ to by
l. That is a HUGE difference!

In my programs I have seen that there is another practical difference
between version 1 and 3:
(1) mylist[:] = []
(3) mylist = []
If you create a big mylist again and again many times, the version 1
uses the memory more efficiently (probably less work for the garbage
collector), and the program can be (quite) faster (this is true in some
implementations different from CPython too).

Bye,
bearophile
 
W

Will McGugan

Steven said:
I know that the standard idioms for clearing a list are:

(1) mylist[:] = []
(2) del mylist[:]

I guess I'm not in the "slicing frame of mind", as someone put it, but
can someone explain what the difference is between these and:

(3) mylist = []

Why are (1) and (2) preferred? I think the first two are changing the
list in-place, but why is that better? Isn't the end result the same?

I'm wondering why there is no 'clear' for lists. It feels like a common
operation for mutable containers. :-/


Will McGugan
 
S

Steve Holden

Will said:
Steven said:
I know that the standard idioms for clearing a list are:

(1) mylist[:] = []
(2) del mylist[:]

I guess I'm not in the "slicing frame of mind", as someone put it, but
can someone explain what the difference is between these and:

(3) mylist = []

Why are (1) and (2) preferred? I think the first two are changing the
list in-place, but why is that better? Isn't the end result the same?


I'm wondering why there is no 'clear' for lists. It feels like a common
operation for mutable containers. :-/
Because it's just as easy to create and assign a new empty list (and
have the old unused one garbage collected).

l = []

is all you need!

regards
Steve
 
S

Steven D'Aprano

I'm wondering why there is no 'clear' for lists. It feels like a common
operation for mutable containers. :-/
Because it's just as easy to create and assign a new empty list (and
have the old unused one garbage collected).

l = []

is all you need!

Not so. If that logic were correct, then dicts wouldn't need a clear
method either, because you could just assign a new empty dict with d = {}.

But your own sentence tells us why this is not sufficient: because you
aren't emptying the list, you are reassigning (rebinding) the name. The
old list still exists, and there is no guarantee that it will be garbage
collected, because there is no guarantee that it isn't in use somewhere
else:

L = [0,1,2]
D = {"key": L}
L = [] # rebinds the name L, but the list instance still exists

Perhaps it is arguable that there is no need for a clear method because
L[:] = [] is so easy to do. Personally, while I agree that it is easy, it
is hardly intuitive or obvious, and I too would prefer an explicit clear
method for mutable sequences.
 
D

Dan Sommers

I'm wondering why there is no 'clear' for lists. It feels like a common
operation for mutable containers. :-/
Because it's just as easy to create and assign a new empty list (and
have the old unused one garbage collected).

l = []

is all you need!
Not so. If that logic were correct, then dicts wouldn't need a clear
method either, because you could just assign a new empty dict with d = {}.
But your own sentence tells us why this is not sufficient: because you
aren't emptying the list, you are reassigning (rebinding) the name. The
old list still exists, and there is no guarantee that it will be garbage
collected, because there is no guarantee that it isn't in use somewhere
else:
L = [0,1,2]
D = {"key": L}
L = [] # rebinds the name L, but the list instance still exists

That is a red herring. Consider this:

L = [object(), object()]
O = L[1]
L = [] # or insert your favorite list-clearing/emptying statement here

What damage is done now that O is still referring to one of the items
that used to be in L?

The trouble begins when references to "the list to which L refers" end
up somewhere else. Then we have to wonder if rebinding L will leave
some other block of code with an outdated list.

Regards,
Dan
 
P

Peter Hansen

Steven said:
Perhaps it is arguable that there is no need for a clear method because
L[:] = [] is so easy to do. Personally, while I agree that it is easy, it
is hardly intuitive or obvious, and I too would prefer an explicit clear
method for mutable sequences.

Possibly another case where "patches are welcome"...
 
F

Fredrik Lundh

Peter said:
Perhaps it is arguable that there is no need for a clear method because
L[:] = [] is so easy to do. Personally, while I agree that it is easy, it
is hardly intuitive or obvious, and I too would prefer an explicit clear
method for mutable sequences.

Possibly another case where "patches are welcome"...

so we can have three ways to do the same thing? the right way to
nuke a sequence is to do "del L[:]". this is explained in Python 101.

(del doesn't work on dictionaries)

</F>
 
M

Mel Wilson

Fredrik said:
(del doesn't work on dictionaries)

.... or rather [:] doesn't work on dictionaries ...

Python 2.4.2 (#1, Jan 23 2006, 21:24:54)
[GCC 3.3.4] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> d={'a':1, 'b':2, 'c':3}
>>> print d {'a': 1, 'c': 3, 'b': 2}
>>> del d['b']
>>> print d {'a': 1, 'c': 3}
>>> del d[:]
Traceback (most recent call last):

Regards. Mel.
 
S

Steven D'Aprano

I'm wondering why there is no 'clear' for lists. It feels like a common
operation for mutable containers. :-/

Because it's just as easy to create and assign a new empty list (and
have the old unused one garbage collected).

l = []

is all you need!
Not so. If that logic were correct, then dicts wouldn't need a clear
method either, because you could just assign a new empty dict with d = {}.
But your own sentence tells us why this is not sufficient: because you
aren't emptying the list, you are reassigning (rebinding) the name. The
old list still exists, and there is no guarantee that it will be garbage
collected, because there is no guarantee that it isn't in use somewhere
else:
L = [0,1,2]
D = {"key": L}
L = [] # rebinds the name L, but the list instance still exists

That is a red herring. Consider this:

L = [object(), object()]
O = L[1]
L = [] # or insert your favorite list-clearing/emptying statement here

What damage is done now that O is still referring to one of the items
that used to be in L?

What relevance is this? If there is one and only one reference to the list
L, then it will be garbage collected when L is rebound. I never denied
that. I pointed out that, in the general case, you may have multiple
references to the list (not all of which are bound to names), and
rebinding the name L will NOT have the side-effect of clearing the list.

The trouble begins when references to "the list to which L refers" end
up somewhere else. Then we have to wonder if rebinding L will leave
some other block of code with an outdated list.

Precisely, just as my example shows.
 
S

Steven D'Aprano

Fredrik said:
Peter Hansen wrote:

Perhaps it is arguable that there is no need for a clear method because
L[:] = [] is so easy to do. Personally, while I agree that it is easy, it
is hardly intuitive or obvious, and I too would prefer an explicit clear
method for mutable sequences.

Possibly another case where "patches are welcome"...


so we can have three ways to do the same thing? the right way to
nuke a sequence is to do "del L[:]". this is explained in Python 101.

The Zen isn't "only one way to do it". If it were, we
wouldn't need iterators, list comps or for loops,
because they can all be handled with a while loop (at
various costs of efficiency, clarity or obviousness).

del L[:] works, but unless you are Dutch, it fails the
obviousness test. It also fails the introspection test:
neither dir(list) nor help(list) make it easy to
discover how to empty a list. In my opinion, the
primary advantage for a clear() method would be that it
is self-documenting.
 
F

Fredrik Lundh

Steven said:
so we can have three ways to do the same thing? the right way to
nuke a sequence is to do "del L[:]". this is explained in Python 101.

The Zen isn't "only one way to do it". If it were, we
wouldn't need iterators, list comps or for loops,
because they can all be handled with a while loop (at
various costs of efficiency, clarity or obviousness).

del L[:] works, but unless you are Dutch, it fails the
obviousness test.

unless you read some documentation, that is. del on sequences
and mappings is a pretty fundamental part of Python. so are slicings.

both are things that you're likely to need and learn long before you
end up in situation where you need to be able to clear an aliased
sequence.
It also fails the introspection test:

so does "print L".

</F>
 
R

Raymond Hettinger

[Steven D'Aprano]
The Zen isn't "only one way to do it". If it were, we
wouldn't need iterators, list comps or for loops,
because they can all be handled with a while loop (at
various costs of efficiency, clarity or obviousness).

del L[:] works, but unless you are Dutch, it fails the
obviousness test.

[Fredrik Lundh]
unless you read some documentation, that is. del on sequences
and mappings is a pretty fundamental part of Python. so are slicings.

both are things that you're likely to need and learn long before you
end up in situation where you need to be able to clear an aliased
sequence.

Fred is exactly correct. Slicing is absolutely basic to Python.
Accordingly, it gets covered right at the beginning of the tutorial
(section 3.1). Likewise, the del keyword is fundamental -- if you
can't get, set, and del, then you need to go back to collections
school.

Also specious is the suggestion that dir(L) or help(L) is useless. The
entries for the __delitem__ and __delslice__ methods are no more hidden
than for __getitem__ or __add__. The help entry goes a step further
and shows the translation to del x[y] and del x[i:j].

While the sentiment behind the list.clear() suggestion is noble, it is
an exercise in futility to design a language around the presumption
that other programmers are incapable of learning the basics of the
language.

There was a pithy Tim Peters quotation to the effect that he was
unpersuaded by language proposals predicated on some hypothetical
average programmer not being smart enough to understand something that
the rest of us find to be basic.


Raymond Hettinger
 
R

Raymond Hettinger

[[email protected]]
In my programs I have seen that there is another practical difference
between version 1 and 3:
(1) mylist[:] = []
(3) mylist = []
If you create a big mylist again and again many times, the version 1
uses the memory more efficiently (probably less work for the garbage
collector), and the program can be (quite) faster (this is true in some
implementations different from CPython too).

There should be almost no difference in runtime between the two. The
underlying CPython implementation caches list objects and is usually
able to create the new list without any calls to the memory allocator.
Likewise, garbage collection performs the same for both -- any objects
left with no references are immediately freed and any with only
circular references get freed when the collector runs.

The speed-up you observed likely occured with different code. For
instance, given a large list, you can typically update or replace
individual elements faster than you can build-up a new list:

L = [0] * 1000 # starting list
for i in xrange(len(L)):
L += 1

beats:

L = [0] * 1000 # starting list
L = [L+1 for i in xrange(len(L))]


Raymond
 
B

Ben Sizer

Raymond said:
[Steven D'Aprano]
The Zen isn't "only one way to do it". If it were, we
wouldn't need iterators, list comps or for loops,
because they can all be handled with a while loop (at
various costs of efficiency, clarity or obviousness).

del L[:] works, but unless you are Dutch, it fails the
obviousness test.

[Fredrik Lundh]
unless you read some documentation, that is. del on sequences
and mappings is a pretty fundamental part of Python. so are slicings.

both are things that you're likely to need and learn long before you
end up in situation where you need to be able to clear an aliased
sequence.

Fred is exactly correct. Slicing is absolutely basic to Python.
Accordingly, it gets covered right at the beginning of the tutorial
(section 3.1).

Yes, right after UTF encoding details, complex numbers, and various
mentions of shell scripts. I don't want to criticise the hard work that
went into making the tutorial but let's not pretend it's the epitome of
documentation or even necessary the most obvious reference for users.
Likewise, the del keyword is fundamental -- if you
can't get, set, and del, then you need to go back to collections
school.

I have hardly used the del keyword in several years of coding in
Python. Why should it magically spring to mind in this occasion?
Similarly I hardly ever find myself using slices, never mind in a
mutable context.

del L[:] is not obvious, especially given the existence of clear() in
dictionaries. I'm not necessarily requesting a clear() method, but I am
requesting a little more understanding towards those who expected one.
The list interface is full of redundant convenience methods, so one
more would hardly be a surprise or an unreasonable thing for people to
expect. Again we unfortunately have a bit of an attitude problem
towards anyone posting here that doesn't know whatever the experts
think is obvious.
 
S

Steve Holden

Raymond said:
[Steven D'Aprano] [...]

While the sentiment behind the list.clear() suggestion is noble, it is
an exercise in futility to design a language around the presumption
that other programmers are incapable of learning the basics of the
language.
+1 QOTW
There was a pithy Tim Peters quotation to the effect that he was
unpersuaded by language proposals predicated on some hypothetical
average programmer not being smart enough to understand something that
the rest of us find to be basic.
Well, for those who can't find it you just provided a fairly pithy
Raymond Hettinger quote :)

regards
Steve
 
A

Arthur

Raymond said:
There was a pithy Tim Peters quotation to the effect that he was
unpersuaded by language proposals predicated on some hypothetical
average programmer not being smart enough to understand something that
the rest of us find to be basic.

Peters pithy ;)

As someone coming to Python as a well less than the average programmer,
I agree wholeheartedly with Mr. Peters.

My ability to grok Python to any extent, from that starting point, was
based on an understanding that it was my responsibility to come to
Python, not it to me. And concerted, almost obsessive effort, And time.

OTOH - because of exactly these factors - I have actively resented and
resisted promulgation of the Python is Easy meme.

Pity that this stance has separated me from a sense of belonging to the
community, which seems to so highly value that meme.

Art
 
A

Arthur

Arthur said:
My ability to grok Python to any extent, from that starting point, was
based on an understanding that it was my responsibility to come to
Python, not it to me. And concerted, almost obsessive effort, And time.

OTOH - because of exactly these factors - I have actively resented and
resisted promulgation of the Python is Easy meme.

Pity that this stance has separated me from a sense of belonging to the
community, which seems to so highly value that meme.

What Python *is*, to me, is a game worth playing. Not something I come
across all that often. And easy games, almost invariably, are not.

Art
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

No members online now.

Forum statistics

Threads
473,731
Messages
2,569,432
Members
44,835
Latest member
KetoRushACVBuy

Latest Threads

Top