Question about idioms for clearing a list

Discussion in 'Python' started by Steven Watanabe, Jan 31, 2006.

  1. I know that the standard idioms for clearing a list are:

    (1) mylist[:] = []
    (2) del mylist[:]

    I guess I'm not in the "slicing frame of mind", as someone put it, but
    can someone explain what the difference is between these and:

    (3) mylist = []

    Why are (1) and (2) preferred? I think the first two are changing the
    list in-place, but why is that better? Isn't the end result the same?

    Thanks in advance.
    --
    Steven.
     
    Steven Watanabe, Jan 31, 2006
    #1
    1. Advertising

  2. Steven Watanabe wrote:

    > I know that the standard idioms for clearing a list are:
    >
    > (1) mylist[:] = []
    > (2) del mylist[:]
    >
    > I guess I'm not in the "slicing frame of mind", as someone put it, but
    > can someone explain what the difference is between these and:
    >
    > (3) mylist = []
    >
    > Why are (1) and (2) preferred? I think the first two are changing the
    > list in-place, but why is that better? Isn't the end result the same?


    No. Consider this simple example:

    class Foo(object):
    def __init__(self, all_my_thingies):
    self.all_my_thingies = all_my_thingies


    things = [1,2,3,4,5]

    f = Foo(things)

    things = [] # I've been robbed

    print f.all_my_thingies # or not?


    The reason is that l = [] just rebinds a new object (a list, but it could be
    anything) to a name, while l[:] = [] will alter the object _referred_ to by
    l. That is a HUGE difference!


    Diez
     
    Diez B. Roggisch, Jan 31, 2006
    #2
    1. Advertising

  3. Steven Watanabe

    Xavier Morel Guest

    Steven Watanabe wrote:
    > I know that the standard idioms for clearing a list are:
    >
    > (1) mylist[:] = []
    > (2) del mylist[:]
    >
    > I guess I'm not in the "slicing frame of mind", as someone put it, but
    > can someone explain what the difference is between these and:
    >
    > (3) mylist = []
    >
    > Why are (1) and (2) preferred? I think the first two are changing the
    > list in-place, but why is that better? Isn't the end result the same?
    >
    > Thanks in advance.
    > --
    > Steven.


    The solution (1) and (2) clear the content of the list (replace the
    content of the list by an empty list for (1), or delete the content of
    the list for (2) while the solution (3) creates a new list and binds the
    old name to the new list. This means that you don't actually modify
    (clear) the initial list.

    Just try it out:

    >>> a = range(10)
    >>> a

    [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
    >>> b = a
    >>> b

    [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
    >>> c = a
    >>> c

    [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
    >>> a.append(10) # assert that a, b and c point to the same list
    >>> a

    [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
    >>> b

    [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
    >>> c

    [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
    >>> b = []
    >>> b

    []
    >>> a

    [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
    >>> del c[:]
    >>> c

    []
    >>> a

    []
    >>>


    Here we can see that using method (3) on b didn't modify the content of
    a (even though we asserted that they were the same list).
    Using method (2) on c, the other hand, did modify the content of a.

    This is because method (3) on b actually created a new list and bound
    "b" (as a name) to that new list, without modifying the list "b" used to
    be bound to (which is the one "a" and "c" are still bound to).
     
    Xavier Morel, Jan 31, 2006
    #3
  4. Steven Watanabe

    Guest

    Diez B. Roggisch:
    > The reason is that l = [] just rebinds a new object (a list, but it could be
    > anything) to a name, while l[:] = [] will alter the object _referred_ to by
    > l. That is a HUGE difference!


    In my programs I have seen that there is another practical difference
    between version 1 and 3:
    (1) mylist[:] = []
    (3) mylist = []
    If you create a big mylist again and again many times, the version 1
    uses the memory more efficiently (probably less work for the garbage
    collector), and the program can be (quite) faster (this is true in some
    implementations different from CPython too).

    Bye,
    bearophile
     
    , Jan 31, 2006
    #4
  5. Steven Watanabe

    Will McGugan Guest

    Steven Watanabe wrote:
    > I know that the standard idioms for clearing a list are:
    >
    > (1) mylist[:] = []
    > (2) del mylist[:]
    >
    > I guess I'm not in the "slicing frame of mind", as someone put it, but
    > can someone explain what the difference is between these and:
    >
    > (3) mylist = []
    >
    > Why are (1) and (2) preferred? I think the first two are changing the
    > list in-place, but why is that better? Isn't the end result the same?


    I'm wondering why there is no 'clear' for lists. It feels like a common
    operation for mutable containers. :-/


    Will McGugan
     
    Will McGugan, Feb 6, 2006
    #5
  6. Steven Watanabe

    Steve Holden Guest

    Will McGugan wrote:
    > Steven Watanabe wrote:
    >
    >>I know that the standard idioms for clearing a list are:
    >>
    >> (1) mylist[:] = []
    >> (2) del mylist[:]
    >>
    >>I guess I'm not in the "slicing frame of mind", as someone put it, but
    >>can someone explain what the difference is between these and:
    >>
    >> (3) mylist = []
    >>
    >>Why are (1) and (2) preferred? I think the first two are changing the
    >>list in-place, but why is that better? Isn't the end result the same?

    >
    >
    > I'm wondering why there is no 'clear' for lists. It feels like a common
    > operation for mutable containers. :-/
    >

    Because it's just as easy to create and assign a new empty list (and
    have the old unused one garbage collected).

    l = []

    is all you need!

    regards
    Steve
    --
    Steve Holden +44 150 684 7255 +1 800 494 3119
    Holden Web LLC www.holdenweb.com
    PyCon TX 2006 www.python.org/pycon/
     
    Steve Holden, Feb 6, 2006
    #6
  7. On Mon, 06 Feb 2006 13:35:10 +0000, Steve Holden wrote:

    >> I'm wondering why there is no 'clear' for lists. It feels like a common
    >> operation for mutable containers. :-/
    >>

    > Because it's just as easy to create and assign a new empty list (and
    > have the old unused one garbage collected).
    >
    > l = []
    >
    > is all you need!


    Not so. If that logic were correct, then dicts wouldn't need a clear
    method either, because you could just assign a new empty dict with d = {}.

    But your own sentence tells us why this is not sufficient: because you
    aren't emptying the list, you are reassigning (rebinding) the name. The
    old list still exists, and there is no guarantee that it will be garbage
    collected, because there is no guarantee that it isn't in use somewhere
    else:

    L = [0,1,2]
    D = {"key": L}
    L = [] # rebinds the name L, but the list instance still exists

    Perhaps it is arguable that there is no need for a clear method because
    L[:] = [] is so easy to do. Personally, while I agree that it is easy, it
    is hardly intuitive or obvious, and I too would prefer an explicit clear
    method for mutable sequences.


    --
    Steven.
     
    Steven D'Aprano, Feb 6, 2006
    #7
  8. Steven Watanabe

    Dan Sommers Guest

    On Tue, 07 Feb 2006 01:01:43 +1100,
    Steven D'Aprano <> wrote:

    > On Mon, 06 Feb 2006 13:35:10 +0000, Steve Holden wrote:
    >>> I'm wondering why there is no 'clear' for lists. It feels like a common
    >>> operation for mutable containers. :-/
    >>>

    >> Because it's just as easy to create and assign a new empty list (and
    >> have the old unused one garbage collected).
    >>
    >> l = []
    >>
    >> is all you need!


    > Not so. If that logic were correct, then dicts wouldn't need a clear
    > method either, because you could just assign a new empty dict with d = {}.


    > But your own sentence tells us why this is not sufficient: because you
    > aren't emptying the list, you are reassigning (rebinding) the name. The
    > old list still exists, and there is no guarantee that it will be garbage
    > collected, because there is no guarantee that it isn't in use somewhere
    > else:


    > L = [0,1,2]
    > D = {"key": L}
    > L = [] # rebinds the name L, but the list instance still exists


    That is a red herring. Consider this:

    L = [object(), object()]
    O = L[1]
    L = [] # or insert your favorite list-clearing/emptying statement here

    What damage is done now that O is still referring to one of the items
    that used to be in L?

    The trouble begins when references to "the list to which L refers" end
    up somewhere else. Then we have to wonder if rebinding L will leave
    some other block of code with an outdated list.

    Regards,
    Dan

    --
    Dan Sommers
    <http://www.tombstonezero.net/dan/>
     
    Dan Sommers, Feb 6, 2006
    #8
  9. Steven Watanabe

    Peter Hansen Guest

    Steven D'Aprano wrote:
    > Perhaps it is arguable that there is no need for a clear method because
    > L[:] = [] is so easy to do. Personally, while I agree that it is easy, it
    > is hardly intuitive or obvious, and I too would prefer an explicit clear
    > method for mutable sequences.


    Possibly another case where "patches are welcome"...
     
    Peter Hansen, Feb 6, 2006
    #9
  10. Peter Hansen wrote:

    > > Perhaps it is arguable that there is no need for a clear method because
    > > L[:] = [] is so easy to do. Personally, while I agree that it is easy, it
    > > is hardly intuitive or obvious, and I too would prefer an explicit clear
    > > method for mutable sequences.

    >
    > Possibly another case where "patches are welcome"...


    so we can have three ways to do the same thing? the right way to
    nuke a sequence is to do "del L[:]". this is explained in Python 101.

    (del doesn't work on dictionaries)

    </F>
     
    Fredrik Lundh, Feb 6, 2006
    #10
  11. Steven Watanabe

    Mel Wilson Guest

    Fredrik Lundh wrote:

    > (del doesn't work on dictionaries)


    .... or rather [:] doesn't work on dictionaries ...

    Python 2.4.2 (#1, Jan 23 2006, 21:24:54)
    [GCC 3.3.4] on linux2
    Type "help", "copyright", "credits" or "license" for more information.
    >>> d={'a':1, 'b':2, 'c':3}
    >>> print d

    {'a': 1, 'c': 3, 'b': 2}
    >>> del d['b']
    >>> print d

    {'a': 1, 'c': 3}
    >>> del d[:]

    Traceback (most recent call last):
    File "<stdin>", line 1, in ?
    TypeError: unhashable type
    >>>


    Regards. Mel.
     
    Mel Wilson, Feb 6, 2006
    #11
  12. On Mon, 06 Feb 2006 09:39:32 -0500, Dan Sommers wrote:

    > On Tue, 07 Feb 2006 01:01:43 +1100,
    > Steven D'Aprano <> wrote:
    >
    >> On Mon, 06 Feb 2006 13:35:10 +0000, Steve Holden wrote:
    >>>> I'm wondering why there is no 'clear' for lists. It feels like a common
    >>>> operation for mutable containers. :-/
    >>>>
    >>> Because it's just as easy to create and assign a new empty list (and
    >>> have the old unused one garbage collected).
    >>>
    >>> l = []
    >>>
    >>> is all you need!

    >
    >> Not so. If that logic were correct, then dicts wouldn't need a clear
    >> method either, because you could just assign a new empty dict with d = {}.

    >
    >> But your own sentence tells us why this is not sufficient: because you
    >> aren't emptying the list, you are reassigning (rebinding) the name. The
    >> old list still exists, and there is no guarantee that it will be garbage
    >> collected, because there is no guarantee that it isn't in use somewhere
    >> else:

    >
    >> L = [0,1,2]
    >> D = {"key": L}
    >> L = [] # rebinds the name L, but the list instance still exists

    >
    > That is a red herring. Consider this:
    >
    > L = [object(), object()]
    > O = L[1]
    > L = [] # or insert your favorite list-clearing/emptying statement here
    >
    > What damage is done now that O is still referring to one of the items
    > that used to be in L?


    What relevance is this? If there is one and only one reference to the list
    L, then it will be garbage collected when L is rebound. I never denied
    that. I pointed out that, in the general case, you may have multiple
    references to the list (not all of which are bound to names), and
    rebinding the name L will NOT have the side-effect of clearing the list.


    > The trouble begins when references to "the list to which L refers" end
    > up somewhere else. Then we have to wonder if rebinding L will leave
    > some other block of code with an outdated list.


    Precisely, just as my example shows.


    --
    Steven.
     
    Steven D'Aprano, Feb 6, 2006
    #12
  13. Fredrik Lundh wrote:

    > Peter Hansen wrote:
    >
    >
    >>>Perhaps it is arguable that there is no need for a clear method because
    >>>L[:] = [] is so easy to do. Personally, while I agree that it is easy, it
    >>>is hardly intuitive or obvious, and I too would prefer an explicit clear
    >>>method for mutable sequences.

    >>
    >>Possibly another case where "patches are welcome"...

    >
    >
    > so we can have three ways to do the same thing? the right way to
    > nuke a sequence is to do "del L[:]". this is explained in Python 101.


    The Zen isn't "only one way to do it". If it were, we
    wouldn't need iterators, list comps or for loops,
    because they can all be handled with a while loop (at
    various costs of efficiency, clarity or obviousness).

    del L[:] works, but unless you are Dutch, it fails the
    obviousness test. It also fails the introspection test:
    neither dir(list) nor help(list) make it easy to
    discover how to empty a list. In my opinion, the
    primary advantage for a clear() method would be that it
    is self-documenting.



    --
    Steven.
     
    Steven D'Aprano, Feb 7, 2006
    #13
  14. Steven D'Aprano wrote:

    > > so we can have three ways to do the same thing? the right way to
    > > nuke a sequence is to do "del L[:]". this is explained in Python 101.

    >
    > The Zen isn't "only one way to do it". If it were, we
    > wouldn't need iterators, list comps or for loops,
    > because they can all be handled with a while loop (at
    > various costs of efficiency, clarity or obviousness).
    >
    > del L[:] works, but unless you are Dutch, it fails the
    > obviousness test.


    unless you read some documentation, that is. del on sequences
    and mappings is a pretty fundamental part of Python. so are slicings.

    both are things that you're likely to need and learn long before you
    end up in situation where you need to be able to clear an aliased
    sequence.

    > It also fails the introspection test:


    so does "print L".

    </F>
     
    Fredrik Lundh, Feb 7, 2006
    #14
  15. [Steven D'Aprano]
    > > The Zen isn't "only one way to do it". If it were, we
    > > wouldn't need iterators, list comps or for loops,
    > > because they can all be handled with a while loop (at
    > > various costs of efficiency, clarity or obviousness).
    > >
    > > del L[:] works, but unless you are Dutch, it fails the
    > > obviousness test.


    [Fredrik Lundh]
    > unless you read some documentation, that is. del on sequences
    > and mappings is a pretty fundamental part of Python. so are slicings.
    >
    > both are things that you're likely to need and learn long before you
    > end up in situation where you need to be able to clear an aliased
    > sequence.


    Fred is exactly correct. Slicing is absolutely basic to Python.
    Accordingly, it gets covered right at the beginning of the tutorial
    (section 3.1). Likewise, the del keyword is fundamental -- if you
    can't get, set, and del, then you need to go back to collections
    school.

    Also specious is the suggestion that dir(L) or help(L) is useless. The
    entries for the __delitem__ and __delslice__ methods are no more hidden
    than for __getitem__ or __add__. The help entry goes a step further
    and shows the translation to del x[y] and del x[i:j].

    While the sentiment behind the list.clear() suggestion is noble, it is
    an exercise in futility to design a language around the presumption
    that other programmers are incapable of learning the basics of the
    language.

    There was a pithy Tim Peters quotation to the effect that he was
    unpersuaded by language proposals predicated on some hypothetical
    average programmer not being smart enough to understand something that
    the rest of us find to be basic.


    Raymond Hettinger
     
    Raymond Hettinger, Feb 7, 2006
    #15
  16. []
    > In my programs I have seen that there is another practical difference
    > between version 1 and 3:
    > (1) mylist[:] = []
    > (3) mylist = []
    > If you create a big mylist again and again many times, the version 1
    > uses the memory more efficiently (probably less work for the garbage
    > collector), and the program can be (quite) faster (this is true in some
    > implementations different from CPython too).


    There should be almost no difference in runtime between the two. The
    underlying CPython implementation caches list objects and is usually
    able to create the new list without any calls to the memory allocator.
    Likewise, garbage collection performs the same for both -- any objects
    left with no references are immediately freed and any with only
    circular references get freed when the collector runs.

    The speed-up you observed likely occured with different code. For
    instance, given a large list, you can typically update or replace
    individual elements faster than you can build-up a new list:

    L = [0] * 1000 # starting list
    for i in xrange(len(L)):
    L += 1

    beats:

    L = [0] * 1000 # starting list
    L = [L+1 for i in xrange(len(L))]


    Raymond
     
    Raymond Hettinger, Feb 7, 2006
    #16
  17. Steven Watanabe

    Ben Sizer Guest

    Raymond Hettinger wrote:
    > [Steven D'Aprano]
    > > > The Zen isn't "only one way to do it". If it were, we
    > > > wouldn't need iterators, list comps or for loops,
    > > > because they can all be handled with a while loop (at
    > > > various costs of efficiency, clarity or obviousness).
    > > >
    > > > del L[:] works, but unless you are Dutch, it fails the
    > > > obviousness test.

    >
    > [Fredrik Lundh]
    > > unless you read some documentation, that is. del on sequences
    > > and mappings is a pretty fundamental part of Python. so are slicings.
    > >
    > > both are things that you're likely to need and learn long before you
    > > end up in situation where you need to be able to clear an aliased
    > > sequence.

    >
    > Fred is exactly correct. Slicing is absolutely basic to Python.
    > Accordingly, it gets covered right at the beginning of the tutorial
    > (section 3.1).


    Yes, right after UTF encoding details, complex numbers, and various
    mentions of shell scripts. I don't want to criticise the hard work that
    went into making the tutorial but let's not pretend it's the epitome of
    documentation or even necessary the most obvious reference for users.

    > Likewise, the del keyword is fundamental -- if you
    > can't get, set, and del, then you need to go back to collections
    > school.


    I have hardly used the del keyword in several years of coding in
    Python. Why should it magically spring to mind in this occasion?
    Similarly I hardly ever find myself using slices, never mind in a
    mutable context.

    del L[:] is not obvious, especially given the existence of clear() in
    dictionaries. I'm not necessarily requesting a clear() method, but I am
    requesting a little more understanding towards those who expected one.
    The list interface is full of redundant convenience methods, so one
    more would hardly be a surprise or an unreasonable thing for people to
    expect. Again we unfortunately have a bit of an attitude problem
    towards anyone posting here that doesn't know whatever the experts
    think is obvious.

    --
    Ben Sizer
     
    Ben Sizer, Feb 7, 2006
    #17
  18. Steven Watanabe

    Steve Holden Guest

    Raymond Hettinger wrote:
    > [Steven D'Aprano]

    [...]
    >
    > While the sentiment behind the list.clear() suggestion is noble, it is
    > an exercise in futility to design a language around the presumption
    > that other programmers are incapable of learning the basics of the
    > language.
    >

    +1 QOTW

    > There was a pithy Tim Peters quotation to the effect that he was
    > unpersuaded by language proposals predicated on some hypothetical
    > average programmer not being smart enough to understand something that
    > the rest of us find to be basic.
    >

    Well, for those who can't find it you just provided a fairly pithy
    Raymond Hettinger quote :)

    regards
    Steve
    --
    Steve Holden +44 150 684 7255 +1 800 494 3119
    Holden Web LLC www.holdenweb.com
    PyCon TX 2006 www.python.org/pycon/
     
    Steve Holden, Feb 7, 2006
    #18
  19. Steven Watanabe

    Arthur Guest

    Raymond Hettinger wrote:
    >
    > There was a pithy Tim Peters quotation to the effect that he was
    > unpersuaded by language proposals predicated on some hypothetical
    > average programmer not being smart enough to understand something that
    > the rest of us find to be basic.


    Peters pithy ;)

    As someone coming to Python as a well less than the average programmer,
    I agree wholeheartedly with Mr. Peters.

    My ability to grok Python to any extent, from that starting point, was
    based on an understanding that it was my responsibility to come to
    Python, not it to me. And concerted, almost obsessive effort, And time.

    OTOH - because of exactly these factors - I have actively resented and
    resisted promulgation of the Python is Easy meme.

    Pity that this stance has separated me from a sense of belonging to the
    community, which seems to so highly value that meme.

    Art
     
    Arthur, Feb 7, 2006
    #19
  20. Steven Watanabe

    Arthur Guest

    Arthur wrote:
    >
    > My ability to grok Python to any extent, from that starting point, was
    > based on an understanding that it was my responsibility to come to
    > Python, not it to me. And concerted, almost obsessive effort, And time.
    >
    > OTOH - because of exactly these factors - I have actively resented and
    > resisted promulgation of the Python is Easy meme.
    >
    > Pity that this stance has separated me from a sense of belonging to the
    > community, which seems to so highly value that meme.


    What Python *is*, to me, is a game worth playing. Not something I come
    across all that often. And easy games, almost invariably, are not.

    Art
     
    Arthur, Feb 7, 2006
    #20
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Eli Bendersky
    Replies:
    9
    Views:
    715
    Allan Herriman
    Apr 7, 2006
  2. Tad Marko

    Need help with Python class idioms

    Tad Marko, Jan 21, 2004, in forum: Python
    Replies:
    17
    Views:
    502
    engsol
    Jan 25, 2004
  3. Raymond Hettinger

    Thoughts on new vs traditional idioms

    Raymond Hettinger, Feb 29, 2004, in forum: Python
    Replies:
    7
    Views:
    357
    Michele Simionato
    Mar 1, 2004
  4. Delaney, Timothy C (Timothy)

    RE: Thoughts on new vs traditional idioms

    Delaney, Timothy C (Timothy), Mar 1, 2004, in forum: Python
    Replies:
    3
    Views:
    305
    Peter Otten
    Mar 2, 2004
  5. Ben Charrow

    Idioms and Anti-Idioms Question

    Ben Charrow, Jun 22, 2009, in forum: Python
    Replies:
    11
    Views:
    553
    Lawrence D'Oliveiro
    Jul 4, 2009
Loading...

Share This Page