list.clear() missing?!?

Discussion in 'Python' started by Ville Vainio, Apr 11, 2006.

  1. Ville Vainio

    Ville Vainio Guest

    I tried to clear a list today (which I do rather rarely, considering
    that just doing l = [] works most of the time) and was shocked, SHOCKED
    to notice that there is no clear() method. Dicts have it, sets have it,
    why do lists have to be second class citizens?
    Ville Vainio, Apr 11, 2006
    #1
    1. Advertising

  2. Ville Vainio wrote:

    > I tried to clear a list today (which I do rather rarely, considering
    > that just doing l = [] works most of the time) and was shocked, SHOCKED
    > to notice that there is no clear() method. Dicts have it, sets have it,
    > why do lists have to be second class citizens?


    because Python already has a perfectly valid way to clear a list,
    perhaps ?

    del l[:]

    (lists are not mappings, so the duck typing argument don't really
    apply here.)

    </F>
    Fredrik Lundh, Apr 11, 2006
    #2
    1. Advertising

  3. Ville Vainio

    Ville Vainio Guest

    Fredrik Lundh wrote:

    > > I tried to clear a list today (which I do rather rarely, considering
    > > that just doing l = [] works most of the time) and was shocked, SHOCKED
    > > to notice that there is no clear() method. Dicts have it, sets have it,
    > > why do lists have to be second class citizens?

    >
    > because Python already has a perfectly valid way to clear a list,
    > perhaps ?
    >
    > del l[:]


    Ok. That's pretty non-obvious but now that I've seen it I'll probably
    remember it. I did a stupid "while l: l.pop()" loop myself.

    > (lists are not mappings, so the duck typing argument don't really
    > apply here.)


    I was thinking of list as a "mutable collection", and clear() is
    certainly a very natural operation for them.
    Ville Vainio, Apr 11, 2006
    #3
  4. Ville Vainio wrote:
    > I tried to clear a list today (which I do rather rarely, considering
    > that just doing l = [] works most of the time) and was shocked, SHOCKED
    > to notice that there is no clear() method. Dicts have it, sets have it,
    > why do lists have to be second class citizens?


    This gets brought up all the time (search the archives for your
    favorite), but your options are basically (renaming your list to lst for
    readability) one of::

    del lst[:]

    lst[:] = []

    or if you don't need to modify the list in place,

    lst = []

    Personally, I tend to go Fredrik's route and use the first.

    If you feel really strongly about this though, you might consider
    writing up a PEP. It's been contentious enough that there's not much
    chance of getting a change without one.

    STeVe
    Steven Bethard, Apr 11, 2006
    #4
  5. Em Ter, 2006-04-11 às 10:42 -0600, Steven Bethard escreveu:
    > one of::
    >
    > del lst[:]
    >
    > lst[:] = []
    >
    > or if you don't need to modify the list in place,
    >
    > lst = []
    >
    > Personally, I tend to go Fredrik's route and use the first.


    I love benchmarks, so as I was testing the options, I saw something very
    strange:

    $ python2.4 -mtimeit 'x = range(100000); '
    100 loops, best of 3: 6.7 msec per loop
    $ python2.4 -mtimeit 'x = range(100000); del x[:]'
    100 loops, best of 3: 6.35 msec per loop
    $ python2.4 -mtimeit 'x = range(100000); x[:] = []'
    100 loops, best of 3: 6.36 msec per loop
    $ python2.4 -mtimeit 'x = range(100000); del x'
    100 loops, best of 3: 6.46 msec per loop

    Why the first benchmark is the slowest? I don't get it... could someone
    test this, too?

    Cheers,

    --
    Felipe.
    Felipe Almeida Lessa, Apr 11, 2006
    #5
  6. Ville Vainio

    Ville Vainio Guest

    Steven Bethard wrote:

    > If you feel really strongly about this though, you might consider
    > writing up a PEP. It's been contentious enough that there's not much
    > chance of getting a change without one.


    No strong feelings here, and I'm sure greater minds than me have
    already hashed this over sufficiently.

    It's just that, when I have an object, and am wondering how I can clear
    it, I tend to look what methods it has first and go to google looking
    for "idioms" second.

    Perhaps "clear" method could be added that raises
    PedagogicException("Use del lst[:], stupid!")?

    *ducks*
    Ville Vainio, Apr 11, 2006
    #6
  7. Felipe Almeida Lessa wrote:
    > I love benchmarks, so as I was testing the options, I saw something very
    > strange:
    >
    > $ python2.4 -mtimeit 'x = range(100000); '
    > 100 loops, best of 3: 6.7 msec per loop
    > $ python2.4 -mtimeit 'x = range(100000); del x[:]'
    > 100 loops, best of 3: 6.35 msec per loop
    > $ python2.4 -mtimeit 'x = range(100000); x[:] = []'
    > 100 loops, best of 3: 6.36 msec per loop
    > $ python2.4 -mtimeit 'x = range(100000); del x'
    > 100 loops, best of 3: 6.46 msec per loop
    >
    > Why the first benchmark is the slowest? I don't get it... could someone
    > test this, too?


    In the first benchmark, you need space for two lists: the old one and
    the new one; the other benchmarks you need only a single block of
    memory (*). Concluding from here gets difficult - you would have to study
    the malloc implementation to find out whether it works better in one
    case over the other. Could also be an issue of processor cache: one
    may fit into the cache, but the other may not.

    Regards,
    Martin

    (*) plus, you also need the integer objects twice.
    =?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=, Apr 11, 2006
    #7
  8. Ville Vainio wrote:
    > It's just that, when I have an object, and am wondering how I can clear
    > it, I tend to look what methods it has first and go to google looking
    > for "idioms" second.


    I guess del on a list is not that common, so people tend to not know
    that it works on lists (and slices!), too. It's too bad that lists have
    a pop() method these days, so people can do x.pop() even if they don't
    need the value, instead of doing del x[-1]. I don't think I ever needed
    to del a slice except for clearing the entire list (and I don't need to
    do that often, either - I just throw the list away).

    Regards,
    Martin
    =?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=, Apr 11, 2006
    #8
  9. Ville Vainio

    John Salerno Guest

    Steven Bethard wrote:


    > lst[:] = []
    > lst = []


    What's the difference here?
    John Salerno, Apr 11, 2006
    #9
  10. Em Ter, 2006-04-11 às 17:56 +0000, John Salerno escreveu:
    > Steven Bethard wrote:
    >
    >
    > > lst[:] = []
    > > lst = []

    >
    > What's the difference here?


    lst[:] = [] makes the specified slice become []. As we specified ":", it
    transforms the entire list into [].

    lst = [] assigns the value [] to the variable lst, deleting any previous
    one.

    This might help:

    >>> lst = range(10)
    >>> id(lst), lst

    (-1210826356, [0, 1, 2, 3, 4, 5, 6, 7, 8, 9])
    >>> lst[:] = []
    >>> id(lst), lst

    (-1210826356, [])

    >>> lst = range(10)
    >>> id(lst), lst

    (-1210844052, [0, 1, 2, 3, 4, 5, 6, 7, 8, 9])
    >>> lst = []
    >>> id(lst), lst

    (-1210826420, [])


    You see? lst[:] removes all elements from the list that lst refers to,
    while lst = [] just creates a new list and discard the only one. The
    difference is, for example:

    >>> lst = range(3)
    >>> x = [lst, lst, lst]
    >>> x

    [[0, 1, 2], [0, 1, 2], [0, 1, 2]]
    >>> lst[:] = []
    >>> x

    [[], [], []]

    >>> lst = range(3)
    >>> x = [lst, lst, lst]
    >>> x

    [[0, 1, 2], [0, 1, 2], [0, 1, 2]]
    >>> lst = []
    >>> x

    [[0, 1, 2], [0, 1, 2], [0, 1, 2]]

    HTH,

    --
    Felipe.
    Felipe Almeida Lessa, Apr 11, 2006
    #10
  11. Ville Vainio

    Duncan Smith Guest

    John Salerno wrote:
    > Steven Bethard wrote:
    >
    >
    >> lst[:] = []
    >> lst = []

    >
    >
    > What's the difference here?


    >>> lst = [1,2,3]
    >>> lst2 = lst
    >>> lst[:] = []
    >>> lst2

    []
    >>> lst = [1,2,3]
    >>> lst2 = lst
    >>> lst = []
    >>> lst2

    [1, 2, 3]
    >>>


    Duncan
    Duncan Smith, Apr 11, 2006
    #11
  12. John Salerno wrote:

    > Steven Bethard wrote:
    >
    >
    > > lst[:] = []
    > > lst = []

    >
    > What's the difference here?


    L[:]= modifies the object in place, L=[] binds the variable to a
    new object. compare and contrast:

    >>> L = ["a", "b", "c"]
    >>> M = L
    >>> L

    ['a', 'b', 'c']
    >>> M

    ['a', 'b', 'c']
    >>> L is M

    True
    >>> L[:] = []
    >>> L

    []
    >>> M

    []
    >>> L is M

    True

    >>> L = ["a", "b", "c"]
    >>> M = L
    >>> L

    ['a', 'b', 'c']
    >>> M

    ['a', 'b', 'c']
    >>> L = []
    >>> L

    []
    >>> M

    ['a', 'b', 'c']
    >>> L is M

    False

    </F>
    Fredrik Lundh, Apr 11, 2006
    #12
  13. Ville Vainio

    John Salerno Guest

    Felipe Almeida Lessa wrote:

    > You see? lst[:] removes all elements from the list that lst refers to,
    > while lst = [] just creates a new list and discard the only one. The
    > difference is, for example:


    Thanks, your explanation was great!
    John Salerno, Apr 11, 2006
    #13
  14. Ville Vainio

    John Salerno Guest

    Fredrik Lundh wrote:
    > John Salerno wrote:
    >
    >> Steven Bethard wrote:
    >>
    >>
    >>> lst[:] = []
    >>> lst = []

    >> What's the difference here?

    >
    > L[:]= modifies the object in place, L=[] binds the variable to a
    > new object. compare and contrast:


    Thanks guys, your explanations are really helpful. I think what had me
    confused at first was my understanding of what L[:] does on either side
    of the assignment operator. On the left, it just chooses those elements
    and edits them in place; on the right, it makes a copy of that list,
    right? (Which I guess is still more or less *doing* the same thing, just
    for different purposes)
    John Salerno, Apr 11, 2006
    #14
  15. Ville Vainio

    Ville Vainio Guest

    John Salerno wrote:

    > Thanks guys, your explanations are really helpful. I think what had me
    > confused at first was my understanding of what L[:] does on either side
    > of the assignment operator. On the left, it just chooses those elements
    > and edits them in place; on the right, it makes a copy of that list,
    > right? (Which I guess is still more or less *doing* the same thing, just
    > for different purposes)


    Interestingly, if it was just a "clear" method nobody would be confused.
    Ville Vainio, Apr 11, 2006
    #15
  16. Felipe Almeida Lessa <> writes:

    > I love benchmarks, so as I was testing the options, I saw something very
    > strange:
    >
    > $ python2.4 -mtimeit 'x = range(100000); '
    > 100 loops, best of 3: 6.7 msec per loop
    > $ python2.4 -mtimeit 'x = range(100000); del x[:]'
    > 100 loops, best of 3: 6.35 msec per loop
    > $ python2.4 -mtimeit 'x = range(100000); x[:] = []'
    > 100 loops, best of 3: 6.36 msec per loop
    > $ python2.4 -mtimeit 'x = range(100000); del x'
    > 100 loops, best of 3: 6.46 msec per loop
    >
    > Why the first benchmark is the slowest? I don't get it... could someone
    > test this, too?


    I get similar behaviour. No idea why.

    $ python2.4 -mtimeit 'x = range(100000); '
    100 loops, best of 3: 6.99 msec per loop
    $ python2.4 -mtimeit 'x = range(100000); del x[:]'
    100 loops, best of 3: 6.49 msec per loop
    $ python2.4 -mtimeit 'x = range(100000); x[:] = []'
    100 loops, best of 3: 6.47 msec per loop
    $ python2.4 -mtimeit 'x = range(100000); del x'
    100 loops, best of 3: 6.6 msec per loop

    Dan
    Dan Christensen, Apr 12, 2006
    #16
  17. On Tue, 11 Apr 2006 14:49:04 -0700, Ville Vainio wrote:

    > John Salerno wrote:
    >
    >> Thanks guys, your explanations are really helpful. I think what had me
    >> confused at first was my understanding of what L[:] does on either side
    >> of the assignment operator. On the left, it just chooses those elements
    >> and edits them in place; on the right, it makes a copy of that list,
    >> right? (Which I guess is still more or less *doing* the same thing, just
    >> for different purposes)

    >
    > Interestingly, if it was just a "clear" method nobody would be confused.


    Even more importantly, you could say help(list.clear) and learn something
    useful, instead of trying help(del) and getting a syntax error.

    >>> help(del)

    File "<stdin>", line 1
    help(del)
    ^
    SyntaxError: invalid syntax

    I know Python isn't a purely OOP language, but in my opinion using
    statements like del should be avoided when there is an easy and natural OO
    way of doing it. Something like name.del() which means "delete the
    reference to name in name's namespace" feels wrong, and may not even be
    possible with Python's object model, so del name is a natural way to do
    it.

    But name.clear() meaning "mutate the object referenced by name to the
    empty state" is a very natural candidate for a method, and I don't
    understand why lists shouldn't have it.

    For lists, it would be natural to have a hypothetical clear() method
    accept an index or slice as an argument, so that these are equivalent:

    del L[:] <=> L.clear()
    del L[n] <=> L.clear(n)
    del L[a:b] <=> L.clear(slice(a, b))


    # untested reference implementation:
    class ClearableList(list):
    def clear(self, obj=None):
    if obj is None:
    obj = slice(0, len(self))
    if isinstance(obj, int):
    self.__delitem__(obj)
    elif isinstance(obj, slice):
    self.__delslice__(obj.start, obj.stop)
    else:
    raise TypeError


    --
    Steven.
    Steven D'Aprano, Apr 12, 2006
    #17
  18. On Tue, 11 Apr 2006 19:15:18 +0200, Martin v. Löwis wrote:

    > Felipe Almeida Lessa wrote:
    >> I love benchmarks, so as I was testing the options, I saw something very
    >> strange:
    >>
    >> $ python2.4 -mtimeit 'x = range(100000); '
    >> 100 loops, best of 3: 6.7 msec per loop
    >> $ python2.4 -mtimeit 'x = range(100000); del x[:]'
    >> 100 loops, best of 3: 6.35 msec per loop
    >> $ python2.4 -mtimeit 'x = range(100000); x[:] = []'
    >> 100 loops, best of 3: 6.36 msec per loop
    >> $ python2.4 -mtimeit 'x = range(100000); del x'
    >> 100 loops, best of 3: 6.46 msec per loop
    >>
    >> Why the first benchmark is the slowest? I don't get it... could someone
    >> test this, too?

    >
    > In the first benchmark, you need space for two lists: the old one and
    > the new one;


    Er, what new list? I see only one list, x = range(100000), which is merely
    created then nothing done to it. Have I missed something?

    I understood Felipe to be asking, why does it take longer to just create a
    list, than it takes to create a list AND then do something to it?



    --
    Steven.
    Steven D'Aprano, Apr 12, 2006
    #18
  19. Em Qua, 2006-04-12 às 11:36 +1000, Steven D'Aprano escreveu:
    > On Tue, 11 Apr 2006 19:15:18 +0200, Martin v. Löwis wrote:
    >
    > > Felipe Almeida Lessa wrote:
    > >> I love benchmarks, so as I was testing the options, I saw something very
    > >> strange:
    > >>
    > >> $ python2.4 -mtimeit 'x = range(100000); '
    > >> 100 loops, best of 3: 6.7 msec per loop
    > >> $ python2.4 -mtimeit 'x = range(100000); del x[:]'
    > >> 100 loops, best of 3: 6.35 msec per loop
    > >> $ python2.4 -mtimeit 'x = range(100000); x[:] = []'
    > >> 100 loops, best of 3: 6.36 msec per loop
    > >> $ python2.4 -mtimeit 'x = range(100000); del x'
    > >> 100 loops, best of 3: 6.46 msec per loop
    > >>
    > >> Why the first benchmark is the slowest? I don't get it... could someone
    > >> test this, too?

    > >
    > > In the first benchmark, you need space for two lists: the old one and
    > > the new one;

    >
    > Er, what new list? I see only one list, x = range(100000), which is merely
    > created then nothing done to it. Have I missed something?


    He's talking about the garbage collector.

    Talking about the GC, do you want to see something *really* odd?

    $ python2.4 -mtimeit -s 'from gc import collect' 'collect(); x =
    range(100000); '
    100 loops, best of 3: 13 msec per loop

    $ python2.4 -mtimeit -s 'from gc import collect' 'collect(); x =
    range(100000); del x[:]'
    100 loops, best of 3: 8.19 msec per loop

    $ python2.4 -mtimeit -s 'from gc import collect' 'collect(); x =
    range(100000); x[:] = []'
    100 loops, best of 3: 8.16 msec per loop

    $ python2.4 -mtimeit -s 'from gc import collect' 'collect(); x =
    range(100000); del x'
    100 loops, best of 3: 8.3 msec per loop


    But in this case I got the answer (I think):
    - When you explicitly delete the objects, the GC already know that it
    can be collected, so it just throw the objects away.
    - When we let the "x" variable continue to survive, the GC has to look
    at all the 100001 objects to see if they can be collected -- just to see
    that it can't.

    Also, IIRC "del x" is slower than "x = []" because removing a name from
    the namespace is more expensive than just assigning something else to
    it. Right?

    > I understood Felipe to be asking, why does it take longer to just create a
    > list, than it takes to create a list AND then do something to it?


    I see dead people... ;-)

    --
    Felipe.
    Felipe Almeida Lessa, Apr 12, 2006
    #19
  20. Ville Vainio

    Serge Orlov Guest

    Martin v. Löwis wrote:
    > Felipe Almeida Lessa wrote:
    > > I love benchmarks, so as I was testing the options, I saw something very
    > > strange:
    > >
    > > $ python2.4 -mtimeit 'x = range(100000); '
    > > 100 loops, best of 3: 6.7 msec per loop
    > > $ python2.4 -mtimeit 'x = range(100000); del x[:]'
    > > 100 loops, best of 3: 6.35 msec per loop
    > > $ python2.4 -mtimeit 'x = range(100000); x[:] = []'
    > > 100 loops, best of 3: 6.36 msec per loop
    > > $ python2.4 -mtimeit 'x = range(100000); del x'
    > > 100 loops, best of 3: 6.46 msec per loop
    > >
    > > Why the first benchmark is the slowest? I don't get it... could someone
    > > test this, too?

    >
    > In the first benchmark, you need space for two lists: the old one and
    > the new one; the other benchmarks you need only a single block of
    > memory (*).


    I don't follow you here :)

    > Concluding from here gets difficult - you would have to study
    > the malloc implementation to find out whether it works better in one
    > case over the other.


    That's not the case here. The following program prints the same
    addresses whether you comment or uncomment del
    ---------------------------------
    ids = range(10)
    for i in xrange(10):
    x = range(100000)
    #del x[:]
    ids = id(x)

    print ids
    --------------------------------


    > Could also be an issue of processor cache: one
    > may fit into the cache, but the other may not.


    That's not the reason, since the same time difference happens with
    smaller arrays. I think the difference is how items are deallocated in
    these two cases.

    Calling del invokes list_ass_slice that deallocates from 0 to end
    whereas ordinary removal of a list was changed to backward iteration
    (the reason is in the comment:
    /* Do it backwards, for Christian Tismer.
    There's a simple test case where somehow this reduces
    thrashing when a *very* large list is created and
    immediately deleted. */

    Usually iterating from low addresses to higher addresses is better for
    CPU. On my CPU (Pentium M, 1.7Ghz) it's 20% faster:

    Here is my results:

    C:\py>python -mtimeit "x = range(10000); del x[:]"
    1000 loops, best of 3: 213 usec per loop

    C:\py>python -mtimeit "x = range(10000); del x"
    1000 loops, best of 3: 257 usec per loop

    C:\py>python -mtimeit "x = range(10000); "
    1000 loops, best of 3: 258 usec per loop

    C:\py>python -mtimeit "x = range(1000); del x[:]"
    10000 loops, best of 3: 21.4 usec per loop

    C:\py>python -mtimeit "x = range(1000); del x"
    10000 loops, best of 3: 25.2 usec per loop

    C:\py>python -mtimeit "x = range(1000); "
    10000 loops, best of 3: 25.6 usec per loop

    I don't have a development environment on my computer so I can't test
    my thoughts. I could be wrong about the reason.
    Serge Orlov, Apr 12, 2006
    #20
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. =?Utf-8?B?RGF2aWQ=?=

    Clear Browser History List

    =?Utf-8?B?RGF2aWQ=?=, Feb 19, 2004, in forum: ASP .Net
    Replies:
    1
    Views:
    6,125
    Chris Jackson
    Feb 19, 2004
  2. Michael Jasn
    Replies:
    1
    Views:
    562
    John Harrison
    Nov 1, 2004
  3. Gary Herron
    Replies:
    2
    Views:
    651
    Bruno Desthuilliers
    Jul 4, 2006
  4. David

    Response.Clear() doesn't clear

    David, Jan 31, 2008, in forum: ASP .Net
    Replies:
    2
    Views:
    993
    Mark Fitzpatrick
    Jan 31, 2008
  5. InvalidLastName

    Unrecognized element 'add' after <clear></clear>

    InvalidLastName, Feb 26, 2007, in forum: ASP .Net Web Services
    Replies:
    3
    Views:
    924
    Steven Cheng[MSFT]
    Mar 6, 2007
Loading...

Share This Page