properly delete item during "for item in..."

Discussion in 'Python' started by Ratko, Jul 17, 2008.

  1. Ratko

    Ratko Guest

    Say you have something like this:

    for item in myList:
    del item

    Would this actually delete the item from the list or just decrement
    the reference counter because the item in myList is not associated
    with name "item" anymore (but still is with myList[itemIndex])? In
    other words, is "item" a temporary reference to myList[itemIndex] or
    is it actually that reference that myList has stored?

    I am not sure if this question even makes any sense anymore. I've been
    using python for years and never had any problems (and I don't now
    either) but now that I had to revisit c++/STL, I had to deal about
    these issues and was wondering how python does it.

    Thanks,
    Ratko
     
    Ratko, Jul 17, 2008
    #1
    1. Advertising

  2. On Thu, 17 Jul 2008 09:27:27 -0700, Ratko wrote:

    > for item in myList:
    > del item
    >
    > Would this actually delete the item from the list or just decrement
    > the reference counter because the item in myList is not associated
    > with name "item" anymore (but still is with myList[itemIndex])? In
    > other words, is "item" a temporary reference to myList[itemIndex] or
    > is it actually that reference that myList has stored?


    The latter. Names are always bound to objects, you can't bind a name to
    another name or reference in Python.

    Ciao,
    Marc 'BlackJack' Rintsch
     
    Marc 'BlackJack' Rintsch, Jul 17, 2008
    #2
    1. Advertising

  3. Ratko

    Gary Herron Guest

    Ratko wrote:
    > Say you have something like this:
    >
    > for item in myList:
    > del item
    >
    > Would this actually delete the item from the list or just decrement
    > the reference counter because the item in myList is not associated
    > with name "item" anymore (but still is with myList[itemIndex])? In
    > other words, is "item" a temporary reference to myList[itemIndex] or
    > is it actually that reference that myList has stored?
    >


    The 'del' statement does not delete an object, it deletes a reference to
    an object. In this case, the variable item is deleted from the scope,
    and the referred-to object will have its reference counter decremented
    by 1. (But, as you surmise, not to zero, because the list will still
    reference it.)

    You could remove the object from the list with
    del myList
    if you knew i. HOWEVER, don't do that while looping through the list!
    Changing a list's length will interact badly with the for loop's
    indexing through the list, causing the loop to mis the element following
    the deleted item.

    Gary Herron

    > I am not sure if this question even makes any sense anymore. I've been
    > using python for years and never had any problems (and I don't now
    > either) but now that I had to revisit c++/STL, I had to deal about
    > these issues and was wondering how python does it.
    >
    > Thanks,
    > Ratko
    > --
    > http://mail.python.org/mailman/listinfo/python-list
    >
     
    Gary Herron, Jul 17, 2008
    #3
  4. Ratko

    mk Guest

    Gary Herron wrote:
    > You could remove the object from the list with
    > del myList
    > if you knew i. HOWEVER, don't do that while looping through the list!
    > Changing a list's length will interact badly with the for loop's
    > indexing through the list, causing the loop to mis the element following
    > the deleted item.


    Jumping into a thread, I know how not to do it, but not how to do it
    properly?

    Iterating over a copy may _probably_ work:

    >>> t=['a', 'c', 'b', 'd']
    >>>
    >>> for el in t[:]:

    del t[t.index(el)]


    >>> t

    []


    However, is it really safe? Defining safe as "works reliably in every
    corner case for every indexable data type"?


    Con: suppose the data structure t is really, really big. Just deleting
    some items from t temporarily doubles the memory consumption.
     
    mk, Jul 17, 2008
    #4
  5. Ratko

    Ratko Guest

    On Jul 17, 9:57 am, mk <> wrote:
    > Gary Herron wrote:
    > > You could remove the object from the list with
    > > del myList
    > > if you knew i. HOWEVER, don't do that while looping through the list!
    > > Changing a list's length will interact badly with the for loop's
    > > indexing through the list, causing the loop to mis the element following
    > > the deleted item.

    >
    > Jumping into a thread, I know how not to do it, but not how to do it
    > properly?
    >
    > Iterating over a copy may _probably_ work:
    >
    > >>> t=['a', 'c', 'b', 'd']
    > >>>
    > >>> for el in t[:]:

    > del t[t.index(el)]
    >
    > >>> t

    > []
    >
    > However, is it really safe? Defining safe as "works reliably in every
    > corner case for every indexable data type"?
    >
    > Con: suppose the data structure t is really, really big. Just deleting
    > some items from t temporarily doubles the memory consumption.




    Would this work (safely) then? It does in my test cases but that of
    course doesn't prove it works in a general case...

    for item in myList:
    myList.remove(item)


    For dictionaries we can just iterate over values() or items() as
    opposed to itervalues() or iteritems() since that's technically a copy
    of values or items in the dict, right?


    R
     
    Ratko, Jul 17, 2008
    #5
  6. Ratko

    Gary Herron Guest

    Ratko wrote:
    > On Jul 17, 9:57 am, mk <> wrote:
    >
    >> Gary Herron wrote:
    >>
    >>> You could remove the object from the list with
    >>> del myList
    >>> if you knew i. HOWEVER, don't do that while looping through the list!
    >>> Changing a list's length will interact badly with the for loop's
    >>> indexing through the list, causing the loop to mis the element following
    >>> the deleted item.
    >>>

    >> Jumping into a thread, I know how not to do it, but not how to do it
    >> properly?
    >>
    >> Iterating over a copy may _probably_ work:
    >>
    >> >>> t=['a', 'c', 'b', 'd']
    >> >>>
    >> >>> for el in t[:]:

    >> del t[t.index(el)]
    >>
    >> >>> t

    >> []
    >>
    >> However, is it really safe? Defining safe as "works reliably in every
    >> corner case for every indexable data type"?
    >>
    >> Con: suppose the data structure t is really, really big. Just deleting
    >> some items from t temporarily doubles the memory consumption.
    >>

    >
    >
    >
    > Would this work (safely) then? It does in my test cases but that of
    > course doesn't prove it works in a general case...
    >
    > for item in myList:
    > myList.remove(item)
    >


    No. Same problem, The for loop iterates through the list by keeping
    and incrementing an internal index. Any modification of the list does
    not change the index correspondingly.

    One proper way:
    newList = []
    for item in myList:
    if ... whatever...
    newList.append(item)
    myList = newList

    Another, using list comprehension (it's the same thing really as the above):
    myList = [item for item in myList if ... whatever...]




    >
    > For dictionaries we can just iterate over values() or items() as
    > opposed to itervalues() or iteritems() since that's technically a copy
    > of values or items in the dict, right?
    >


    No! In fact the whole point of iteritems and itervalues and iterkeys is
    that they *DO NOT* make copies, so changing the dictionary out from
    under them is a programming error.

    If you use dict.items(), dict.keys() or dict.values(), then you're OK,
    because these methods *do* create new lists for both.

    Gary Herron

    >
    > R
    >
    > --
    > http://mail.python.org/mailman/listinfo/python-list
    >
     
    Gary Herron, Jul 17, 2008
    #6
  7. Ratko

    Ratko Guest

    > > For dictionaries we can just iterate over values() or items() as
    > > opposed to itervalues() or iteritems() since that's technically a copy
    > > of values or items in the dict, right?

    >
    > No! In fact the whole point of iteritems and itervalues and iterkeys is
    > that they *DO NOT* make copies, so changing the dictionary out from
    > under them is a programming error.
    >
    > If you use dict.items(), dict.keys() or dict.values(), then you're OK,
    > because these methods *do* create new lists for both.



    That's what I meant, it just didn't come across correctly I guess.
    Thanks for clarifying these issues. I think I have a better
    understanding now.
    R
     
    Ratko, Jul 17, 2008
    #7

  8. > -----Original Message-----
    > From: python-list-bounces+jr9445= [mailto:python-
    > list-bounces+jr9445=] On Behalf Of Ratko
    > Sent: Thursday, July 17, 2008 12:27 PM
    > To:
    > Subject: properly delete item during "for item in..."
    >
    > Say you have something like this:
    >
    > for item in myList:
    > del item
    >
    > Would this actually delete the item from the list or just decrement
    > the reference counter because the item in myList is not associated
    > with name "item" anymore (but still is with myList[itemIndex])? In
    > other words, is "item" a temporary reference to myList[itemIndex] or
    > is it actually that reference that myList has stored?
    >
    > I am not sure if this question even makes any sense anymore. I've been
    > using python for years and never had any problems (and I don't now
    > either) but now that I had to revisit c++/STL, I had to deal about
    > these issues and was wondering how python does it.
    >



    Walk the list backwards when deleting.



    master = ['a', 'b', 'c', 'd', 'e', 'f', 'g']


    print "Deletes nothing"
    a = master[:]
    print a
    for i in a:
    del i
    print a

    print
    print

    print "Deletes safely from end"
    a = master[:]
    print a
    for i in range(len(a)-1, -1, -1):
    print i
    if i % 2 == 0:
    print " removing ", master
    del a
    print " ", a,
    if master in a:
    print "Ooops, deleted wrong thing...",
    print
    print a

    print
    print

    print "Delete from front. Deletes wrong things and throws an
    exception..."
    a = master[:]
    print a
    #for i in range(len(a)-1, -1, -1):
    for i in range(len(a)):
    print i
    if i % 2 == 0:
    print " removing ", master
    del a
    print " ", a,
    if master in a:
    print "Ooops, deleted wrong thing...",
    print
    print a
     
    Reedick, Andrew, Jul 17, 2008
    #8
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. =?Utf-8?B?UGFuYXlvdGlzIEtvdXZhcmFraXM=?=

    2-digit year dates not validated properly during client validation

    =?Utf-8?B?UGFuYXlvdGlzIEtvdXZhcmFraXM=?=, Dec 2, 2005, in forum: ASP .Net
    Replies:
    0
    Views:
    583
    =?Utf-8?B?UGFuYXlvdGlzIEtvdXZhcmFraXM=?=
    Dec 2, 2005
  2. Howard
    Replies:
    0
    Views:
    404
    Howard
    Jul 1, 2003
  3. traveller
    Replies:
    0
    Views:
    1,242
    traveller
    Jan 8, 2008
  4. Dmitry Korolyov

    Datagrid not updated during delete, but updated during insert and update

    Dmitry Korolyov, Sep 22, 2003, in forum: ASP .Net Datagrid Control
    Replies:
    0
    Views:
    439
    Dmitry Korolyov
    Sep 22, 2003
  5. Namor
    Replies:
    0
    Views:
    331
    Namor
    Jan 19, 2006
Loading...

Share This Page