iter

Discussion in 'Python' started by daryn, Aug 9, 2010.

  1. daryn

    daryn Guest

    I'm just playing around with the iter function and I realize that I
    can use the iterator returned by it long after the original object has
    any name bound to it. Example:

    >>>a=[1,2,3,4]
    >>>b=iter(a)
    >>>b.next()

    1
    >>>a[1]=99
    >>>a[3]=101
    >>>del a
    >>>b.next()

    99
    >>>b.next()

    3
    >>>b.next()

    101

    it seems as if the original object is never being garbage collected
    even though there is no name bound to it. Does the name bound to the
    iterator object count as a reference to the original object for
    garbage collection purposes? Is there some way to retrieve/manipulate
    the original object via the iterator?

    Just trying to understand how this all works.

    -thanks for any help you can give
    daryn
     
    daryn, Aug 9, 2010
    #1
    1. Advertising

  2. daryn

    Peter Otten Guest

    daryn wrote:

    > I'm just playing around with the iter function and I realize that I
    > can use the iterator returned by it long after the original object has
    > any name bound to it. Example:
    >
    >>>>a=[1,2,3,4]
    >>>>b=iter(a)
    >>>>b.next()

    > 1
    >>>>a[1]=99
    >>>>a[3]=101
    >>>>del a
    >>>>b.next()

    > 99
    >>>>b.next()

    > 3
    >>>>b.next()

    > 101
    >
    > it seems as if the original object is never being garbage collected
    > even though there is no name bound to it. Does the name bound to the
    > iterator object count as a reference to the original object for
    > garbage collection purposes?


    The listiterator object internally holds a reference to the list, but
    doesn't make it available to Python code.

    > Is there some way to retrieve/manipulate
    > the original object via the iterator?


    Not without dirty tricks (ctypes).

    > Just trying to understand how this all works.


    If you can read C look here:

    http://svn.python.org/view/python/branches/py3k/Objects/listobject.c?revision=81032&view=markup

    Peter
     
    Peter Otten, Aug 9, 2010
    #2
    1. Advertising

  3. daryn

    Terry Reedy Guest

    On 8/9/2010 12:11 PM, daryn wrote:
    > I'm just playing around with the iter function and I realize that I
    > can use the iterator returned by it long after the original object has
    > any name bound to it. Example:
    >
    >>>> a=[1,2,3,4]
    >>>> b=iter(a)
    >>>> b.next()

    > 1
    >>>> a[1]=99


    Changing a list while iterating through it is possible, sometimes
    useful, but error prone, especially with insert or delete. Changing a
    dict while iterating through it is prohibited since the iteration order
    depends on the exact internal structure. That in turn depends on the
    history of additions and deletions.

    >>>> a[3]=101
    >>>> del a
    >>>> b.next()

    > 99
    >>>> b.next()

    > 3
    >>>> b.next()

    > 101
    >
    > it seems as if the original object is never being garbage collected
    > even though there is no name bound to it.


    The fact that CPython currently deletes some things immediately is a
    current implementation detail, subject to change.

    > Does the name bound to the
    > iterator object count as a reference to the original object for
    > garbage collection purposes?


    Not quite. The iterator obviously has to have an internal reference,
    which amount to the almost the same thing. However, if you put the
    iterator in a list and deleted the name binding, both the iterator and
    list are still kept around.

    > Is there some way to retrieve/manipulate
    > the original object via the iterator?


    If you do dir(b), you will only see the standard __xx__ methods, most of
    which are inherited. Iterators can be written to expose an underlying
    object, if there is one, but some do not have one and this is not part
    of the simple iterator protocol. Hence builtin iterators for builtins do
    not do so.

    If one actually needed such lookup, this wrapper should work.

    class myiter():
    def __init__(self, ob):
    self.ob = ob
    self.__itnext__ = iter(ob).__next__
    def __iter__(self):
    return self
    def __next__(self):
    return self.__itnext__()

    it = myiter([1,2,3])
    print (it.ob, list(it))

    # [1, 2, 3] [1, 2, 3]

    > Just trying to understand how this all works.


    Keep experimenting. Python makes is so easy, especially with an edit
    window such as with IDLE and a Run command. I initially tried not
    defining myiter.__next__ and instead wrote:
    self.__next__ = iter(ob).__next__
    but that does not work because special __xx__ methods are typically
    looked up on the class, not the instance. I know that but was not
    completely sure about this case.

    --
    Terry Jan Reedy
     
    Terry Reedy, Aug 9, 2010
    #3
  4. On Mon, 09 Aug 2010 09:11:37 -0700, daryn wrote:

    > I'm just playing around with the iter function and I realize that I can
    > use the iterator returned by it long after the original object has any
    > name bound to it.


    Yes, the same as everything else in Python. Iterators aren't unique here.

    >>> a = [1,2,3]
    >>> b = [None, a, None]
    >>> id(a) == id(b[1]) # Same object, not a copy?

    True
    >>> del a
    >>> print b

    [None, [1, 2, 3], None]



    > it seems as if the original object is never being garbage collected even
    > though there is no name bound to it.


    Of course not. That would be a Bad Thing if Python garbage collected an
    object while other objects were still using it. Can you say "core dump"?



    > Does the name bound to the
    > iterator object count as a reference to the original object for garbage
    > collection purposes?


    No, but the iterator itself does.

    The technique is called *reference* counting, not "name counting". Each
    name is a reference, but not every reference is a name.


    > Is there some way to retrieve/manipulate the
    > original object via the iterator?


    Depends on the iterator. For the standard iterator created by iter(), I
    don't think so. But for a custom iterator type, there can be if you want:

    class MyIter(object):
    """Quick and dirty iterator."""
    def __init__(self, data):
    self.data = data
    self.i = 0
    def __iter__(self):
    return self
    def next(self):
    try:
    o = self.data[self.i]
    except IndexError:
    raise StopIteration
    self.i += 1
    return o


    >>> it = MyIter([1,2,3,4])
    >>> it.next()

    1
    >>> del it.data[1:3]
    >>> it.next()

    4




    --
    Steven
     
    Steven D'Aprano, Aug 10, 2010
    #4
  5. daryn

    Aahz Guest

    In article <>,
    Terry Reedy <> wrote:
    >
    >Changing a list while iterating through it is possible, sometimes
    >useful, but error prone, especially with insert or delete. Changing a
    >dict while iterating through it is prohibited since the iteration order
    >depends on the exact internal structure. That in turn depends on the
    >history of additions and deletions.


    Although I agree in general with your warning, you are factually
    incorrect about dicts:

    >>> d = {1:2, 3:4}
    >>> i = iter(d)
    >>> i.next()

    1
    >>> d[1] = 'foo'
    >>> d

    {1: 'foo', 3: 4}

    Essentially, the prohibition is against changing the *keys* of lists and
    dicts (where list keys are the indexes). So what you can't do is add or
    delete dict keys and changing the position or order of list elements is
    a Bad Idea. But changing dict or list values is fine as long as you're
    careful that's *all* you're doing. Python newcomers are best off simply
    avoiding any list/dict mutation during iteration.
    --
    Aahz () <*> http://www.pythoncraft.com/

    "...if I were on life-support, I'd rather have it run by a Gameboy than a
    Windows box." --Cliff Wells
     
    Aahz, Aug 24, 2010
    #5
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Steven Brent

    iter jitters

    Steven Brent, May 4, 2004, in forum: Python
    Replies:
    5
    Views:
    737
    Peter Otten
    May 5, 2004
  2. dw

    bugs at iter file() ?

    dw, Jul 15, 2004, in forum: Python
    Replies:
    0
    Views:
    369
  3. dw
    Replies:
    0
    Views:
    271
  4. thomas
    Replies:
    23
    Views:
    881
    James Kanze
    Feb 26, 2008
  5. Gennaro Prota
    Replies:
    1
    Views:
    347
    Gennaro Prota
    Aug 21, 2008
Loading...

Share This Page