Conditionally implementing __iter__ in new style classes

Discussion in 'Python' started by Thomas Heller, Jul 6, 2005.

  1. I'm trying to implement __iter__ on an abstract base class while I don't
    know whether subclasses support that or not.
    Hope that makes sense, if not, this code should be clearer:

    class Base:
    def __getattr__(self, name):
    if name == "__iter__" and hasattr(self, "Iterator"):
    return self.Iterator
    raise AttributeError, name

    class Concrete(Base):
    def Iterator(self):
    yield 1
    yield 2
    yield 3

    The idea is that if a subclass of Base defines an 'Iterator' method,
    instances are iterable. They are not iterable otherwise.

    The above gives the expected behaviour: iter(Base()) raises a
    "TypeError: iteration over non-sequence", and iter(Concrete()) returns a
    generator.

    If, however, I make Base a newstyle class, this will not work any
    longer. __getattr__ is never called for "__iter__" (neither is
    __getattribute__, btw). Probably this has to do with data descriptors
    and non-data descriptors, but I'm too tired at the moment to think
    further about this.

    Is there any way I could make the above code work with new style
    classes?

    Thanks,

    Thomas
    Thomas Heller, Jul 6, 2005
    #1
    1. Advertising

  2. Thomas Heller <> writes:

    > I'm trying to implement __iter__ on an abstract base class while I don't
    > know whether subclasses support that or not.
    > Hope that makes sense, if not, this code should be clearer:
    >
    > class Base:
    > def __getattr__(self, name):
    > if name == "__iter__" and hasattr(self, "Iterator"):
    > return self.Iterator
    > raise AttributeError, name
    >
    > class Concrete(Base):
    > def Iterator(self):
    > yield 1
    > yield 2
    > yield 3
    >
    > The idea is that if a subclass of Base defines an 'Iterator' method,
    > instances are iterable. They are not iterable otherwise.
    >
    > The above gives the expected behaviour: iter(Base()) raises a
    > "TypeError: iteration over non-sequence", and iter(Concrete()) returns a
    > generator.
    >
    > If, however, I make Base a newstyle class, this will not work any
    > longer. __getattr__ is never called for "__iter__" (neither is
    > __getattribute__, btw). Probably this has to do with data descriptors
    > and non-data descriptors, but I'm too tired at the moment to think
    > further about this.
    >
    > Is there any way I could make the above code work with new style
    > classes?


    I forgot to mention this: The Base class also implements a __getitem__
    method which should be used for iteration if the .Iterator method in the
    subclass is not available. So it seems impossible to raise an exception
    in the __iter__ method if .Iterator is not found - __iter__ MUST return
    an iterator if present.

    Thomas
    Thomas Heller, Jul 6, 2005
    #2
    1. Advertising

  3. Thomas Heller

    infidel Guest

    I'm not sure I understand why you would want to. Just don't define
    __iter__ on your newstyle class and you'll get the expected behavior.
    infidel, Jul 6, 2005
    #3
  4. Thomas Heller

    infidel Guest

    Why not define an Iterator method in your Base class that does the
    iteration using __getitem__, and any subclass that wants to do
    something else just defines its own Iterator method? For that matter,
    you could just use the __iter__ methods of Base and Concrete instead of
    a separate method.
    infidel, Jul 6, 2005
    #4
  5. > I'm trying to implement __iter__ on an abstract base class while I
    > don't
    > know whether subclasses support that or not.
    > Hope that makes sense, if not, this code should be clearer:
    >
    > class Base:
    > def __getattr__(self, name):
    > if name == "__iter__" and hasattr(self, "Iterator"):
    > return self.Iterator
    > raise AttributeError, name
    >
    > class Concrete(Base):
    > def Iterator(self):
    > yield 1
    > yield 2
    > yield 3


    I don't know how to achieve it, but why don't you simply use

    class Base:
    pass

    class Concrete(Base):
    def __iter__(self) :
    yield 1
    yield 2
    yield 3


    What is the advantage to have a baseclass that essentially does
    some method renaming __iter__ ==> Iterator?

    - harold -

    --
    Always remember that you are unique;
    just like everyone else.
    --
    harold fellermann, Jul 6, 2005
    #5
  6. Thomas Heller

    infidel Guest

    Something like this:

    >>> class Base(object):

    .... def __getitem__(self, key):
    .... return key
    .... def __iter__(self):
    .... yield self[1]
    .... yield self['foo']
    .... yield self[3.0]
    ....
    >>> class ConcreteIterable(Base):

    .... def __iter__(self):
    .... yield True
    .... yield 'Blue'
    .... yield 'Foo'
    ....
    >>> class ConcreteNotIterable(Base):

    .... pass
    ....
    >>> [x for x in Base()]

    [1, 'foo', 3.0]
    >>> [x for x in ConcreteIterable()]

    [True, 'Blue', 'Foo']
    >>> [x for x in ConcreteNotIterable()]

    [1, 'foo', 3.0]
    >>>
    infidel, Jul 6, 2005
    #6
  7. Thomas Heller wrote:
    > I forgot to mention this: The Base class also implements a __getitem__
    > method which should be used for iteration if the .Iterator method in the
    > subclass is not available. So it seems impossible to raise an exception
    > in the __iter__ method if .Iterator is not found - __iter__ MUST return
    > an iterator if present.


    def Iterator(self):
    for index in xrange(len(self)):
    yield self[index]

    def __iter__(self):
    return self.Iterator()

    ....and then override Iterator in subclasses. But this raises the
    question of why you need to use a specially-named method instead of
    having subclasses override the __iter__.
    Leif K-Brooks, Jul 6, 2005
    #7
  8. Thomas Heller

    Peter Otten Guest

    Thomas Heller wrote:

    > I'm trying to implement __iter__ on an abstract base class while I don't
    > know whether subclasses support that or not.
    > Hope that makes sense, if not, this code should be clearer:
    >
    > class Base:
    > def __getattr__(self, name):
    > if name == "__iter__" and hasattr(self, "Iterator"):
    > return self.Iterator
    > raise AttributeError, name


    > Is there any way I could make the above code work with new style
    > classes?


    Obligatory metaclass approach:

    class Base:
    class __metaclass__(type):
    def __new__(mcl, name, bases, classdict):
    try:
    classdict["__iter__"] = classdict["Iterator"]
    except KeyError:
    pass
    return type.__new__(mcl, name, bases, classdict)

    class Alpha(Base):
    def Iterator(self): yield 42

    class Beta(Base):
    def __getitem__(self, index):
    return [1, 2, 3, "ganz viele"][index]


    for item in Alpha(): print item
    for item in Beta(): print item,
    print

    Peter
    Peter Otten, Jul 6, 2005
    #8
  9. On Wed, 06 Jul 2005 17:57:42 +0200, Thomas Heller <> wrote:

    >I'm trying to implement __iter__ on an abstract base class while I don't
    >know whether subclasses support that or not.
    >Hope that makes sense, if not, this code should be clearer:
    >
    >class Base:
    > def __getattr__(self, name):
    > if name == "__iter__" and hasattr(self, "Iterator"):
    > return self.Iterator
    > raise AttributeError, name
    >
    >class Concrete(Base):
    > def Iterator(self):
    > yield 1
    > yield 2
    > yield 3
    >
    >The idea is that if a subclass of Base defines an 'Iterator' method,
    >instances are iterable. They are not iterable otherwise.
    >
    >The above gives the expected behaviour: iter(Base()) raises a
    >"TypeError: iteration over non-sequence", and iter(Concrete()) returns a
    >generator.
    >
    >If, however, I make Base a newstyle class, this will not work any
    >longer. __getattr__ is never called for "__iter__" (neither is
    >__getattribute__, btw). Probably this has to do with data descriptors
    >and non-data descriptors, but I'm too tired at the moment to think
    >further about this.
    >
    >Is there any way I could make the above code work with new style
    >classes?

    Will a property or custom descriptor do what you want? E.g.

    >>> class Base(object):

    ... def __getIter(self):
    ... if hasattr(self, "Iterator"):
    ... return self.Iterator
    ... raise AttributeError, name
    ... __iter__ = property(__getIter)
    ...
    >>> class Concrete(Base):

    ... def Iterator(self):
    ... yield 1
    ... yield 2
    ... yield 3
    ...
    >>> iter(Base())

    Traceback (most recent call last):
    File "<stdin>", line 1, in ?
    TypeError: iteration over non-sequence
    >>> iter(Concrete())

    <generator object at 0x02EF152C>
    >>> list(iter(Concrete()))

    [1, 2, 3]

    Regards,
    Bengt Richter
    Bengt Richter, Jul 6, 2005
    #9
  10. (Bengt Richter) writes:

    > On Wed, 06 Jul 2005 17:57:42 +0200, Thomas Heller <> wrote:
    >
    >>I'm trying to implement __iter__ on an abstract base class while I don't
    >>know whether subclasses support that or not.


    > Will a property or custom descriptor do what you want? E.g.
    >
    > >>> class Base(object):

    > ... def __getIter(self):
    > ... if hasattr(self, "Iterator"):
    > ... return self.Iterator
    > ... raise AttributeError, name
    > ... __iter__ = property(__getIter)
    > ...
    > >>> class Concrete(Base):

    > ... def Iterator(self):
    > ... yield 1
    > ... yield 2
    > ... yield 3
    > ...
    > >>> iter(Base())

    > Traceback (most recent call last):
    > File "<stdin>", line 1, in ?
    > TypeError: iteration over non-sequence
    > >>> iter(Concrete())

    > <generator object at 0x02EF152C>
    > >>> list(iter(Concrete()))

    > [1, 2, 3]


    Yep, that's exactly what I need - thanks.

    Thomas
    Thomas Heller, Jul 7, 2005
    #10
  11. On Thu, 07 Jul 2005 09:51:42 +0200, Thomas Heller <> wrote:

    > (Bengt Richter) writes:
    >
    >> On Wed, 06 Jul 2005 17:57:42 +0200, Thomas Heller <> wrote:
    >>
    >>>I'm trying to implement __iter__ on an abstract base class while I don't
    >>>know whether subclasses support that or not.

    >
    >> Will a property or custom descriptor do what you want? E.g.
    >>
    >> >>> class Base(object):

    >> ... def __getIter(self):
    >> ... if hasattr(self, "Iterator"):
    >> ... return self.Iterator
    >> ... raise AttributeError, name
    >> ... __iter__ = property(__getIter)

    [...]
    >
    >Yep, that's exactly what I need - thanks.
    >

    BTW, I forgot to mention that you could use property as a decorator
    in the above single-argument case:

    >>> class Base(object):

    ... @property
    ... def __iter__(self):
    ... if hasattr(self, "Iterator"):
    ... return self.Iterator
    ... raise AttributeError, name
    ...
    >>> class Concrete(Base):

    ... def Iterator(self):
    ... yield 1
    ... yield 2
    ... yield 3
    ...
    >>> iter(Base())

    Traceback (most recent call last):
    File "<stdin>", line 1, in ?
    TypeError: iteration over non-sequence
    >>> iter(Concrete())

    <generator object at 0x02EF152C>
    >>> list(iter(Concrete()))

    [1, 2, 3]

    Hope there isn't a gotcha for your use case in the way an instance attribute
    of the same name is allowed. A custom descriptor could eliminate that.

    >>> inst = Concrete()
    >>> list(iter(inst))

    [1, 2, 3]
    >>> inst.__init__ = 'abc'
    >>> list(iter(inst))

    [1, 2, 3]
    >>> inst.__init__

    'abc'



    Regards,
    Bengt Richter
    Bengt Richter, Jul 7, 2005
    #11
  12. Thomas Heller <> writes on Wed, 06 Jul 2005 18:07:10 +0200:
    > Thomas Heller <> writes:
    > ...
    > > class Base:
    > > def __getattr__(self, name):
    > > if name == "__iter__" and hasattr(self, "Iterator"):
    > > return self.Iterator
    > > raise AttributeError, name
    > >
    > > class Concrete(Base):
    > > def Iterator(self):
    > > yield 1

    > ...
    > > If, however, I make Base a newstyle class, this will not work any
    > > longer. __getattr__ is never called for "__iter__" (neither is
    > > __getattribute__, btw). Probably this has to do with data descriptors
    > > and non-data descriptors, but I'm too tired at the moment to think
    > > further about this.
    > >
    > > Is there any way I could make the above code work with new style
    > > classes?

    >
    > I forgot to mention this: The Base class also implements a __getitem__
    > method which should be used for iteration if the .Iterator method in the
    > subclass is not available. So it seems impossible to raise an exception
    > in the __iter__ method if .Iterator is not found - __iter__ MUST return
    > an iterator if present.


    Then, it should return an interator (a new object) that uses
    the "__getitem__" method to iterate.


    Dieter
    Dieter Maurer, Jul 7, 2005
    #12
  13. (Bengt Richter) writes:

    > On Thu, 07 Jul 2005 09:51:42 +0200, Thomas Heller <> wrote:
    >
    >> (Bengt Richter) writes:
    >>
    >>> On Wed, 06 Jul 2005 17:57:42 +0200, Thomas Heller <> wrote:
    >>>
    >>>>I'm trying to implement __iter__ on an abstract base class while I don't
    >>>>know whether subclasses support that or not.

    >>
    >>> Will a property or custom descriptor do what you want? E.g.
    >>>
    >>> >>> class Base(object):
    >>> ... def __getIter(self):
    >>> ... if hasattr(self, "Iterator"):
    >>> ... return self.Iterator
    >>> ... raise AttributeError, name
    >>> ... __iter__ = property(__getIter)

    > [...]
    >>
    >>Yep, that's exactly what I need - thanks.
    >>

    > BTW, I forgot to mention that you could use property as a decorator
    > in the above single-argument case:
    >
    > >>> class Base(object):

    > ... @property
    > ... def __iter__(self):
    > ... if hasattr(self, "Iterator"):
    > ... return self.Iterator
    > ... raise AttributeError, name


    Of course. I didn't spot this, but I cannot use this anyway for 2.3
    compatibility.

    > ...
    > >>> class Concrete(Base):

    > ... def Iterator(self):
    > ... yield 1
    > ... yield 2
    > ... yield 3
    > ...
    > >>> iter(Base())

    > Traceback (most recent call last):
    > File "<stdin>", line 1, in ?
    > TypeError: iteration over non-sequence
    > >>> iter(Concrete())

    > <generator object at 0x02EF152C>
    > >>> list(iter(Concrete()))

    > [1, 2, 3]
    >
    > Hope there isn't a gotcha for your use case in the way an instance attribute
    > of the same name is allowed. A custom descriptor could eliminate that.
    >
    > >>> inst = Concrete()
    > >>> list(iter(inst))

    > [1, 2, 3]
    > >>> inst.__init__ = 'abc'
    > >>> list(iter(inst))

    > [1, 2, 3]
    > >>> inst.__init__

    > 'abc'


    I don't understand what you mean here. A __iter__ instance attribute?

    Thomas
    Thomas Heller, Jul 7, 2005
    #13
  14. On Thu, 07 Jul 2005 22:04:31 +0200, Thomas Heller <> wrote:

    > (Bengt Richter) writes:
    >
    >> On Thu, 07 Jul 2005 09:51:42 +0200, Thomas Heller <> wrote:
    >>
    >>> (Bengt Richter) writes:
    >>>
    >>>> On Wed, 06 Jul 2005 17:57:42 +0200, Thomas Heller <> wrote:
    >>>>
    >>>>>I'm trying to implement __iter__ on an abstract base class while I don't
    >>>>>know whether subclasses support that or not.
    >>>
    >>>> Will a property or custom descriptor do what you want? E.g.
    >>>>
    >>>> >>> class Base(object):
    >>>> ... def __getIter(self):
    >>>> ... if hasattr(self, "Iterator"):
    >>>> ... return self.Iterator
    >>>> ... raise AttributeError, name
    >>>> ... __iter__ = property(__getIter)

    >> [...]
    >>>
    >>>Yep, that's exactly what I need - thanks.
    >>>

    >> BTW, I forgot to mention that you could use property as a decorator
    >> in the above single-argument case:
    >>
    >> >>> class Base(object):

    >> ... @property
    >> ... def __iter__(self):
    >> ... if hasattr(self, "Iterator"):
    >> ... return self.Iterator
    >> ... raise AttributeError, name

    >
    >Of course. I didn't spot this, but I cannot use this anyway for 2.3
    >compatibility.
    >
    >> ...
    >> >>> class Concrete(Base):

    >> ... def Iterator(self):
    >> ... yield 1
    >> ... yield 2
    >> ... yield 3
    >> ...
    >> >>> iter(Base())

    >> Traceback (most recent call last):
    >> File "<stdin>", line 1, in ?
    >> TypeError: iteration over non-sequence
    >> >>> iter(Concrete())

    >> <generator object at 0x02EF152C>
    >> >>> list(iter(Concrete()))

    >> [1, 2, 3]
    >>
    >> Hope there isn't a gotcha for your use case in the way an instance attribute
    >> of the same name is allowed. A custom descriptor could eliminate that.
    >>
    >> >>> inst = Concrete()
    >> >>> list(iter(inst))

    >> [1, 2, 3]
    >> >>> inst.__init__ = 'abc'
    >> >>> list(iter(inst))

    >> [1, 2, 3]
    >> >>> inst.__init__

    >> 'abc'

    >
    >I don't understand what you mean here. A __iter__ instance attribute?
    >

    Yes, but it seems very unlikely to cause a problem, especially since iter(inst)
    bypasses it, as you probably would want. In other words, never mind ;-)

    Regards,
    Bengt Richter
    Bengt Richter, Jul 8, 2005
    #14
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Steven Bethard

    docs on for-loop with no __iter__?

    Steven Bethard, Sep 4, 2004, in forum: Python
    Replies:
    21
    Views:
    646
    Terry Reedy
    Sep 8, 2004
  2. Steven Bethard
    Replies:
    8
    Views:
    395
    Nick Coghlan
    Jan 5, 2005
  3. mrquantum

    string: __iter__()?

    mrquantum, Oct 4, 2006, in forum: Python
    Replies:
    10
    Views:
    566
    Terry Reedy
    Oct 5, 2006
  4. duccio

    __iter__ yield

    duccio, Mar 9, 2008, in forum: Python
    Replies:
    6
    Views:
    977
  5. Martin Drautzburg

    new.instancemethod __iter__

    Martin Drautzburg, Feb 6, 2010, in forum: Python
    Replies:
    7
    Views:
    297
    Steve Holden
    Feb 7, 2010
Loading...

Share This Page