Conditionally implementing __iter__ in new style classes

[Thomas Heller]

I'm trying to implement __iter__ on an abstract base class while I don't
know whether subclasses support that or not.
Hope that makes sense, if not, this code should be clearer:

class Base:
def __getattr__(self, name):
if name == "__iter__" and hasattr(self, "Iterator"):
return self.Iterator
raise AttributeError, name

class Concrete(Base):
def Iterator(self):
yield 1
yield 2
yield 3

The idea is that if a subclass of Base defines an 'Iterator' method,
instances are iterable. They are not iterable otherwise.

The above gives the expected behaviour: iter(Base()) raises a
"TypeError: iteration over non-sequence", and iter(Concrete()) returns a
generator.

If, however, I make Base a newstyle class, this will not work any
longer. __getattr__ is never called for "__iter__" (neither is
__getattribute__, btw). Probably this has to do with data descriptors
and non-data descriptors, but I'm too tired at the moment to think
further about this.

Is there any way I could make the above code work with new style
classes?

Thanks,

Thomas

 

[Thomas Heller]

I'm trying to implement __iter__ on an abstract base class while I don't
know whether subclasses support that or not.
Hope that makes sense, if not, this code should be clearer:

class Base:
def __getattr__(self, name):
if name == "__iter__" and hasattr(self, "Iterator"):
return self.Iterator
raise AttributeError, name

class Concrete(Base):
def Iterator(self):
yield 1
yield 2
yield 3

The idea is that if a subclass of Base defines an 'Iterator' method,
instances are iterable. They are not iterable otherwise.

The above gives the expected behaviour: iter(Base()) raises a
"TypeError: iteration over non-sequence", and iter(Concrete()) returns a
generator.

If, however, I make Base a newstyle class, this will not work any
longer. __getattr__ is never called for "__iter__" (neither is
__getattribute__, btw). Probably this has to do with data descriptors
and non-data descriptors, but I'm too tired at the moment to think
further about this.

Is there any way I could make the above code work with new style
classes?

I forgot to mention this: The Base class also implements a __getitem__
method which should be used for iteration if the .Iterator method in the
subclass is not available. So it seems impossible to raise an exception
in the __iter__ method if .Iterator is not found - __iter__ MUST return
an iterator if present.

Thomas

 

[infidel]

I'm not sure I understand why you would want to. Just don't define
__iter__ on your newstyle class and you'll get the expected behavior.

 

[infidel]

Why not define an Iterator method in your Base class that does the
iteration using __getitem__, and any subclass that wants to do
something else just defines its own Iterator method? For that matter,
you could just use the __iter__ methods of Base and Concrete instead of
a separate method.

 

[harold fellermann]

I'm trying to implement __iter__ on an abstract base class while I

don't
know whether subclasses support that or not.
Hope that makes sense, if not, this code should be clearer:

class Base:
def __getattr__(self, name):
if name == "__iter__" and hasattr(self, "Iterator"):
return self.Iterator
raise AttributeError, name

class Concrete(Base):
def Iterator(self):
yield 1
yield 2
yield 3

I don't know how to achieve it, but why don't you simply use

class Base:
pass

class Concrete(Base):
def __iter__(self) :
yield 1
yield 2
yield 3


What is the advantage to have a baseclass that essentially does
some method renaming __iter__ ==> Iterator?

- harold -

 

[infidel]

Something like this:
.... def __getitem__(self, key):
.... return key
.... def __iter__(self):
.... yield self[1]
.... yield self['foo']
.... yield self[3.0]
........ def __iter__(self):
.... yield True
.... yield 'Blue'
.... yield 'Foo'
........ pass
....

[x for x in Base()] [1, 'foo', 3.0]
[x for x in ConcreteIterable()] [True, 'Blue', 'Foo']
[x for x in ConcreteNotIterable()] [1, 'foo', 3.0]

 

[Leif K-Brooks]

I forgot to mention this: The Base class also implements a __getitem__
method which should be used for iteration if the .Iterator method in the
subclass is not available. So it seems impossible to raise an exception
in the __iter__ method if .Iterator is not found - __iter__ MUST return
an iterator if present.

def Iterator(self):
for index in xrange(len(self)):
yield self[index]

def __iter__(self):
return self.Iterator()

....and then override Iterator in subclasses. But this raises the
question of why you need to use a specially-named method instead of
having subclasses override the __iter__.

 

[Peter Otten]

I'm trying to implement __iter__ on an abstract base class while I don't
know whether subclasses support that or not.
Hope that makes sense, if not, this code should be clearer:

class Base:
def __getattr__(self, name):
if name == "__iter__" and hasattr(self, "Iterator"):
return self.Iterator
raise AttributeError, name

Is there any way I could make the above code work with new style
classes?

Obligatory metaclass approach:

class Base:
class __metaclass__(type):
def __new__(mcl, name, bases, classdict):
try:
classdict["__iter__"] = classdict["Iterator"]
except KeyError:
pass
return type.__new__(mcl, name, bases, classdict)

class Alpha(Base):
def Iterator(self): yield 42

class Beta(Base):
def __getitem__(self, index):
return [1, 2, 3, "ganz viele"][index]


for item in Alpha(): print item
for item in Beta(): print item,
print

Peter

 

[Bengt Richter]

I'm trying to implement __iter__ on an abstract base class while I don't
know whether subclasses support that or not.
Hope that makes sense, if not, this code should be clearer:

class Base:
def __getattr__(self, name):
if name == "__iter__" and hasattr(self, "Iterator"):
return self.Iterator
raise AttributeError, name

class Concrete(Base):
def Iterator(self):
yield 1
yield 2
yield 3

The idea is that if a subclass of Base defines an 'Iterator' method,
instances are iterable. They are not iterable otherwise.

The above gives the expected behaviour: iter(Base()) raises a
"TypeError: iteration over non-sequence", and iter(Concrete()) returns a
generator.

If, however, I make Base a newstyle class, this will not work any
longer. __getattr__ is never called for "__iter__" (neither is
__getattribute__, btw). Probably this has to do with data descriptors
and non-data descriptors, but I'm too tired at the moment to think
further about this.

Is there any way I could make the above code work with new style
classes?

Will a property or custom descriptor do what you want? E.g.
... def __getIter(self):
... if hasattr(self, "Iterator"):
... return self.Iterator
... raise AttributeError, name
... __iter__ = property(__getIter)
... ... def Iterator(self):
... yield 1
... yield 2
... yield 3
... Traceback (most recent call last):
[1, 2, 3]

Regards,
Bengt Richter

 

[Thomas Heller]

Will a property or custom descriptor do what you want? E.g.
... def __getIter(self):
... if hasattr(self, "Iterator"):
... return self.Iterator
... raise AttributeError, name
... __iter__ = property(__getIter)
...... def Iterator(self):
... yield 1
... yield 2
... yield 3
...Traceback (most recent call last):
[1, 2, 3]

Yep, that's exactly what I need - thanks.

Thomas

 

[Bengt Richter]

Will a property or custom descriptor do what you want? E.g.

... def __getIter(self):
... if hasattr(self, "Iterator"):
... return self.Iterator
... raise AttributeError, name
... __iter__ = property(__getIter)

[...]

Yep, that's exactly what I need - thanks.

BTW, I forgot to mention that you could use property as a decorator
in the above single-argument case:
... @property
... def __iter__(self):
... if hasattr(self, "Iterator"):
... return self.Iterator
... raise AttributeError, name
... ... def Iterator(self):
... yield 1
... yield 2
... yield 3
... Traceback (most recent call last):
[1, 2, 3]

Hope there isn't a gotcha for your use case in the way an instance attribute
of the same name is allowed. A custom descriptor could eliminate that.

>>> inst = Concrete()
>>> list(iter(inst)) [1, 2, 3]
>>> inst.__init__ = 'abc'
>>> list(iter(inst)) [1, 2, 3]
>>> inst.__init__

'abc'



Regards,
Bengt Richter

 

[Dieter Maurer]

I forgot to mention this: The Base class also implements a __getitem__
method which should be used for iteration if the .Iterator method in the
subclass is not available. So it seems impossible to raise an exception
in the __iter__ method if .Iterator is not found - __iter__ MUST return
an iterator if present.

Then, it should return an interator (a new object) that uses
the "__getitem__" method to iterate.


Dieter

 

[Thomas Heller]

I'm trying to implement __iter__ on an abstract base class while I don't
know whether subclasses support that or not.

Will a property or custom descriptor do what you want? E.g.

class Base(object):
... def __getIter(self):
... if hasattr(self, "Iterator"):
... return self.Iterator
... raise AttributeError, name
... __iter__ = property(__getIter)

[...]

Yep, that's exactly what I need - thanks.

BTW, I forgot to mention that you could use property as a decorator
in the above single-argument case:
... @property
... def __iter__(self):
... if hasattr(self, "Iterator"):
... return self.Iterator
... raise AttributeError, name

Of course. I didn't spot this, but I cannot use this anyway for 2.3
compatibility.

...... def Iterator(self):
... yield 1
... yield 2
... yield 3
...Traceback (most recent call last):
[1, 2, 3]

Hope there isn't a gotcha for your use case in the way an instance attribute
of the same name is allowed. A custom descriptor could eliminate that.

inst = Concrete()
list(iter(inst)) [1, 2, 3]
inst.__init__ = 'abc'
list(iter(inst)) [1, 2, 3]
inst.__init__

'abc'

I don't understand what you mean here. A __iter__ instance attribute?

Thomas

 

[Bengt Richter]

(e-mail address removed) (Bengt Richter) writes:


I'm trying to implement __iter__ on an abstract base class while I don't
know whether subclasses support that or not.

Will a property or custom descriptor do what you want? E.g.

class Base(object):
... def __getIter(self):
... if hasattr(self, "Iterator"):
... return self.Iterator
... raise AttributeError, name
... __iter__ = property(__getIter) [...]

Yep, that's exactly what I need - thanks.

BTW, I forgot to mention that you could use property as a decorator
in the above single-argument case:

class Base(object):

... @property
... def __iter__(self):
... if hasattr(self, "Iterator"):
... return self.Iterator
... raise AttributeError, name

Of course. I didn't spot this, but I cannot use this anyway for 2.3
compatibility.

...

class Concrete(Base):

... def Iterator(self):
... yield 1
... yield 2
... yield 3
...

iter(Base())

Traceback (most recent call last):

iter(Concrete())

list(iter(Concrete()))

[1, 2, 3]

Hope there isn't a gotcha for your use case in the way an instance attribute
of the same name is allowed. A custom descriptor could eliminate that.

inst = Concrete()
list(iter(inst)) [1, 2, 3]
inst.__init__ = 'abc'
list(iter(inst)) [1, 2, 3]
inst.__init__

'abc'

I don't understand what you mean here. A __iter__ instance attribute?

Yes, but it seems very unlikely to cause a problem, especially since iter(inst)
bypasses it, as you probably would want. In other words, never mind ;-)

Regards,
Bengt Richter