More baby squeaking - iterators in a class

[Bulba!]

Hello Mr Everyone,

From:
http://docs.python.org/tut/node11.html#SECTION0011900000000000000000

"Define a __iter__() method which returns an object with a next()
method. If the class defines next(), then __iter__() can just return
self:"

The thing is, I tried to define __iter__() directly without explicit
defining next (after all, the conclusion from this passage should
be that it's possible).


class R:
def __init__(self, d):
self.d=d
self.i=len(d)
def __iter__(self):
if self.i == 0:
raise StopIteration
self.i -= 1
return self.d[self.i]

['__doc__', '__init__', '__iter__', '__module__', 'd', 'i']

Apparently no, there is no next() method. Let's see
if iterator works:
Traceback (most recent call last):
File "<interactive input>", line 1, in ?
File "<interactive input>", line 7, in __iter__
StopIteration

OK, this part works. But this:
print i

Traceback (most recent call last):
File "<interactive input>", line 1, in ?
TypeError: __iter__ returned non-iterator of type 'str'

So which is it? Does next() method HAS to be defined
explicitly? That's what Wikipedia says:

http://en.wikipedia.org/wiki/Iterator#Python

"Any user defined class can support standard iteration (either
implicit or explicit) by defining an __iter__() method which creates
an iterator object. The iterator object then needs to define both an
__iter__() method as well as a next() method."

 

[Russell Blau]

Hello Mr Everyone,

From:
http://docs.python.org/tut/node11.html#SECTION0011900000000000000000

"Define a __iter__() method which returns an object with a next()
method. If the class defines next(), then __iter__() can just return
self:"

The thing is, I tried to define __iter__() directly without explicit
defining next (after all, the conclusion from this passage should
be that it's possible).

I don't get that from the passage quoted, at all, although it is somewhat
opaque. It says that your __iter__() method must *return an object* with a
next() method; your __iter__() method below doesn't return such an object,
but instead returns a string. It then says that *if* your class defines
next(), which yours doesn't, __iter__() can return self.

[spaces inserted; you should note that many newsreaders strip the TAB
character...]

class R:
def __init__(self, d):
self.d=d
self.i=len(d)
def __iter__(self):
if self.i == 0:
raise StopIteration
self.i -= 1
return self.d[self.i]

Solution: replace "__iter__" with "next" in the class definition above,
then add to the end:

def __iter__(self):
return self

 

[Alex Martelli]

So which is it? Does next() method HAS to be defined
explicitly?

It has to be defined, whether explicitly or not (e.g. via a generator).


Alex

 

[Terry Reedy]

"Define a __iter__() method which returns an object with a next()
method. If the class defines next(), then __iter__() can just return
self:"

The thing is, I tried to define __iter__() directly without explicit
defining next (after all, the conclusion from this passage should
be that it's possible).

It is, see below.

class R:
def __init__(self, d):
self.d=d
self.i=len(d)
def __iter__(self):
if self.i == 0:
raise StopIteration
self.i -= 1
return self.d[self.i]

Change 'return' to 'yield'. Then r.__iter__() *will* return a
(generator)iterator with a .next method, as required. But it will only
yield one value before running off the end. Since __iter__ is meant to be
called only once, you need to loop explicitly in the generator. For
instance

def __iter__(self):
i,d = self.i, self.d
while i
i =- 1
yield d

Copying i makes the generator non-destuctive.

(PS, use spaces, not tabs, in posted code.)

Terry J. Reedy

 

[Scott David Daniels]

Hello Mr Everyone,

From:
http://docs.python.org/tut/node11.html#SECTION0011900000000000000000

"Define a __iter__() method which returns an object with a next()
method. If the class defines next(), then __iter__() can just return
self:"

The thing is, I tried to define __iter__() directly without explicit
defining next (after all, the conclusion from this passage should
be that it's possible).


class R:
def __init__(self, d):
self.d=d
self.i=len(d)
def __iter__(self):
if self.i == 0:
raise StopIteration
self.i -= 1
return self.d[self.i]

Here's one way: # (Make __iter__ an iterator)

Py> class R1(object):
def __init__(self, data):
self.data = data
self.i = len(data)
def __iter__(self):
while self.i > 0:
self.i -= 1
yield self.data[self.i]

Py> s=R1('spam')
Py> list(s)
['m', 'a', 'p', 's']
Py> list(s)
[]
Py> s.i = 3
Py> list(s)
['a', 'p', 's']

Here's another way: # (Return something with __iter__ and next methods)

Py> class R2(object):
def __init__(self, data):
self.d = data
self.i = len(data)
def __iter__(self):
return iter(self.d)
Py> s = R2('spam')
Py> list(s)
['s', 'p', 'a', 'm']
Py> list(s)
['s', 'p', 'a', 'm']

--Scott David Daniels
(e-mail address removed)

 

[Bulba!]

Here's one way: # (Make __iter__ an iterator)
Py> class R1(object):
def __init__(self, data):
self.data = data
self.i = len(data)
def __iter__(self):
while self.i > 0:
self.i -= 1
yield self.data[self.i]

Thanks to everyone for their responses, but it still doesn't work re
returning next() method:class R3:
def __init__(self, d):
self.d=d
self.i=len(d)
def __iter__(self):
d,i = self.d, self.i
while i>0:
i-=1
yield d
Traceback (most recent call last):

dir(p) ['__doc__', '__init__', '__iter__', '__module__', 'd', 'i']
list(p) ['s', 'g', 'g', 'e']

I tried all the methods advised by you and other posters and they do
return an object with __iter__, but not with the next method.

What's strange is that when it comes to function, it does return
the .next method:

def rev(d):
for i in range (len(d)-1, -1, -1):
yield d
['__class__', '__delattr__', '__doc__', '__getattribute__',
'__hash__', '__init__', '__iter__', '__new__', '__reduce__',
'__reduce_ex__', '__repr__', '__setattr__', '__str__', 'gi_frame',
'gi_running', 'next']'e'


The function returns 'generator object', as shown above,
while p is a class instance:
<__main__.R3 instance at 0x0123CA58>

 

[M.E.Farmer]

Reread Russel Blau post he is spot on with his comments:

I don't get that from the passage quoted, at all, although it is somewhat
opaque. It says that your __iter__() method must *return an object* with a
next() method; your __iter__() method below doesn't return such an object,
but instead returns a string. It then says that *if* your class defines
next(), which yours doesn't, __iter__() can return self.

[spaces inserted; you should note that many newsreaders strip the TAB
character...]

class R:
def __init__(self, d):
self.d=d
self.i=len(d)
def __iter__(self):
if self.i == 0:
raise StopIteration
self.i -= 1
return self.d[self.i]

Solution: replace "__iter__" with "next" in the class definition above,
then add to the end:

That works exactly as advertised.

py> s = R('54645656')
.... s.next()
'6'
.... s.next()
'5'
.... s.next()
'6'
.... s.next()
'5'
.... s.next()
'4'
... s.next()
'6'
.... s.next()
'4'
.... s.next()
'5'
.... s.next()
Traceback (most recent call last):
File "<input>", line 1, in ?
File "<input>", line 7, in next
StopIteration

And the other posters showed you how to create an iterator without a
next() .
Other than having no next() method they still work the same.

 

[Terry Reedy]

Thanks to everyone for their responses, but it still doesn't work re
returning next() method:class R3:
def __init__(self, d):
self.d=d
self.i=len(d)
def __iter__(self):
d,i = self.d, self.i
while i>0:
i-=1
yield d

This is the wrong test for what I and some others thought you were asking.
The requirement for p to be an *iterable* and useable in code such as 'for
i in p' is that iter(p), not p itself, have a .next method, and iter(p)
will. Try ip=iter(p) followed by ip.next and ip.next() instead.

If, for whatever reason, you instead want p to actually be an *iterator*
with a .next (or .getitem) method, then, of course, you have to actually
give it a .next (or .getitem) method.

Terry J. Reedy

 

[Steven Bethard]

Thanks to everyone for their responses, but it still doesn't work re
returning next() method:

class R3:
def __init__(self, d):
self.d=d
self.i=len(d)
def __iter__(self):
d,i = self.d, self.i
while i>0:
i-=1
yield d

[snip]

What's strange is that when it comes to function, it does return
the .next method:

def rev(d):
for i in range (len(d)-1, -1, -1):
yield d

o=rev('eggs')

[snip]

o.next()

's'

Note the difference here. When you're using the function, you call the
iter function (called rev in your example). When you're using the
class, you haven't called the iter function, only instantiated the class
(i.e. called the __init__ function). Try one of the following:

py> p = R3('eggs')
py> i = p.__iter__()
py> i.next()
's'

or

py> p = R3('eggs')
py> i = iter(p)
py> i.next()
's'

Steve

 

[M.E.Farmer]

This is the wrong test for what I and some others thought you were asking.
The requirement for p to be an *iterable* and useable in code such as 'for
i in p' is that iter(p), not p itself, have a .next method, and iter(p)
will. Try ip=iter(p) followed by ip.next and ip.next() instead.

Does that mean if you dont't call iter(() on your instance or have a
next() method you can't do this:

class R3:
def __init__(self, d):
self.d=d
self.i=len(d)
def __iter__(self):
d,i = self.d, self.i
while i>0:
i-=1
yield d

I am asking because I want to fully understand what makes an
*iterator*.

M.E.Farmer

 

[Bulba!]

py> p = R3('eggs')
py> i = p.__iter__()
py> i.next()
's'

py> p = R3('eggs')
py> i = iter(p)
py> i.next()
's'

And that is precisely what I needed to know. Thanks, to you,
Terry and everyone who took time to look at it.

 

[Terry Reedy]

Does that mean if you dont't call iter(() on your instance or have a
next() method you can't do this:

Ignoring the older iteration method based on __getitem__, which I recommend
you do ignore, yes, you cannot do that.

I am asking because I want to fully understand what makes an
*iterator*.

An iterator is an object with an __iter__ method that returns 'self' and a
parameterless .next() method that on each call either returns an object or
raises StopIteration.

An iterable is an object with an __iter__ method that returns an iterator.

A generator function (one containing yield in the code body), when called,
returns a generator, which is a particular type of iterator. So, a class
with .__iter__ written as a generator function has iterable instances.

I can think of three reasons to go this latter route:
1. Writing a generator function may be easier that writing a standard
function for .next.
2. The resulting iteration may be faster.
3. Iterating with a separate generator allows iteration without disturbing
the data of the instance (ie, non destructively). This means that one can
iterate more than once, even more than once at the same time. Consider
'multiplying' a sequence by itself:

(i,j) for i in iterable for j in iterable

iter(iterable) is called once and then once again for each item yielded by
the first call.

Terry J. Reedy

 

[M.E.Farmer]

Terry ,
Thank you for the explanation . That is much clearer now, I have played
a bit with generators but have never tried to create a custom iterator.
I am just now getting into the internals part of python objects... this
langauage is still amazing to me!
The reason I asked the question was because I tried that code I posted
and it worked fine in python2.2.3 ?

Ignoring the older iteration method based on __getitem__, which I recommend
you do ignore, yes, you cannot do that.

py>class R3:
.... def __init__(self, d):
.... self.d=d
.... self.i=len(d)
.... def __iter__(self):
.... d,i = self.d, self.i
.... while i>0:
.... i-=1
.... yield d
....
py>a = R3('qwerty')
py>dir(a)
['__doc__', '__init__', '__iter__', '__module__', 'd', 'i']
py>for i in a:
.... print i
....
y
t
r
e
w
q
Ok I don't see __getitem__ anywhere in this object.
Still scratchin my head? Why does this work?
M.E.Farmer

 

[M.E.Farmer]

py> from __future__ import generators
Hello again,
I was thinking about the __iter__ and what you were saying about
generators, so I opened up pyshell and started typing.
py> class R3:
.... def __init__(self, d):
.... self.d=d
.... self.i=len(d)
.... def __iter__(self):
.... d,i = self.d, self.i
.... while i>0:
.... i-=1
.... yield d
....
py> r = R3('qwerty')
py> r.__class__
<class __main__.R3 at 0x01440A38>
py> r.__iter__
<bound method R3.__iter__ of <__main__.R3 instance at 0x0141BFB0>>
py> r.__iter__()
<stackless.generator object at 0x0143FE38>
py> a = r.__iter__()
py> a.next()
'y'

py> a.next()
't'
py> a.next()
'r'
py> a.next()
'e'
py> a.next()
'w'
py> a.next()
'q'
py> a.next()
Traceback (most recent call last):

Doh! ok now I get it __iter__ RETURNS a generator
It seems so obvious now I look at it.
Sometimes I am a bit slow to catch on , but I never forget, much.

M.E.Farmer
Deeper down the rabbit hole.

Terry ,
Thank you for the explanation . That is much clearer now, I have played
a bit with generators but have never tried to create a custom iterator.
I am just now getting into the internals part of python objects... this
langauage is still amazing to me!
The reason I asked the question was because I tried that code I posted
and it worked fine in python2.2.3 ?

Ignoring the older iteration method based on __getitem__, which I recommend
you do ignore, yes, you cannot do that.

py>class R3:
... def __init__(self, d):
... self.d=d
... self.i=len(d)
... def __iter__(self):
... d,i = self.d, self.i
... while i>0:
... i-=1
... yield d
...
py>a = R3('qwerty')
py>dir(a)
['__doc__', '__init__', '__iter__', '__module__', 'd', 'i']
py>for i in a:
... print i
...
y
t
r
e
w
q
Ok I don't see __getitem__ anywhere in this object.
Still scratchin my head? Why does this work?
M.E.Farmer