keeping state in an iterator object by rebinding next()

[Wilbert Berendsen]

Hi,

i am writing a simple parser, that generates tokens. The parser needs to
maintain some state, because some parts of the file consist of different
tokens. I thought the object could simply remember its state by assigning
it's next() method to the method that is currently parsing. When the state
changes, the called method rebinds next() and the next token will be returned
by that function. Here's an example, proving that this indeed works.
.... def a(self):
.... self.next = self.b
.... return 1
.... def b(self):
.... self.next = self.a
.... return 2
.... def __iter__(self):
.... return self
........ j += 1
.... if j > 10: break # prevent from running endlessly
.... print i
....
2
1
2
1
2
1
2
1
2
1
my question is: is this legal Python? An iterator could save the next() method
object, and in that case it could stop working.... It works now, because
apparently the for- construct resolves 'next' each time for the object before
calling it.

The other solution would be just jumping to the correct method from within the
next() method. But that gives an extra call...


Met vriendelijke groet,
Wilbert Berendsen

 

[Peter Otten]

Hi,

i am writing a simple parser, that generates tokens. The parser needs to
maintain some state, because some parts of the file consist of different
tokens. I thought the object could simply remember its state by assigning
it's next() method to the method that is currently parsing. When the state
changes, the called method rebinds next() and the next token will be
returned by that function. Here's an example, proving that this indeed
works.

... def a(self):
... self.next = self.b
... return 1
... def b(self):
... self.next = self.a
... return 2
... def __iter__(self):
... return self
...
... j += 1
... if j > 10: break # prevent from running endlessly
... print i
...
2
1
2
1
2
1
2
1
2
1

my question is: is this legal Python? An iterator could save the next()
method object, and in that case it could stop working.... It works now,
because apparently the for- construct resolves 'next' each time for the
object before calling it.

The other solution would be just jumping to the correct method from within
the next() method. But that gives an extra call...

It doesn't work with newstyle classes though as next() is looked up in the
class rather than the instance, just like the __xxx__() methods:
.... def __iter__(self): return self
.... def next(self): return "class-next()"
........
class-next()
class-next()
class-next()

Without the class-next() it isn't even recognized as an iterator
....
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: iter() returned non-iterator of type 'A'

even though the instance-next() is still there:
'instance-next()'

So I recommend that you avoid that can of worms and go with the extra
indirection.

Peter

 

[iteration.nth]

Hi,

i am writing a simple parser, that generates tokens. The parser needs to
maintain some state, because some parts of the file consist of different
tokens. I thought the object could simply remember its state by assigning
it's next() method to the method that is currently parsing. When the state
changes, the called method rebinds next() and the next token will be returned
by that function. Here's an example, proving that this indeed works.


... def a(self):
... self.next = self.b
... return 1
... def b(self):
... self.next = self.a
... return 2
... def __iter__(self):
... return self
...>>> a=A()

... j += 1
... if j > 10: break # prevent from running endlessly
... print i
...
2
1
2
1
2
1
2
1
2
1



my question is: is this legal Python? An iterator could save the next() method
object, and in that case it could stop working.... It works now, because
apparently the for- construct resolves 'next' each time for the object before
calling it.

The other solution would be just jumping to the correct method from within the
next() method. But that gives an extra call...

Met vriendelijke groet,
Wilbert Berendsen

--http://www.wilbertberendsen.nl/
"You must be the change you wish to see in the world."
-- Mahatma Gandi

""""

"You must be the change you wish to see in the world."
-- Mahatma Gandi
""""
JFYI: Mahatma Gandhi (NOT Gandi).

 

[Wilbert Berendsen]

Thanks for all the elaborate answers and help! It helped me deepening my
understanding of Python.
Sincere,
Wilbert Berendsen