# iterator idea

Discussion in 'Python' started by Paul Rubin, Oct 26, 2006.

1. ### Paul RubinGuest

I wonder if Python would benefit from letting a generator jump to
another iterator, i.e. something like

yield return *seq

would be the equivalent of

for s in seq: yield s
return

except that it would work by transferring control directly to seq's
iterator (like a goto) instead of calling it like a subroutine. The
idea is to solve one of the perennial iterator annoyances, the
inability to push items back on an iterator. For example, the
annoyance means itertools.takewhile consumes an extra element with no
reasonable way to retrieve it. The new feature would let us write
something like (obviously untested):

def takewhile_exact (pred, seq):
itertools.takewhile(pred,seq). tail is the rest of the
elements of seq, with none missing. You can't use tail
buf = []
for s in seq:
if pred(s): yield s
# save non-matching item so that tail can find it
buf.append(s)

def tail():
if not buf:
yield buf.pop()
yield return *seq # transfer control to seq

The reason tail() can't just iterate through seq is that you
might call takewhile_exact many times:

def is_even(n): return (n%2 == 0)
def is_odd(n): return (n%2 != 0)

# we want to do something with all the runs of even numbers
# in the sequence, and similarly with the odds. seq is an
# infinite sequence.
while True:
evens, seq = takewhile_exact(is_even, seq)
do_something_with (evens)
odds, seq = takewhile_exact(is_odd, seq)
do_something_else_with (odds)

Without the "goto"-like transfer, we'd get deeper and deeper in nested
iterators and eventually overflow the call stack.

Paul Rubin, Oct 26, 2006

2. ### Duncan BoothGuest

Paul Rubin <http://> wrote:
....
> For example, the annoyance means itertools.takewhile consumes an extra
> element with no reasonable way to retrieve it.

....
>
> def is_even(n): return (n%2 == 0)
> def is_odd(n): return (n%2 != 0)
>
> # we want to do something with all the runs of even numbers
> # in the sequence, and similarly with the odds. seq is an
> # infinite sequence.
> while True:
> evens, seq = takewhile_exact(is_even, seq)
> do_something_with (evens)
> odds, seq = takewhile_exact(is_odd, seq)
> do_something_else_with (odds)
>
> Without the "goto"-like transfer, we'd get deeper and deeper in nested
> iterators and eventually overflow the call stack.

I wouldn't agree that there is no way reasonable way to get the terminating
value with takewhile, you just need another generator or two:

>>> def repeatable(iterator):

it = iter(iterator)
for v in it:
while (yield v):
yield None

>>> def takeparts(predicates, iterator):

iterator = repeatable(iterator)
while True:
for pred in predicates:
yield takewhile(pred, iterator)
iterator.send(True)

>>> for seq in takeparts([is_even, is_odd], [2,2,4,5,6,6,7]):

print list(seq)

[2, 2, 4]
[5]
[6, 6]
[7]

Duncan Booth, Oct 26, 2006

3. ### Paul RubinGuest

Duncan Booth <> writes:
> I wouldn't agree that there is no way reasonable way to get the terminating
> value with takewhile, you just need another generator or two:

Hmm, I hadn't thought about iterator.send. I'll have to look at that
more carefully and re-read the PEP.

Paul Rubin, Oct 26, 2006