Generator oddity

[opstad]

I'm a little baffled by the inconsistency here. Anyone have any
explanations?
.... yield 'a'
.... yield 'b'
.... yield 'c'
....

[c1 + c2 for c1 in gen() for c2 in gen()]

['aa', 'ab', 'ac', 'ba', 'bb', 'bc', 'ca', 'cb', 'cc']
['aa', 'ab', 'ac', 'ba', 'bb', 'bc', 'ca', 'cb', 'cc']
['aa', 'ab', 'ac']

Why does this last list only have three elements instead of nine?

 

[Steven D'Aprano]

I'm a little baffled by the inconsistency here. Anyone have any
explanations?
... yield 'a'
... yield 'b'
... yield 'c'
...

[c1 + c2 for c1 in gen() for c2 in gen()]

['aa', 'ab', 'ac', 'ba', 'bb', 'bc', 'ca', 'cb', 'cc']
['aa', 'ab', 'ac', 'ba', 'bb', 'bc', 'ca', 'cb', 'cc']
['aa', 'ab', 'ac']

Why does this last list only have three elements instead of nine?

Ah, good one! That had me puzzled for a while too.

The answer is to write it out as a nested for-loop, using print in place
of yield. Here's the first way:

for c1 in gen():
for c2 in gen():
print c1 + c2

And the second:

it1 = gen()
it2 = gen()
for c1 in it1:
for c2 in it2:
print c1 + c2


In the first example, the inner loop gets refreshed each time through the
outer loop with a brand new instance of the generator. Expanding the
loops in full:

# First method:
c1 = 'a'
call gen() to make an iterable
step through the fresh iterable, giving 'aa' 'ab' 'ac'
c1 = 'b'
call gen() to make an iterable
step through the fresh iterable, giving 'ba' 'bb' 'bc'
c1 = 'c'
call gen() to make an iterable
step through the fresh iterable, giving 'ca' 'cb' 'cc'


# Second method:
c1 = 'a'
inner iterable already exists
step through the iterable, giving 'aa' 'ab' 'ac'
c1 = 'b'
inner iterable is exhausted, so do nothing
c1 = 'c'
inner iterable is exhausted, so do nothing


And there you have it. A nice Gotcha for the books.

 

[opstad]

D'oh! Many thanks for the explanation; I should have seen that myself.

 

[Dave Angel]

I'm a little baffled by the inconsistency here. Anyone have any
explanations?

... yield 'a'
... yield 'b'
... yield 'c'
...

[c1 + c2 for c1 in gen() for c2 in gen()]

['aa', 'ab', 'ac', 'ba', 'bb', 'bc', 'ca', 'cb', 'cc']

['aa', 'ab', 'ac', 'ba', 'bb', 'bc', 'ca', 'cb', 'cc']

['aa', 'ab', 'ac']

Why does this last list only have three elements instead of nine?

The problem is in it2, which is initialized only once. Thus the second
time you're going through the c2 loop, it doesn't have any more values.

If you need the indirection provided by those it1 and it2, you need to
postpone the function evaluation, at least for it2.

I'd do
it1 = gen
it2 = gen

list(c1 + c2 for c1 in it1() for c2 in it2())
or more simply:
[c1 + c2 for c1 in it1() for c2 in it2()]

 

[Andre Engels]

The problem is in it2, which is initialized only once.  Thus the second time
you're going through the c2 loop, it doesn't have any more values.

If you need the indirection provided by those it1 and it2, you need to
postpone the function evaluation, at least for it2.

I'd do
it1 = gen
it2 = gen

list(c1 + c2 for c1 in it1() for c2 in it2())
or more simply:
[c1 + c2 for c1 in it1() for c2 in it2()]

Another possibility would be to use the outcome of looping through the
generator rather than the generator itself:

it1 = list(gen())
it2 = list(gen())
[c1 + c2 for c1 in it1() for c2 in it2()]