Generators and their next() and send() methods

[Thomas Mlynarczyk]

Hello,

I was playing around a bit with generators using next() and send(). And
I was wondering why an extra send() method was introduced instead of
simply allowing an argument for next().

Also, I find it a bit counter-intuitive that send(42) not only "sets"
the generator to the specified value, but yields the next value at the
same time.

As an exercise, I wanted to somehow "modify" a generator so that it
would have a next() method accepting an argument, something like

@myway
def gen():
pass

But I failed to come up with an implementation of the myway() function.

Any comments and/or suggestions?

Greetings,
Thomas

 

[Arnaud Delobelle]

Hello,

I was playing around a bit with generators using next() and
send(). And I was wondering why an extra send() method was introduced
instead of simply allowing an argument for next().

Also, I find it a bit counter-intuitive that send(42) not only "sets"
the generator to the specified value, but yields the next value at the
same time.

If you want to simply 'set' the generator (by which I take you mean
'change its state') without without iterating it one step, then what you
need is a class with an __iter__() method. Then you can change the
state of the object between calls to next(). E.g.
.... def __init__(self, v): self.v = v
.... def __iter__(self):
.... for x in range(10):
.... yield self.v
.... .... g.v = input("val:")
.... print i
....
val:4
5
val:10
4
val:34
10
val:56
34
val:

Etc...

 

[Thomas Mlynarczyk]

If you want to simply 'set' the generator (by which I take you mean
'change its state') without without iterating it one step, then what you
need is a class with an __iter__() method. Then you can change the
state of the object between calls to next(). E.g.
... def __init__(self, v): self.v = v
... def __iter__(self):
... for x in range(10):
... yield self.v
...
... g.v = input("val:")
... print i

That's a good idea, thanks! Meanwhile I have succeeded in writing a
decorator which allows an argument for next():

def myway( g ):
class mygen( object ):
def __init__( self, n ):
self.g = g( n )
def __iter__( self ):
return self
def next( self, s = None ):
return self.g.next() if s is None else self.g.send( s )
def send( self, s ):
return self.g.send( s )
return mygen

@myway
def gen( n ):
i = 0
while i <= n:
s = yield i
if s is not None:
i = s
else:
i += 1

Of course, this is just an exercise -- it doesn't look as if it's worth
using @mayway in any "real" code just to be able to write g.next(42)
instead of g.send(42). Still, I would like to know why it was decided to
introduce a send() method instead of allowing an argument for next().

Thanks and greetings,
Thomas

 

[alex23]

Still, I would like to know why it was decided to
introduce a send() method instead of allowing an argument for next().

Hey Thomas,

A great place to gain insight into the reasoning behind changes to
Python is the PEPs:

http://www.python.org/dev/peps/pep-0342/

That links to the original proposal to extend the generator behaviour
to include send. I haven't the time right now to read it but if
anything is going to explain the choice, it'll be there.

At a guess, I'd expect a new method was chosen to provide semantic
distinctness with the original behaviour. Or maybe so it wouldn't
break any existing code with decorators such as your's

 

[Aaron Brady]

That's a good idea, thanks! Meanwhile I have succeeded in writing a
decorator which allows an argument for next():

Here's a different structure if you want it. Untested.

def myway( g ):
def init( *ar, **kw ):
g= g( *ar, **kw )
     class mygen( object ):
         def __iter__( self ):
             return self
         def next( self, s= None ):
             return g.next() if s is None else g.send( s )
         def send( self, s ):
             return g.send( s )
     return mygen
return init

And, if you don't intend to use 'myway' on 'listiterator's and such,
'send( None )' is equivalent to 'next( )'.

def myway( g ):
def init( *ar, **kw ):
g= g( *ar, **kw )
class mygen( object ):
def __iter__( self ):
return self
def next( self, s= None ):
return g.send( s )
def send( self, s ):
return g.send( s )
return mygen
return init

There's probably a typo in it.

 

[Thomas Mlynarczyk]

http://www.python.org/dev/peps/pep-0342/
That links to the original proposal to extend the generator behaviour

After some searching, I found this as a remark in parentheses:

"Introducing a new method instead of overloading next() minimizes
overhead for simple next() calls."

And also a (little) more detailed explanation. So, basically, allowing
an argument for next() would have meant more overhead in the Python
implementation. Still, I don't quite see why, as this could be detected
at compile time, couldn't it?

On the other hand, if I understood correctly, I can use send(None) as an
equivalent of next(). So, while I cannot have next(argument) instead of
send(argument), I can have send(None) instead of next().

At a guess, I'd expect a new method was chosen to provide semantic
distinctness with the original behaviour.

But the semantics would not have changed, they would only be "extended"
and thus remain "compatible", wouldn't they?

Greetings,
Thomas

 

[Thomas Mlynarczyk]

And, if you don't intend to use 'myway' on 'listiterator's and such,
'send( None )' is equivalent to 'next( )'.

I didn't know that. But doesn't that impose a restriction somehow? It
makes it impossible to send a None to a generator.

Greetings,
Thomas

 

[Aaron Brady]

I didn't know that. But doesn't that impose a restriction somehow? It
makes it impossible to send a None to a generator.

No, None is a valid value to use with send. Not all iterators support
send.