Request for elucidation: enhanced generators

[Ben Sizer]

A simple question - can anybody give a short example of how these work
and what they are good for? I've read PEP 342 and the associated bit in
the What's New section and it's still all Greek to me. The latter seems
to focus on how to do it, rather than why you'd do it, so it doesn't
aid the understanding too much.

 

[Steve Holden]

A simple question - can anybody give a short example of how these work
and what they are good for? I've read PEP 342 and the associated bit in
the What's New section and it's still all Greek to me. The latter seems
to focus on how to do it, rather than why you'd do it, so it doesn't
aid the understanding too much.

Unti 2.5 the yield keyword could only be used to produce a value from a
generator - it introduced a statement.

Now the yield keyword can be used as an expression inside a generator,
allowing you to send values into the generator by calling its .send()
method.

If you have no use case for this you are, as always, perfectly free to
ignore it, as the majority of Python users may well choose to do. Your
existing generators should continue to work.

regards
Steve

 

[Ben Sizer]

Unti 2.5 the yield keyword could only be used to produce a value from a
generator - it introduced a statement.

Now the yield keyword can be used as an expression inside a generator,
allowing you to send values into the generator by calling its .send()
method.

If you have no use case for this you are, as always, perfectly free to
ignore it, as the majority of Python users may well choose to do. Your
existing generators should continue to work.

But do you have an example of such a use case? That's what I'm missing
here. Without ever having used proper coroutines elsewhere, I have no
real way to appreciate their benefits without a good example.

I don't think it's feasible to just ignore any new feature, as sooner
or later you're going to encounter someone else's code that relies upon
it. Hence why I'd rather not see all sorts of 'optional' extras added
later (like type declarations and so on).

 

[Fredrik Lundh]

But do you have an example of such a use case?

here's what I think is the first published example that uses "yield" for
co-routine-like behaviour:

http://effbot.org/zone/asyncore-generators.htm

that snippet uses a generator to implement a simple network protocol.
the generator yields a status code when it needs more data from the
framework, and the framework uses an instance variable (self.data) to
pass information into the generator.

Python 2.5 lets you eliminate the instance variable; instead of using an
external shared variable, the framework can use "send" to pass data into
the generator, and the generator can use ordinary expression syntax to
pick it up.

</F>

 

[Duncan Booth]

But do you have an example of such a use case? That's what I'm missing
here. Without ever having used proper coroutines elsewhere, I have no
real way to appreciate their benefits without a good example.

One example which is commonly given is to implement a web application using
continuations. So in Python terms a session is handled by a generator which
yields a form to be sent back to the user and the user's response is sent
in to the request using the generator's send method. State would be
maintained in the generator. Whether this actually works well in practice
(and how you handle things like a back button on the forms) is another
matter.

So very roughly your web code might look like this:

def checkout(basket):
orderOk = False

while not OrderOk:
order = (yield OrderForm(basket))
address = (yield AddressForm())
payment = (yield PaymentForm(order.total))
confirmation = (yield ConfirmationForm(order, address, payment))
orderOk = confirmation.confirmed

placeOrder(order, address, payment)
yield OrderPlacedPage()

 

[Ant]

Ben Sizer wrote:
....

But do you have an example of such a use case? That's what I'm missing
here.

I've been looking at this myself, trying to understand the point of
coroutines. I believe that they boil down to generators which are able
to take in data as well as provide it. A simple way of looking at it
seems to be that a coroutine is a generator which can have its state
changed whilst it is still active. A silly example to illustrate:

# coroutine.py
def stateful_generator(people):
greeting = "Hello"
for person in people:
received = (yield greeting + " " + person)
if received:
greeting = received

if __name__ == "__main__":
people = ["bob", "henry", "jim-lad", "boney", "greebo", "badger"]
gen = stateful_generator(people)

print gen.next() # Hello bob
print gen.next() # Hello henry
print gen.send("Yo! ") # Yo! jim-lad
print gen.next() # Yo! boney
print gen.send("Meow ") # Meow greebo
print gen.next() # Meow badger

So you can change the behaviour of the coroutine whilst it is running.
That's as far as I have got with them though - there are presumably
many other uses of coroutines out there as I guess this use case could
be simulated using generators and globals.

 

[Michele Simionato]

One example which is commonly given is to implement a web application using
continuations. So in Python terms a session is handled by a generator which
yields a form to be sent back to the user and the user's response is sent
in to the request using the generator's send method. State would be
maintained in the generator. Whether this actually works well in practice
(and how you handle things like a back button on the forms) is another
matter.

So very roughly your web code might look like this:

def checkout(basket):
orderOk = False

while not OrderOk:
order = (yield OrderForm(basket))
address = (yield AddressForm())
payment = (yield PaymentForm(order.total))
confirmation = (yield ConfirmationForm(order, address, payment))
orderOk = confirmation.confirmed

placeOrder(order, address, payment)
yield OrderPlacedPage()

I too wonder about the back button. I am familiar with this reference
about modal
web servers, that you probably know:
http://www.double.co.nz/scheme/modal-web-server.html
In that case, the back button can be made to work since in Scheme there
are full
continuations and it is possibile to store the state at a given point
in time and go
back to that point later on (i.e. when the user press the back button).

Python generators however are not full continuations and you cannot go
back
to a previous time. I remember I did some experiment and it was
possible to
solve the problem by generating a lot of closures and associating each
closure to
one state in the time and one page in the UI. But is was complicated
enough
and finally I ended up simply storing the values entered by the user in
HTML hidden widgets.
Sometimes the simplest solutions are the better ones

Michele Simionato

 

[Michele Simionato]

But do you have an example of such a use case?

Here is a 69 lines implementation of the idea of applying extended
generators to manage Web forms (obviously this is only a proof of
concept and it contains many mistakes, but you have something to get
started).
Notice that I am not claiming that this is a good idea.

Michele Simionato

---

import datetime
import cherrypy as cp

# each user (but really should be each session) has her input loop
# one should disable the back button and implement an undo mechanism
def inputloop(user):
start_time = datetime.datetime.today()
cart = []
while True: # fill the shopping cart
item = yield locals()
if item == 'start':
continue
elif item == 'end':
break
elif item:
cart.append(item)
finish_time = datetime.datetime.today()
yield locals()

class User(object):
def __init__(self, name):
self.name = name
self.inputloop = inputloop(self)
self.inputloop.send(None) # start the user loop

users = dict(michele=User('michele'))

class Root(object):
@cp.expose
def login_welcome(self):
yield 'hello!'
yield '<form action="login_action">'
yield "what's your name? "
yield '<input type="text" name="name"/>'
yield '</form>'

@cp.expose
def login_action(self, name):
yield ('<a href="shopping_loop?name=%s&item=start">'
'You may start shopping</a><br/>') % name

@cp.expose
def shopping_loop(self, name, item):
if not name in users:
yield '%r is not a valid name; please retry' % name
return
user = users[name]
status = user.inputloop.send(item)
if item == 'start':
yield 'Today is %s<br/>' % status['start_time']
yield 'Please buy something!<br/>'
if item == 'end':
yield 'Thanks for shopping, %s<br/>' % user.name
yield 'You bought items %s<br/>' % status['cart']
yield 'Today is %s' % status['finish_time']
return
yield '<form action="">'
yield 'The content of your cart is %s<br/>' % status['cart']
yield 'Please enter the item you want to buy: '
yield '<input type="text" name="item"/><br/>'
yield '<input type="hidden" name="name" value="%s"/>' % name
yield '(enter end to finish)'
yield '</form>'

index = login_welcome

if __name__ == '__main__':
cp.root = Root()
cp.server.start()