Beazley on Generators

[Terry Reedy]

At PyCon2008, David Beazley presented an excellent talk on generators.
Generator Tricks for Systems Programmers
http://www.dabeaz.com/generators/index.html

At PyCon2009, he followed up with another talk on more advanced
generator usage, which Guido commended on the python-ideas list:
A Curious Course on Coroutines and Concurrency
http://dabeaz.com/coroutines/

I have just started (this one will take more than one sitting ;-) but it
looks just as good.

tjr

 

[Michele Simionato]

At PyCon2008, David Beazley presented an excellent talk on generators.
Generator Tricks for Systems Programmershttp://www.dabeaz.com/generators/index.html

At PyCon2009, he followed up with another talk on more advanced
generator usage, which Guido commended on the python-ideas list:
A Curious Course on Coroutines and Concurrencyhttp://dabeaz.com/coroutines/

I have just started (this one will take more than one sitting ;-) but it
looks just as good.

tjr

I concur. I have just read the first pages, but it says
all good things I had to discover the hard way in years
past. Excellent reading for everybody wanting to understand
cooperative concurrency!

M. Simionato

 

[Lawrence D'Oliveiro]

In message

Excellent reading for everybody wanting to understand cooperative
concurrency!

Hey, some of us were doing "cooperative concurrency" programming old MacOS
for years. It was generally considered a poor alternative to "true
multitasking".

 

[Michele Simionato]

In message


Hey, some of us were doing "cooperative concurrency" programming old MacOS
for years. It was generally considered a poor alternative to "true
multitasking".

It is a poor alternative compared to OS level processes and even
compared to preemptive threads in some cases. However, there are
situations
when you need thousands of lightweight threads of execution, and
in that case coroutines could be the only viable solution.
Just read David Beazley's presentation.

 

[Kay Schluehr]

At PyCon2008, David Beazley presented an excellent talk on generators.
Generator Tricks for Systems Programmershttp://www.dabeaz.com/generators/index.html

At PyCon2009, he followed up with another talk on more advanced
generator usage, which Guido commended on the python-ideas list:
A Curious Course on Coroutines and Concurrencyhttp://dabeaz.com/coroutines/

I have just started (this one will take more than one sitting ;-) but it
looks just as good.

tjr

There is just one thing I find disappointing. Since the talk is almost
a compendium of advanced uses of generators I'm missing a reference to
Peter Thatchers implementation of monads:

http://www.valuedlessons.com/2008/01/monads-in-python-with-nice-syntax.html

Peters implementation can be simplified but it already contains all
relevant ideas.

 

[Peter Pearson]

At PyCon2008, David Beazley presented an excellent talk on generators.
Generator Tricks for Systems Programmers
http://www.dabeaz.com/generators/index.html

At PyCon2009, he followed up with another talk on more advanced
generator usage, which Guido commended on the python-ideas list:
A Curious Course on Coroutines and Concurrency
http://dabeaz.com/coroutines/

Great presentations. Thanks.

 

[Craig Allen]

this is great, thanks... we have used generators to create something
akin to a cooperative tasking environment... not to implement
multitasking, but to be able to control low level data processing
scripts. These scripts, written as generators, yield control to a
control loop which then can pause, resume, abort, or change the state
of shared context objects which the script uses as it's input and
output space. E.g. the control loop can see there is intermediate
output which an operator (managing a data reduction pipeline) might
want to see.

I can see from the first few slide I need to understand this. It
already seems clear that there are ways to improve our approach to
what we have done, though the overall approach is solid and works
well.

anyway thanks.

 

[Carl Banks]

At PyCon2008, David Beazley presented an excellent talk on generators.
Generator Tricks for Systems Programmershttp://www.dabeaz.com/generators/index.html

At PyCon2009, he followed up with another talk on more advanced
generator usage, which Guido commended on the python-ideas list:
A Curious Course on Coroutines and Concurrencyhttp://dabeaz.com/coroutines/

I have just started (this one will take more than one sitting ;-) but it
looks just as good.

Yet another great thing about Python. The ability to run coroutines
in Matlab would make my working life a lot easier right now.


Carl Banks

 

[Lawrence D'Oliveiro]

In message <13298fc5-5024-4343-

However, there are situations when you need thousands of lightweight
threads of execution ;;;

The Linux kernel has been tested running hundreds of thousands of threads.

 

[Steven D'Aprano]

In message <13298fc5-5024-4343-
(e-mail address removed)>, Michele Simionato
wrote:


The Linux kernel has been tested running hundreds of thousands of
threads.

Did it pass or fail that test?

 

[Paul Rubin]

The Linux kernel has been tested running hundreds of thousands of threads.

Those are still heavyweight threads requiring context switches to
switch from one to another. If you look at the multi-threading
benchmarks in the Alioth shootout, languages with lightweight threads
(such as Erlang or GHC) can switch orders of magnitude faster than
those that use kernel threads.