Beazley on Generators

Discussion in 'Python' started by Terry Reedy, Apr 1, 2009.

  1. Terry Reedy

    Terry Reedy Guest

    At PyCon2008, David Beazley presented an excellent talk on generators.
    Generator Tricks for Systems Programmers
    http://www.dabeaz.com/generators/index.html

    At PyCon2009, he followed up with another talk on more advanced
    generator usage, which Guido commended on the python-ideas list:
    A Curious Course on Coroutines and Concurrency
    http://dabeaz.com/coroutines/

    I have just started (this one will take more than one sitting ;-) but it
    looks just as good.

    tjr
     
    Terry Reedy, Apr 1, 2009
    #1
    1. Advertising

  2. On Apr 1, 7:03 am, Terry Reedy <> wrote:
    > At PyCon2008, David Beazley presented an excellent talk on generators.
    > Generator Tricks for Systems Programmershttp://www.dabeaz.com/generators/index.html
    >
    > At PyCon2009, he followed up with another talk on more advanced
    > generator usage, which Guido commended on the python-ideas list:
    > A Curious Course on Coroutines and Concurrencyhttp://dabeaz.com/coroutines/
    >
    > I have just started (this one will take more than one sitting ;-) but it
    > looks just as good.
    >
    > tjr


    I concur. I have just read the first pages, but it says
    all good things I had to discover the hard way in years
    past. Excellent reading for everybody wanting to understand
    cooperative concurrency!

    M. Simionato
     
    Michele Simionato, Apr 1, 2009
    #2
    1. Advertising

  3. In message
    <>, Michele
    Simionato wrote:

    > Excellent reading for everybody wanting to understand cooperative
    > concurrency!


    Hey, some of us were doing "cooperative concurrency" programming old MacOS
    for years. It was generally considered a poor alternative to "true
    multitasking".
     
    Lawrence D'Oliveiro, Apr 1, 2009
    #3
  4. On Apr 1, 7:57 am, Lawrence D'Oliveiro <l...@geek-
    central.gen.new_zealand> wrote:
    > In message
    > <>, Michele
    >
    > Simionato wrote:
    > > Excellent reading for everybody wanting to understand cooperative
    > > concurrency!

    >
    > Hey, some of us were doing "cooperative concurrency" programming old MacOS
    > for years. It was generally considered a poor alternative to "true
    > multitasking".


    It is a poor alternative compared to OS level processes and even
    compared to preemptive threads in some cases. However, there are
    situations
    when you need thousands of lightweight threads of execution, and
    in that case coroutines could be the only viable solution.
    Just read David Beazley's presentation.
     
    Michele Simionato, Apr 1, 2009
    #4
  5. Terry Reedy

    Kay Schluehr Guest

    On 1 Apr., 07:03, Terry Reedy <> wrote:
    > At PyCon2008, David Beazley presented an excellent talk on generators.
    > Generator Tricks for Systems Programmershttp://www.dabeaz.com/generators/index.html
    >
    > At PyCon2009, he followed up with another talk on more advanced
    > generator usage, which Guido commended on the python-ideas list:
    > A Curious Course on Coroutines and Concurrencyhttp://dabeaz.com/coroutines/
    >
    > I have just started (this one will take more than one sitting ;-) but it
    > looks just as good.
    >
    > tjr


    There is just one thing I find disappointing. Since the talk is almost
    a compendium of advanced uses of generators I'm missing a reference to
    Peter Thatchers implementation of monads:

    http://www.valuedlessons.com/2008/01/monads-in-python-with-nice-syntax.html

    Peters implementation can be simplified but it already contains all
    relevant ideas.
     
    Kay Schluehr, Apr 1, 2009
    #5
  6. On Wed, 01 Apr 2009 01:03:50 -0400, Terry Reedy <> wrote:
    > At PyCon2008, David Beazley presented an excellent talk on generators.
    > Generator Tricks for Systems Programmers
    > http://www.dabeaz.com/generators/index.html
    >
    > At PyCon2009, he followed up with another talk on more advanced
    > generator usage, which Guido commended on the python-ideas list:
    > A Curious Course on Coroutines and Concurrency
    > http://dabeaz.com/coroutines/


    Great presentations. Thanks.

    --
    To email me, substitute nowhere->spamcop, invalid->net.
     
    Peter Pearson, Apr 1, 2009
    #6
  7. Terry Reedy

    Craig Allen Guest

    this is great, thanks... we have used generators to create something
    akin to a cooperative tasking environment... not to implement
    multitasking, but to be able to control low level data processing
    scripts. These scripts, written as generators, yield control to a
    control loop which then can pause, resume, abort, or change the state
    of shared context objects which the script uses as it's input and
    output space. E.g. the control loop can see there is intermediate
    output which an operator (managing a data reduction pipeline) might
    want to see.

    I can see from the first few slide I need to understand this. It
    already seems clear that there are ways to improve our approach to
    what we have done, though the overall approach is solid and works
    well.

    anyway thanks.
     
    Craig Allen, Apr 1, 2009
    #7
  8. Terry Reedy

    Carl Banks Guest

    On Mar 31, 10:03 pm, Terry Reedy <> wrote:
    > At PyCon2008, David Beazley presented an excellent talk on generators.
    > Generator Tricks for Systems Programmershttp://www.dabeaz.com/generators/index.html
    >
    > At PyCon2009, he followed up with another talk on more advanced
    > generator usage, which Guido commended on the python-ideas list:
    > A Curious Course on Coroutines and Concurrencyhttp://dabeaz.com/coroutines/
    >
    > I have just started (this one will take more than one sitting ;-) but it
    > looks just as good.


    Yet another great thing about Python. The ability to run coroutines
    in Matlab would make my working life a lot easier right now.


    Carl Banks
     
    Carl Banks, Apr 1, 2009
    #8
  9. In message <13298fc5-5024-4343-
    >, Michele Simionato wrote:

    > However, there are situations when you need thousands of lightweight
    > threads of execution ;;;


    The Linux kernel has been tested running hundreds of thousands of threads.
     
    Lawrence D'Oliveiro, Apr 1, 2009
    #9
  10. On Thu, 02 Apr 2009 11:37:46 +1300, Lawrence D'Oliveiro wrote:

    > In message <13298fc5-5024-4343-
    > >, Michele Simionato
    > wrote:
    >
    >> However, there are situations when you need thousands of lightweight
    >> threads of execution ;;;

    >
    > The Linux kernel has been tested running hundreds of thousands of
    > threads.


    Did it pass or fail that test?



    --
    Steven
     
    Steven D'Aprano, Apr 2, 2009
    #10
  11. Terry Reedy

    Paul Rubin Guest

    Lawrence D'Oliveiro <_zealand> writes:
    > > However, there are situations when you need thousands of lightweight
    > > threads of execution ;;;

    >
    > The Linux kernel has been tested running hundreds of thousands of threads.


    Those are still heavyweight threads requiring context switches to
    switch from one to another. If you look at the multi-threading
    benchmarks in the Alioth shootout, languages with lightweight threads
    (such as Erlang or GHC) can switch orders of magnitude faster than
    those that use kernel threads.
     
    Paul Rubin, Apr 2, 2009
    #11
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Rhino
    Replies:
    4
    Views:
    5,750
    Roedy Green
    Jan 13, 2006
  2. Pavel

    Code Generators

    Pavel, May 14, 2006, in forum: Java
    Replies:
    7
    Views:
    723
    dimitar
    May 19, 2006
  3. Terry Reedy

    Re: Beazley 4E P.E.R, Page29: Unicode

    Terry Reedy, Jul 14, 2013, in forum: Python
    Replies:
    0
    Views:
    101
    Terry Reedy
    Jul 14, 2013
  4. Joshua Landau

    Re: Beazley 4E P.E.R, Page29: Unicode

    Joshua Landau, Jul 14, 2013, in forum: Python
    Replies:
    1
    Views:
    87
  5. Steven D'Aprano

    Re: Beazley 4E P.E.R, Page29: Unicode

    Steven D'Aprano, Jul 14, 2013, in forum: Python
    Replies:
    1
    Views:
    98
Loading...

Share This Page