Most "active" coroutine library project?

Discussion in 'Python' started by Phillip B Oldham, Aug 23, 2009.

  1. I've been taking a look at the multitude of coroutine libraries
    available for Python, but from the looks of the projects they all seem
    to be rather "quiet". I'd like to pick one up to use on a current
    project but can't deduce which is the most popular/has the largest
    community.

    Libraries I looked at include: cogen, weightless, eventlet and
    circuits (which isn't exactly coroutine-based but it's event-driven
    model was intriguing).

    Firstly, are there any others I've missed? And what would the
    consensus be on the which has the most active community behind it?
    Phillip B Oldham, Aug 23, 2009
    #1
    1. Advertising

  2. Phillip B Oldham <> writes:

    > I've been taking a look at the multitude of coroutine libraries
    > available for Python, but from the looks of the projects they all seem
    > to be rather "quiet". I'd like to pick one up to use on a current
    > project but can't deduce which is the most popular/has the largest
    > community.
    >
    > Libraries I looked at include: cogen, weightless, eventlet and
    > circuits (which isn't exactly coroutine-based but it's event-driven
    > model was intriguing).
    >
    > Firstly, are there any others I've missed?


    There's greenlets

    http://pypi.python.org/pypi/greenlet

    which I think is also fairly described as "quiet".

    -M-
    Matthew Woodcraft, Aug 23, 2009
    #2
    1. Advertising

  3. Phillip B Oldham

    Denis Guest

    You can also at gevent

    http://pypi.python.org/pypi/gevent


    On Aug 23, 10:02 pm, Phillip B Oldham <>
    wrote:
    > I've been taking a look at the multitude of coroutine libraries
    > available for Python, but from the looks of the projects they all seem
    > to be rather "quiet". I'd like to pick one up to use on a current
    > project but can't deduce which is the most popular/has the largest
    > community.
    >
    > Libraries I looked at include: cogen, weightless, eventlet and
    > circuits (which isn't exactly coroutine-based but it's event-driven
    > model was intriguing).
    >
    > Firstly, are there any others I've missed? And what would the
    > consensus be on the which has the most active community behind it?
    Denis, Aug 25, 2009
    #3
  4. On Aug 25, 12:51 am, Denis <> wrote:
    > You can also at gevent
    >
    > http://pypi.python.org/pypi/gevent


    Please, please document this! There are a lot of people who would
    love to use this but give up when they don't find a guide or something
    similar.
    Brian Hammond, Sep 23, 2009
    #4
  5. Phillip B Oldham

    Simon Forman Guest

    On Sun, Aug 23, 2009 at 11:02 AM, Phillip B Oldham
    <> wrote:
    > I've been taking a look at the multitude of coroutine libraries
    > available for Python, but from the looks of the projects they all seem
    > to be rather "quiet". I'd like to pick one up to use on a current
    > project but can't deduce which is the most popular/has the largest
    > community.
    >
    > Libraries I looked at include: cogen, weightless, eventlet and
    > circuits (which isn't exactly coroutine-based but it's event-driven
    > model was intriguing).
    >
    > Firstly, are there any others I've missed? And what would the
    > consensus be on the which has the most active community behind it?
    > --
    > http://mail.python.org/mailman/listinfo/python-list
    >


    Coroutines are built into the language. There's a good talk about
    them here: http://www.dabeaz.com/coroutines/

    HTH,
    ~Simon
    Simon Forman, Sep 23, 2009
    #5
  6. Phillip B Oldham

    Guest

    On 05:00 pm, wrote:
    >On Sun, Aug 23, 2009 at 11:02 AM, Phillip B Oldham
    ><> wrote:
    >>I've been taking a look at the multitude of coroutine libraries
    >>available for Python, but from the looks of the projects they all seem
    >>to be rather "quiet". I'd like to pick one up to use on a current
    >>project but can't deduce which is the most popular/has the largest
    >>community.
    >>
    >>Libraries I looked at include: cogen, weightless, eventlet and
    >>circuits (which isn't exactly coroutine-based but it's event-driven
    >>model was intriguing).
    >>
    >>Firstly, are there any others I've missed? And what would the
    >>consensus be on the which has the most active community behind it?
    >>--
    >>http://mail.python.org/mailman/listinfo/python-list

    >
    >Coroutines are built into the language. There's a good talk about
    >them here: http://www.dabeaz.com/coroutines/


    But what some Python programmers call coroutines aren't really the same
    as what the programming community at large would call a coroutine.

    Jean-Paul
    , Sep 23, 2009
    #6
  7. Phillip B Oldham

    Simon Forman Guest

    On Wed, Sep 23, 2009 at 2:05 PM, <> wrote:
    > On 05:00 pm, wrote:
    >>
    >> On Sun, Aug 23, 2009 at 11:02 AM, Phillip B Oldham
    >> <> wrote:
    >>>
    >>> I've been taking a look at the multitude of coroutine libraries
    >>> available for Python, but from the looks of the projects they all seem
    >>> to be rather "quiet". I'd like to pick one up to use on a current
    >>> project but can't deduce which is the most popular/has the largest
    >>> community.
    >>>
    >>> Libraries I looked at include: cogen, weightless, eventlet and
    >>> circuits (which isn't exactly coroutine-based but it's event-driven
    >>> model was intriguing).
    >>>
    >>> Firstly, are there any others I've missed? And what would the
    >>> consensus be on the which has the most active community behind it?
    >>> --
    >>> http://mail.python.org/mailman/listinfo/python-list

    >>
    >> Coroutines are built into the language.  There's a good talk about
    >> them here: http://www.dabeaz.com/coroutines/

    >
    > But what some Python programmers call coroutines aren't really the same as
    > what the programming community at large would call a coroutine.
    >
    > Jean-Paul


    Really? I'm curious as to the differences. (I just skimmed the entry
    for coroutines in Wikipedia and PEP 342, but I'm not fully
    enlightened.)

    Warm regards,
    ~Simon
    Simon Forman, Sep 23, 2009
    #7
  8. On 2009-09-23, Simon Forman <> wrote:

    >>> Coroutines are built into the language. ?There's a good talk
    >>> about them here: http://www.dabeaz.com/coroutines/

    >>
    >> But what some Python programmers call coroutines aren't really
    >> the same as what the programming community at large would call
    >> a coroutine.

    >
    > Really? I'm curious as to the differences.


    Me too. I read through the presentation above, it it seems to
    describe pretty much exactly what we called co-routines both in
    school and in the workplace.

    Back when I worked on one of the first hand-held cellular
    mobile phones, it used co-routines where the number of
    coroutines was fixed at 2 (one for each register set in a Z80
    CPU). The semantics seem to be identical to the coroutines
    described in the presentation.

    > (I just skimmed the entry for coroutines in Wikipedia and PEP
    > 342, but I'm not fully enlightened.)


    --
    Grant Edwards grante Yow! I was born in a
    at Hostess Cupcake factory
    visi.com before the sexual
    revolution!
    Grant Edwards, Sep 23, 2009
    #8
  9. Phillip B Oldham

    Guest

    On 08:16 pm, wrote:
    >On Wed, Sep 23, 2009 at 2:05 PM, <> wrote:
    >[snip]
    >>
    >>But what some Python programmers call coroutines aren't really the
    >>same as
    >>what the programming community at large would call a coroutine.
    >>
    >>Jean-Paul

    >
    >Really? I'm curious as to the differences. (I just skimmed the entry
    >for coroutines in Wikipedia and PEP 342, but I'm not fully
    >enlightened.)


    The important difference is that coroutines can switch across multiple
    stack frames. Python's "enhanced generators" can still only switch
    across one stack frame - ie, from inside the generator to the frame
    immediately outside the generator. This means that you cannot use
    "enhanced generators" to implement an API like this one:

    def doSomeNetworkStuff():
    s = corolib.socket()
    s.connect(('google.com', 80))
    s.sendall('GET / HTTP/1.1\r\nHost: www.google.com\r\n\r\n')
    response = s.recv(8192)

    where connect, sendall, and recv don't actually block the entire calling
    thread, they only switch away to another coroutine until the underlying
    operation completes. With "real" coroutines, you can do this.

    Jean-Paul
    , Sep 23, 2009
    #9
  10. On 2009-09-23, <> wrote:
    > On 08:16 pm, wrote:
    >>On Wed, Sep 23, 2009 at 2:05 PM, <> wrote:
    >>[snip]
    >>>
    >>>But what some Python programmers call coroutines aren't really the
    >>>same as
    >>>what the programming community at large would call a coroutine.
    >>>
    >>>Jean-Paul

    >>
    >>Really? I'm curious as to the differences. (I just skimmed
    >>the entry for coroutines in Wikipedia and PEP 342, but I'm not
    >>fully enlightened.)

    >
    > The important difference is that coroutines can switch across
    > multiple stack frames. Python's "enhanced generators" can
    > still only switch across one stack frame - ie, from inside the
    > generator to the frame immediately outside the generator.


    Good point. Being unable to "yeild" from inside a function
    called by the "main" coroutine can be limiting once you try to
    do something non-trivial. I had read about that limitation,
    but had forgotten it.

    Some "stackless" threading schemes for C (e.g. Protothreads
    http://www.sics.se/~adam/pt/) have the same issue, and it does
    require some extra effort to work around that limitation.

    --
    Grant Edwards grante Yow! I have accepted
    at Provolone into my life!
    visi.com
    Grant Edwards, Sep 23, 2009
    #10
  11. On Wed, 2009-09-23 at 20:50 +0000, wrote:
    > immediately outside the generator. This means that you cannot use
    > "enhanced generators" to implement an API like this one:
    >
    > def doSomeNetworkStuff():
    > s = corolib.socket()
    > s.connect(('google.com', 80))
    > s.sendall('GET / HTTP/1.1\r\nHost: www.google.com\r\n\r\n')
    > response = s.recv(8192)
    >
    > where connect, sendall, and recv don't actually block the entire calling
    > thread, they only switch away to another coroutine until the underlying
    > operation completes. With "real" coroutines, you can do this.


    I might be missing some subtlety of your point, but I've implemented
    this functionality using generators in a library called Kaa[1]. In kaa,
    your example looks like:

    import kaa

    @kaa.coroutine()
    def do_some_network_stuff():
    s = kaa.Socket()
    yield s.connect('google.com:80')
    yield s.write('GET / HTTP/1.1\nHost: www.google.com\n\n')
    response = yield s.read()

    do_some_network_stuff()
    kaa.main.run()


    Of course, it does require a "coroutine scheduler" be implemented,
    which, in kaa, is taken care of by the main loop.

    Cheers,
    Jason.

    [1] The curious can visit http://doc.freevo.org/api/kaa/base/ and
    http://freevo.org/kaa/ although a 1.0 hasn't yet been released and the
    docs are still rather sketchy.
    Jason Tackaberry, Sep 23, 2009
    #11
  12. Phillip B Oldham

    Guest

    On 09:40 pm, wrote:
    >On Wed, 2009-09-23 at 20:50 +0000, wrote:
    >>immediately outside the generator. This means that you cannot use
    >>"enhanced generators" to implement an API like this one:
    >>
    >> def doSomeNetworkStuff():
    >> s = corolib.socket()
    >> s.connect(('google.com', 80))
    >> s.sendall('GET / HTTP/1.1\r\nHost: www.google.com\r\n\r\n')
    >> response = s.recv(8192)
    >>
    >>where connect, sendall, and recv don't actually block the entire
    >>calling
    >>thread, they only switch away to another coroutine until the
    >>underlying
    >>operation completes. With "real" coroutines, you can do this.

    >
    >I might be missing some subtlety of your point, but I've implemented
    >this functionality using generators in a library called Kaa[1]. In
    >kaa,
    >your example looks like:
    >
    > import kaa
    >
    > @kaa.coroutine()
    > def do_some_network_stuff():
    > s = kaa.Socket()
    > yield s.connect('google.com:80')
    > yield s.write('GET / HTTP/1.1\nHost: www.google.com\n\n')
    > response = yield s.read()
    >
    > do_some_network_stuff()
    > kaa.main.run()


    I specifically left out all "yield" statements in my version, since
    that's exactly the point here. :) With "real" coroutines, they're not
    necessary - coroutine calls look just like any other call. With
    Python's enhanced generators, they are.

    Jean-Paul
    , Sep 23, 2009
    #12
  13. On Wed, 2009-09-23 at 21:53 +0000, wrote:
    > I specifically left out all "yield" statements in my version, since
    > that's exactly the point here. :) With "real" coroutines, they're not
    > necessary - coroutine calls look just like any other call. With
    > Python's enhanced generators, they are.


    Yes, I take your point.

    Explicitly yielding may not be such a terrible thing, though. It's more
    clear when reading such code where it may return back to the scheduler.
    This may be important for performance, unless of course the coroutine
    implementation supports preemption.

    Cheers,
    Jason.
    Jason Tackaberry, Sep 23, 2009
    #13
  14. Phillip B Oldham

    Guest

    On 10:00 pm, wrote:
    >On Wed, 2009-09-23 at 21:53 +0000, wrote:
    >>I specifically left out all "yield" statements in my version, since
    >>that's exactly the point here. :) With "real" coroutines, they're not
    >>necessary - coroutine calls look just like any other call. With
    >>Python's enhanced generators, they are.

    >
    >Yes, I take your point.
    >
    >Explicitly yielding may not be such a terrible thing, though. It's
    >more
    >clear when reading such code where it may return back to the scheduler.
    >This may be important for performance, unless of course the coroutine
    >implementation supports preemption.


    Sure, no value judgement intended, except on the practice of taking
    words with well established meanings and re-using them for something
    else ;)

    Jean-Paul
    , Sep 23, 2009
    #14
  15. On Wed, 2009-09-23 at 22:07 +0000, wrote:
    > Sure, no value judgement intended, except on the practice of taking
    > words with well established meanings and re-using them for something
    > else ;)


    I think it's the behaviour that's important, and not the specific syntax
    needed to implement that behaviour.

    In other words, I disagree (if this is what you're suggesting) that
    sticking "yield" in front of certain expressions makes it any less a
    coroutine.

    Now, requiring explicit yields does mean that the coroutine has
    specific, well-defined points of reentry. But I don't believe it's a
    necessary condition that coroutines allow arbitrary (in the
    non-deterministic sense) reentry points, only multiple.

    Cheers,
    Jason.
    Jason Tackaberry, Sep 23, 2009
    #15
  16. Grant Edwards <invalid <at> invalid.invalid> writes:
    >
    > Back when I worked on one of the first hand-held cellular
    > mobile phones, it used co-routines where the number of
    > coroutines was fixed at 2 (one for each register set in a Z80
    > CPU).


    Gotta love the lightning-fast EXX instruction. :)

    Regards

    Antoine.
    Antoine Pitrou, Sep 24, 2009
    #16
  17. On Aug 23, 5:02 pm, Phillip B Oldham <> wrote:
    > I've been taking a look at the multitude of coroutine libraries
    > available for Python, but from the looks of the projects they all seem
    > to be rather "quiet". I'd like to pick one up to use on a current
    > project but can't deduce which is the most popular/has the largest
    > community.
    >
    > Libraries I looked at include: cogen, weightless, eventlet and
    > circuits (which isn't exactly coroutine-based but it's event-driven
    > model was intriguing).
    >
    > Firstly, are there any others I've missed? And what would the
    > consensus be on the which has the most active community behind it?


    The newest one seems to be diesel, by Simon Willison & Co: http://dieselweb..org/lib
    Michele Simionato, Sep 24, 2009
    #17
  18. On Thursday, 24 September 2009 15:42:36 Antoine Pitrou wrote:
    > Grant Edwards <invalid <at> invalid.invalid> writes:
    > > Back when I worked on one of the first hand-held cellular
    > > mobile phones, it used co-routines where the number of
    > > coroutines was fixed at 2 (one for each register set in a Z80
    > > CPU).

    >
    > Gotta love the lightning-fast EXX instruction. :)


    Using it in the above context is about equivalent to slipping a hand grenade
    in amongst the other eggs in a nest.

    :)

    - Hendrik
    Hendrik van Rooyen, Sep 25, 2009
    #18
  19. On 2009-09-25, Hendrik van Rooyen <> wrote:
    > On Thursday, 24 September 2009 15:42:36 Antoine Pitrou wrote:
    >> Grant Edwards <invalid <at> invalid.invalid> writes:
    >> > Back when I worked on one of the first hand-held cellular
    >> > mobile phones, it used co-routines where the number of
    >> > coroutines was fixed at 2 (one for each register set in a Z80
    >> > CPU).

    >>
    >> Gotta love the lightning-fast EXX instruction. :)

    >
    > Using it in the above context is about equivalent to slipping
    > a hand grenade in amongst the other eggs in a nest.


    EXX accomplised much of the context switch operation. I don't
    remember how much RAM was available, but it wasn't much...

    --
    Grant Edwards grante Yow! And furthermore,
    at my bowling average is
    visi.com unimpeachable!!!
    Grant Edwards, Sep 25, 2009
    #19
  20. >>>>> (e) wrote:

    >e> I specifically left out all "yield" statements in my version, since that's
    >e> exactly the point here. :) With "real" coroutines, they're not necessary -
    >e> coroutine calls look just like any other call. With Python's enhanced
    >e> generators, they are.


    The first time I encountered coroutines was in Simula-67. Coroutine
    switching was certainly explicit there. IIRC, the keyword was resume.
    --
    Piet van Oostrum <>
    URL: http://pietvanoostrum.com [PGP 8DAE142BE17999C4]
    Private email:
    Piet van Oostrum, Sep 25, 2009
    #20
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. H.MuthuKumaraRajan
    Replies:
    3
    Views:
    431
    H.MuthuKumaraRajan
    Feb 4, 2004
  2. Carlos Ribeiro
    Replies:
    3
    Views:
    343
    Carlos Ribeiro
    Sep 26, 2004
  3. Matthew Wilson

    Need help writing coroutine

    Matthew Wilson, Nov 7, 2007, in forum: Python
    Replies:
    1
    Views:
    244
    Paul Hankin
    Nov 7, 2007
  4. xkenneth
    Replies:
    8
    Views:
    330
    Bruno Desthuilliers
    Feb 6, 2008
  5. carlos seramos
    Replies:
    2
    Views:
    464
    carlos seramos
    Aug 1, 2003
Loading...

Share This Page