Most "active" coroutine library project?

[Phillip B Oldham]

I've been taking a look at the multitude of coroutine libraries
available for Python, but from the looks of the projects they all seem
to be rather "quiet". I'd like to pick one up to use on a current
project but can't deduce which is the most popular/has the largest
community.

Libraries I looked at include: cogen, weightless, eventlet and
circuits (which isn't exactly coroutine-based but it's event-driven
model was intriguing).

Firstly, are there any others I've missed? And what would the
consensus be on the which has the most active community behind it?

 

[Matthew Woodcraft]

I've been taking a look at the multitude of coroutine libraries
available for Python, but from the looks of the projects they all seem
to be rather "quiet". I'd like to pick one up to use on a current
project but can't deduce which is the most popular/has the largest
community.

Libraries I looked at include: cogen, weightless, eventlet and
circuits (which isn't exactly coroutine-based but it's event-driven
model was intriguing).

Firstly, are there any others I've missed?

There's greenlets

http://pypi.python.org/pypi/greenlet

which I think is also fairly described as "quiet".

-M-

 

[Denis]

You can also at gevent

http://pypi.python.org/pypi/gevent

 

[Brian Hammond]

You can also at gevent

http://pypi.python.org/pypi/gevent

Please, please document this! There are a lot of people who would
love to use this but give up when they don't find a guide or something
similar.

 

[Simon Forman]

I've been taking a look at the multitude of coroutine libraries
available for Python, but from the looks of the projects they all seem
to be rather "quiet". I'd like to pick one up to use on a current
project but can't deduce which is the most popular/has the largest
community.

Libraries I looked at include: cogen, weightless, eventlet and
circuits (which isn't exactly coroutine-based but it's event-driven
model was intriguing).

Firstly, are there any others I've missed? And what would the
consensus be on the which has the most active community behind it?

Coroutines are built into the language. There's a good talk about
them here: http://www.dabeaz.com/coroutines/

HTH,
~Simon

 

[exarkun]

Coroutines are built into the language. There's a good talk about
them here: http://www.dabeaz.com/coroutines/

But what some Python programmers call coroutines aren't really the same
as what the programming community at large would call a coroutine.

Jean-Paul

 

[Simon Forman]

But what some Python programmers call coroutines aren't really the same as
what the programming community at large would call a coroutine.

Jean-Paul

Really? I'm curious as to the differences. (I just skimmed the entry
for coroutines in Wikipedia and PEP 342, but I'm not fully
enlightened.)

Warm regards,
~Simon

 

[Grant Edwards]

Really? I'm curious as to the differences.

Me too. I read through the presentation above, it it seems to
describe pretty much exactly what we called co-routines both in
school and in the workplace.

Back when I worked on one of the first hand-held cellular
mobile phones, it used co-routines where the number of
coroutines was fixed at 2 (one for each register set in a Z80
CPU). The semantics seem to be identical to the coroutines
described in the presentation.

 

[exarkun]

Really? I'm curious as to the differences. (I just skimmed the entry
for coroutines in Wikipedia and PEP 342, but I'm not fully
enlightened.)

The important difference is that coroutines can switch across multiple
stack frames. Python's "enhanced generators" can still only switch
across one stack frame - ie, from inside the generator to the frame
immediately outside the generator. This means that you cannot use
"enhanced generators" to implement an API like this one:

def doSomeNetworkStuff():
s = corolib.socket()
s.connect(('google.com', 80))
s.sendall('GET / HTTP/1.1\r\nHost: www.google.com\r\n\r\n')
response = s.recv(8192)

where connect, sendall, and recv don't actually block the entire calling
thread, they only switch away to another coroutine until the underlying
operation completes. With "real" coroutines, you can do this.

Jean-Paul

 

[Grant Edwards]

The important difference is that coroutines can switch across
multiple stack frames. Python's "enhanced generators" can
still only switch across one stack frame - ie, from inside the
generator to the frame immediately outside the generator.

Good point. Being unable to "yeild" from inside a function
called by the "main" coroutine can be limiting once you try to
do something non-trivial. I had read about that limitation,
but had forgotten it.

Some "stackless" threading schemes for C (e.g. Protothreads
http://www.sics.se/~adam/pt/) have the same issue, and it does
require some extra effort to work around that limitation.

 

[Jason Tackaberry]

immediately outside the generator. This means that you cannot use
"enhanced generators" to implement an API like this one:

def doSomeNetworkStuff():
s = corolib.socket()
s.connect(('google.com', 80))
s.sendall('GET / HTTP/1.1\r\nHost: www.google.com\r\n\r\n')
response = s.recv(8192)

where connect, sendall, and recv don't actually block the entire calling
thread, they only switch away to another coroutine until the underlying
operation completes. With "real" coroutines, you can do this.

I might be missing some subtlety of your point, but I've implemented
this functionality using generators in a library called Kaa[1]. In kaa,
your example looks like:

import kaa

@kaa.coroutine()
def do_some_network_stuff():
s = kaa.Socket()
yield s.connect('google.com:80')
yield s.write('GET / HTTP/1.1\nHost: www.google.com\n\n')
response = yield s.read()

do_some_network_stuff()
kaa.main.run()


Of course, it does require a "coroutine scheduler" be implemented,
which, in kaa, is taken care of by the main loop.

Cheers,
Jason.

[1] The curious can visit http://doc.freevo.org/api/kaa/base/ and
http://freevo.org/kaa/ although a 1.0 hasn't yet been released and the
docs are still rather sketchy.

 

[exarkun]

immediately outside the generator. This means that you cannot use
"enhanced generators" to implement an API like this one:

def doSomeNetworkStuff():
s = corolib.socket()
s.connect(('google.com', 80))
s.sendall('GET / HTTP/1.1\r\nHost: www.google.com\r\n\r\n')
response = s.recv(8192)

where connect, sendall, and recv don't actually block the entire
calling
thread, they only switch away to another coroutine until the
underlying
operation completes. With "real" coroutines, you can do this.

I might be missing some subtlety of your point, but I've implemented
this functionality using generators in a library called Kaa[1]. In
kaa,
your example looks like:

import kaa

@kaa.coroutine()
def do_some_network_stuff():
s = kaa.Socket()
yield s.connect('google.com:80')
yield s.write('GET / HTTP/1.1\nHost: www.google.com\n\n')
response = yield s.read()

do_some_network_stuff()
kaa.main.run()

I specifically left out all "yield" statements in my version, since
that's exactly the point here. With "real" coroutines, they're not
necessary - coroutine calls look just like any other call. With
Python's enhanced generators, they are.

Jean-Paul

 

[Jason Tackaberry]

I specifically left out all "yield" statements in my version, since
that's exactly the point here. With "real" coroutines, they're not
necessary - coroutine calls look just like any other call. With
Python's enhanced generators, they are.

Yes, I take your point.

Explicitly yielding may not be such a terrible thing, though. It's more
clear when reading such code where it may return back to the scheduler.
This may be important for performance, unless of course the coroutine
implementation supports preemption.

Cheers,
Jason.

 

[exarkun]

Yes, I take your point.

Explicitly yielding may not be such a terrible thing, though. It's
more
clear when reading such code where it may return back to the scheduler.
This may be important for performance, unless of course the coroutine
implementation supports preemption.

Sure, no value judgement intended, except on the practice of taking
words with well established meanings and re-using them for something
else

Jean-Paul

 

[Jason Tackaberry]

Sure, no value judgement intended, except on the practice of taking
words with well established meanings and re-using them for something
else

I think it's the behaviour that's important, and not the specific syntax
needed to implement that behaviour.

In other words, I disagree (if this is what you're suggesting) that
sticking "yield" in front of certain expressions makes it any less a
coroutine.

Now, requiring explicit yields does mean that the coroutine has
specific, well-defined points of reentry. But I don't believe it's a
necessary condition that coroutines allow arbitrary (in the
non-deterministic sense) reentry points, only multiple.

Cheers,
Jason.

 

[Antoine Pitrou]

Back when I worked on one of the first hand-held cellular
mobile phones, it used co-routines where the number of
coroutines was fixed at 2 (one for each register set in a Z80
CPU).

Gotta love the lightning-fast EXX instruction.

Regards

Antoine.

 

[Michele Simionato]

I've been taking a look at the multitude of coroutine libraries
available for Python, but from the looks of the projects they all seem
to be rather "quiet". I'd like to pick one up to use on a current
project but can't deduce which is the most popular/has the largest
community.

Libraries I looked at include: cogen, weightless, eventlet and
circuits (which isn't exactly coroutine-based but it's event-driven
model was intriguing).

Firstly, are there any others I've missed? And what would the
consensus be on the which has the most active community behind it?

The newest one seems to be diesel, by Simon Willison & Co: http://dieselweb..org/lib

 

[Hendrik van Rooyen]

Gotta love the lightning-fast EXX instruction.

Using it in the above context is about equivalent to slipping a hand grenade
in amongst the other eggs in a nest.



- Hendrik

 

[Grant Edwards]

Using it in the above context is about equivalent to slipping
a hand grenade in amongst the other eggs in a nest.

EXX accomplised much of the context switch operation. I don't
remember how much RAM was available, but it wasn't much...

 

[Piet van Oostrum]

e> I specifically left out all "yield" statements in my version, since that's
e> exactly the point here. With "real" coroutines, they're not necessary -
e> coroutine calls look just like any other call. With Python's enhanced
e> generators, they are.

The first time I encountered coroutines was in Simula-67. Coroutine
switching was certainly explicit there. IIRC, the keyword was resume.