I'm pleased to hear that and I hope the ideas are useful at some point in
time! (directly or indirectly
I don't have time to play with the code right now, but it sounds like fun.
I do have one question -- in practice, how difficult have you found it
that Python generators don't maintain separate stacks?
I'm not sure I understand the question! (I think you're either coming from a
false assumption or mean something different) I can see 3 possible
answers...
How difficult is it to build systems using them? Well, we managed to get a
novice programmer up to speed and happy with python and using the core
ideas in Kamaelia in a 2 week period. And over the 3 month period he was
with us he implemented a small system essentially demoing previewing
streamed content from a PVR on a Series 60 mobile (and on a desktop/laptop
PC). (Of course that could be because he was a natural, *or* because it can
be fairly easy - or both)
Regarding the idea that python generators stacks "collide" aren't separate.
If you mean an implementation detail I'm not aware of, it doesn't cause any
problems at all. If you are coming from the assumption that they share the
same state/stack frame...
Consider the example generator function:
.... a,b = 0,1
.... while 1:
.... yield b
.... a,b = b, a+b
....
Then call this twice yielding two different generator objects:<generator object at 0x403b03cc>
If they shared the same stack (by which I assume you mean stack frame), then
advancing one of these should advance both. Since in practice they run
independently you get the following behaviour:8
ie both generators "run" independently with different local values for a,b
since they *don't* share stack frames. (Also as I understand it the stack
frame associated with each generator is effectively put aside between
invocations)
As a result from that interpretation of your question (that it's based on
the mistaken assumption that state is shared between generators), we
haven't found it a problem at all because it's a false assumption - the
generators naturally don't share stacks.
The alternative is that you're talking about the fact that if you put
"yield" in a def statement that it forces that definition to define a
generator function. (ie you can't nest a yield)
For example in some TCP client code we have:
def runClient(self,sock=None):
try:
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM); yield 1
try:
sock.setblocking(0); yield 2
try:
while not self.safeConnect(sock,(self.host, self.port)):
yield 1
.... snip ...
It would be a lot more natural (especially with full co-routines) to have:
def runClient(self,sock=None):
try:
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM); yield 1
try:
sock.setblocking(0); yield 2
try:
self.safeConnect(sock,(self.host, self.port)):
.... snip ...
And to still have runClient be the generator, but have safeConnect doing the
yielding. (You can do this using Greenlets and/or Stackless for example)
However in practice it seems that this limitation is in fact more of a
strength - it encourages smaller simpler generators whose logic is more
exposed. Furthermore we very rarely have to nest calls to generators because
the way we deal with multiple generators is to set them running
independently and wait for communications between them.
These small generators also seem to lend themselves better to composition.
For example:
clientServerTestPort=1500
pipeline(TCPClient("127.0.0.1",clientServerTestPort),
VorbisDecode(),
AOAudioPlaybackAdaptor()
).run()
If you allow nested calls, it's perhaps natural to couple the last two
together as one component - but that is ultimately limiting - because each
component would then be trying to do too much.
Finally, in practice, it's pretty easy to write new components - we tend to
focus on the functionality we want, find it's main loop, add in yields in
key locations to effectively time slice it to turn it into a generator.
That then becomes the core of the component, and we then remove code that
deals with direct input/output (eg from files) and recv/send to/from
in/out-boxes.
It leads to a fairly natural process for creating generator based
components. (And can also lead to a fast assimilation process for
existing python based code, when needed) There's a walk through the
creation of multicast handling components here:
http://kamaelia.sourceforge.net/cgi-bin/blog/blog.cgi?rm=viewpost&postid=1113495151
It's a bit long winded - mainly due to repeatedly copying code to highlight
differences inplace - but should show that in practice it's a number of
small simple incremental steps. (Showing how we can go from standalone test
programs to generators to components) It also shows how the system can end
up causing you to create more general code than originally intended, which
is also more compact and obvious.
I suppose the short answer to your question though is that the defined
limitations of python's (simple) generators do in fact turn out to be a
strength IMO.
Anyhow, I hope that's answered your original question ? (or hopefully
explained why I found your question unclear!)
Best Regards,
Michael
