Ideas for yielding and exception raising

[Calvin Spealman]

I was wondering if it was possible, now or with a patch, to do either of the
following:

1) Cause another thread's most recent function, or a given function in a
given thread, to yield its generator immediately, such that the generator
can be used to pick back up where it left off. That is, the function itself
wouldn't need an actually yield keyword. Could be used to allow a function
to be controlled in how much time it has per second or somewhat?

2) Cause an exception to be raised in all threads that have a reference to a
given object, as if the object itself is raising the exception. This would
allow functions to return Monitor Objects that would raise exceptions if
something bad happened. For example, maybe you want the function to run
continually until someone else requests access to some syncronized data,
and the Monitor would raise the exception at that time.

 

[Peter Hansen]

I was wondering if it was possible, now or with a patch, to do either of the
following:

1) Cause another thread's most recent function, or a given function in a
given thread, to yield its generator immediately, such that the generator
can be used to pick back up where it left off. That is, the function itself
wouldn't need an actually yield keyword. Could be used to allow a function
to be controlled in how much time it has per second or somewhat?

2) Cause an exception to be raised in all threads that have a reference to a
given object, as if the object itself is raising the exception. This would
allow functions to return Monitor Objects that would raise exceptions if
something bad happened. For example, maybe you want the function to run
continually until someone else requests access to some syncronized data,
and the Monitor would raise the exception at that time.

Wouldn't it be better for both of these situations to do it
explicitly and actually write the code that way? What is
the advantage in putting this in the core and having it
happen magically and behind the scenes, as it appears you
want to happen?

-Peter

 

[Calvin Spealman]

Wouldn't it be better for both of these situations to do it
explicitly and actually write the code that way? What is
the advantage in putting this in the core and having it
happen magically and behind the scenes, as it appears you
want to happen?

-Peter

How would/could I do this explicitly?
--

 

[Peter Hansen]

(Reordered questions to make the answers appear next to them.)

How would/could I do this explicitly?

To a large extent that is unanswerable without actually writing
your code. If you know how to use generators, for example, then
the answer is "use generators"... All I meant here is that you
should actually *use* the yield keyword, writing the functions
that you want to control so that they give up the processor to the
calling routine at various intervals in a way that allows you to
control them as you wish. Of course, this just becomes a form of
cooperative multitasking, so you could also write it using threads
and voluntary checking of a "terminate" flag. With Python a
thread that does not block in an extension module cannot consume
all the CPU time unless it has been written deliberately to be
disruptive. The interpreter will only execute sys.checkinterval()
bytecode instructions at a time before switching to another thread.

This one I'm not even sure I understand, as you haven't really
described what you are trying to do. ("something bad happening" is
not an adequate description. ) It sounds like you want
the benefits of synchronization without any of the disadvantages.
If that's so, then you want to implement it yourself again in a
somewhat cooperative fashion, having the "function run continually"
but periodically check a flag that is set when another thread asks
for access. The first thread then blocks itself and releases
control of the shared resource, allowing the second one in.

Having written both the above attempts at answers, it seems to me
that in both cases you are trying to write some kind of threaded
code but without some of the disadvantages of threads. If you
are going to do threading, you should be very clear about what
kinds of synchronization you do or there are many ways it will
bite you in the ass. That's really what I mean about being
explicit. Use a Queue, write the code so that it's very clear
about where and when a generator yields control, or use whatever
other mechanisms are provided to do this already.

Anyway, changes to the core won't even be considered unless someone
can present some very compelling use cases for the behaviour.

And a patch.

-Peter