Re: Co-routines

Discussion in 'Python' started by Peter Hansen, Jul 17, 2003.

  1. Peter Hansen

    Peter Hansen Guest

    wrote:
    >
    > I have an application in which I want the users to be able to create python
    > functions:
    >
    > def f1():
    > print "1-1"
    > print "1-2"
    > print "1-3"
    >
    > def f2():
    > print "2-1"
    > print "2-2"
    > print "3-3"
    >
    > and when my application runs, I want to execute these functions in
    > "lock-step", so that the output looks like:
    >
    > 1-1
    > 2-2
    > 1-2
    > 2-2
    > 1-3
    > 2-3


    I think the problem is underspecified. What do you mean by the quoted
    phrase "lock-step"? Your example includes only simple print statements,
    which generate output to sys.stdout. What are you really trying to
    accomplish? Do you want individual bytecodes to be executed one at a
    time from a series of functions? Do you want any output to be interlaced
    on a line-by-line basis? Any call to any I/O routine? Need more detail.

    -Peter
    Peter Hansen, Jul 17, 2003
    #1
    1. Advertising

  2. Peter Hansen

    Guest Guest

    "Peter Hansen" <> wrote in message
    news:...
    > wrote:
    > >
    > > I have an application in which I want the users to be able to create

    python
    > > functions:
    > >
    > > def f1():
    > > print "1-1"
    > > print "1-2"
    > > print "1-3"
    > >
    > > def f2():
    > > print "2-1"
    > > print "2-2"
    > > print "3-3"
    > >
    > > and when my application runs, I want to execute these functions in
    > > "lock-step", so that the output looks like:
    > >
    > > 1-1
    > > 2-2
    > > 1-2
    > > 2-2
    > > 1-3
    > > 2-3

    >
    > I think the problem is underspecified. What do you mean by the quoted
    > phrase "lock-step"? Your example includes only simple print statements,
    > which generate output to sys.stdout. What are you really trying to
    > accomplish? Do you want individual bytecodes to be executed one at a
    > time from a series of functions? Do you want any output to be interlaced
    > on a line-by-line basis? Any call to any I/O routine? Need more detail.
    >
    > -Peter


    OK. Thanks. I was just giving some simple examples...but I hope this is
    better:

    From the python grammar:

    funcdef ::= "def" funcname "(" [parameter_list] ")" ":" suite

    I want the statements in <suite> to be executed on a statement by statement
    basis, with no particular restrictions on *what* those statements are. It's
    the execution stepping I'm interested in... the I/O in my example was just
    that. I understand that there will be all the problems on contention/race
    etc (just like if the functions were going in separate threads)
    Guest, Jul 17, 2003
    #2
    1. Advertising

  3. Peter Hansen

    Sean Ross Guest

    "Peter Hansen" <> wrote in message
    > I think the problem is underspecified. What do you mean by the quoted
    > phrase "lock-step"?


    He wants to have f1 and f2 alternate executing instructions:

    # pseudo code
    def f1():
    instruction1-1
    instruction1-2

    def f2():
    instruction2-1
    instruction2-2

    So, for the code above, he would like to see something like the following
    execution routine:

    start execution of f1
    execute instruction1-1
    pause execution of f1
    start execution of f2
    execute instruction2-1
    pause execution of f2
    resume execution of f1
    execute instruction1-2
    pause execution of f1
    resume execution of f2
    execute instruction2-2
    pause execution of f2

    Do a piece of one, then do a piece of the other, then do another piece of
    the former, and so on...

    While the following code does not provide a solution to the OP's request, it
    does show something very much like the behaviour that's being asked for.

    class LockStepper:
    def __init__(self, id):
    self.id = id
    self.i = 1
    self.partner = None

    def next(self):
    if self.i > 10:
    return
    print "instruction %s-%s"%(self.id, self.i)
    self.i += 1
    self.partner.next()

    f1 = LockStepper(1)
    f2 = LockStepper(2)
    f1.partner = f2
    f2.partner = f1

    # begin lock-step execution
    f1.next()

    The idea here is to think of each LockStepper as though it were a function,
    and to think of the print statement in each of their next() methods as a
    line of instruction inside the body of that function. The LockStepper
    executes the first line of it's "function body", then pauses. It's partner
    executes the first line of it's function body, then pauses. The partner's
    partner was the original LockStepper, so it "resumes execution"
    (self.partner.next()) and executes it's second line of instructions. And so
    on...

    If someone better at generators than I could put together a version of this
    where the LockStepper's have a __call__() method that yields execution back
    and forth between a partner LockStepper, then we'd be closer to simulating
    the behaviour, but not to solving the problem, which asks for this behaviour
    using arbitrary functions (not necessarily LockSteppers). And, I've know
    idea how to do that :)

    Hope that clears things up a little,
    Sean
    Sean Ross, Jul 17, 2003
    #3
  4. Peter Hansen

    Peter Hansen Guest

    wrote:
    >
    > From the python grammar:
    >
    > funcdef ::= "def" funcname "(" [parameter_list] ")" ":" suite
    >
    > I want the statements in <suite> to be executed on a statement by statement
    > basis, with no particular restrictions on *what* those statements are. It's
    > the execution stepping I'm interested in... the I/O in my example was just
    > that. I understand that there will be all the problems on contention/race
    > etc (just like if the functions were going in separate threads)


    Okay, that's clear. It can't be done. :) Now on to the next step:
    what are you actually trying to do. That is, *why* do you want the
    behaviour you say you want? That might give us a few ideas about what
    can really be done.

    Basically, what you describe would require the interpretation of
    Python at a higher level than it is interpreted now. It is actually
    *compiled* code, turned from source (the lines you refer to) into
    bytecodes, which are effectively machine code for the Python virtual
    machine. It is at the level of bytecodes that one might actually
    start to get the control you're talking about.

    Another approach would be if you reduced the problem from arbitrary
    statements to some kind of subset. Maybe evaluation of expressions
    only, without loops or most statements? Of course, that might be
    far too limited for you, but you haven't said why you want to do this
    yet. If you could take that approach, you would basically retrieve
    the source line by line and pass it to "exec" or maybe eval().

    You could also just define your own mini-language, and write an
    interpreter for that which has the behaviour you need.

    Or, depending again on why you want all this, you could require
    construction of the code using generators or something, as others
    are suggesting.

    -Peter
    Peter Hansen, Jul 17, 2003
    #4
  5. Peter Hansen

    Sean Ross Guest

    Actually, that last bit of code wasn't quite right, since it relied on the
    LockStepper's to control the alternating between each others statement
    execution. What is probably being asked for is something outside the
    LockStepper's that will manage the switching.

    class LockStepper:
    def __init__(self, id, nstmts):
    self.id = id
    self.stmt = 1
    self.nstmts = nstmts

    def next(self):
    print "instruction %s-%s"%(self.id, self.stmt)
    self.stmt += 1

    def hasnext(self):
    return self.stmt <= self.nstmts

    class Switcher:
    def __init__(self, *steppers):
    self.steppers = steppers
    def __call__(self):
    while True:
    for stepper in self.steppers:
    if stepper.hasnext():
    stepper.next()
    if self._done():
    break
    def _done(self):
    return not [f for f in (f1,f2) if f.hasnext()]


    f1 = LockStepper(1, 4)
    f2 = LockStepper(2, 7)
    switch = Switcher(f1,f2)
    switch()
    Sean Ross, Jul 17, 2003
    #5
  6. Peter Hansen

    Sean Ross Guest

    > def _done(self):
    > return not [f for f in (f1,f2) if f.hasnext()]


    # that should have been
    def _done(self):
    return not [f for f in self.steppers if f.hasnext()]
    Sean Ross, Jul 17, 2003
    #6
  7. Peter Hansen

    Duncan Booth Guest

    <> wrote in
    news:pBxRa.308$:

    > OK. Thanks. I was just giving some simple examples...but I hope this
    > is better:
    >
    > From the python grammar:
    >
    > funcdef ::= "def" funcname "(" [parameter_list] ")" ":" suite
    >
    > I want the statements in <suite> to be executed on a statement by
    > statement basis, with no particular restrictions on *what* those
    > statements are. It's the execution stepping I'm interested in... the
    > I/O in my example was just that. I understand that there will be all
    > the problems on contention/race etc (just like if the functions were
    > going in separate threads)


    The only way I can think to implement this 'easily' would be to use the
    parser module to parse your source code, then walk the parse tree looking
    for statements inside one of your functions and add a yield_stmt after
    every other statement. Then compile the tree into bytecode and execute it.

    Of course your users may be surprised to find that they cannot directly
    call any of their functions from another function. You could do a further
    rewrite on the tree to convert obvious calls to your users' functions into
    for loops.

    Why?

    --
    Duncan Booth
    int month(char *p){return(124864/((p[0]+p[1]-p[2]&0x1f)+1)%12)["\5\x8\3"
    "\6\7\xb\1\x9\xa\2\0\4"];} // Who said my code was obscure?
    Duncan Booth, Jul 17, 2003
    #7
  8. Peter Hansen

    Alan Kennedy Guest

    wrote:

    > From the python grammar:
    >
    > funcdef ::= "def" funcname "(" [parameter_list] ")" ":" suite
    >
    > I want the statements in <suite> to be executed on a statement by
    > statement basis, with no particular restrictions on *what* those
    > statements are. It's the execution stepping I'm interested in...


    Might your requirements be better met by a debugger, which is feeding
    step instructions to a normal python program? This could be done
    extending the Pdb debugger class in module Lib/pdb.py to implement
    your own single step method. This could control various executing
    suites of code, according to your chosen scheduling algorithm.

    Or might you be better off to actually spawn separate threads, and
    have each thread carry out each instruction, line by line, and then
    enter a blocking wait for either a "continue" token from a
    Queue.Queue(), or "Continue" message signalled on a
    threading.Condition()?

    > the I/O in my example was just that.


    Or is that you want to control the states of your objects, depending
    on the readiness status of various I/O channels? And you want to run
    everything in one thread? In which case you might be looking for
    ultra-lightweight threads, based on generators, combined with an
    asynchronous I/O detection and reaction platform, such as asyncore,
    twisted or medusa.

    > I understand that there will
    > be all the problems on contention/race etc (just like if the
    > functions were going in separate threads)


    If the objects which are being distributed out to your
    (ultra-lightweight) threads are local to the functions/objects (i.e.
    not shared with other functions/objects) that process them, then there
    is no contention, and thus no locking required.

    How many separate "threads" of execution are you seeking to create?

    --
    alan kennedy
    -----------------------------------------------------
    check http headers here: http://xhaus.com/headers
    email alan: http://xhaus.com/mailto/alan
    Alan Kennedy, Jul 17, 2003
    #8
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Andrew Rich

    Proper serial routines ?

    Andrew Rich, Oct 4, 2003, in forum: Perl
    Replies:
    1
    Views:
    439
    Al Tobey
    Oct 24, 2003
  2. Soenke
    Replies:
    0
    Views:
    547
    Soenke
    Dec 28, 2005
  3. Matthew Hood

    How to create global Funtions & Routines

    Matthew Hood, Jun 24, 2003, in forum: ASP .Net
    Replies:
    2
    Views:
    1,912
    Matthew Hood
    Jun 25, 2003
  4. Keith A. Rowe
    Replies:
    2
    Views:
    293
    Keith A. Rowe
    Feb 14, 2004
  5. tshad
    Replies:
    2
    Views:
    698
    tshad
    Jan 28, 2005
Loading...

Share This Page