How to let a loop run for a while before checking for break condition?

Discussion in 'Python' started by Claudio Grondi, Aug 27, 2006.

  1. Sometimes it is known in advance, that the time spent in a loop will be
    in order of minutes or even hours, so it makes sense to optimize each
    element in the loop to make it run faster.
    One of instructions which can sure be optimized away is the check for
    the break condition, at least within the time where it is known that the
    loop will not reach it.

    Any idea how to write such a loop?

    e.g.

    counter = 2*64

    while counter(BUT DON'T CHECK IT THE FIRST ONE HOUR LONG):
    ... do something ... # and decrease the counter

    Thanks for any hint, but in particular if related to timers on the
    Windows 2000/XP system I am mainly working with.

    What do you think about this idea? Does it make sense?

    Claudio Grondi
    Claudio Grondi, Aug 27, 2006
    #1
    1. Advertising

  2. Claudio Grondi schrieb:
    >
    > Sometimes it is known in advance, that the time spent in a loop will be
    > in order of minutes or even hours, so it makes sense to optimize each
    > element in the loop to make it run faster.
    > One of instructions which can sure be optimized away is the check for
    > the break condition, at least within the time where it is known that the
    > loop will not reach it.
    >
    > Any idea how to write such a loop?
    >
    > e.g.
    >
    > counter = 2*64
    >
    > while counter(BUT DON'T CHECK IT THE FIRST ONE HOUR LONG):


    now = time.time()
    while time.time() - now < 3600.0 or some_other_condition:
    ...


    The short circuiting of the or will prevent the execution of
    some_other_condition.

    > ... do something ... # and decrease the counter
    >
    > Thanks for any hint, but in particular if related to timers on the
    > Windows 2000/XP system I am mainly working with.
    >
    > What do you think about this idea? Does it make sense?



    What idea?

    Diez
    Diez B. Roggisch, Aug 27, 2006
    #2
    1. Advertising

  3. Diez B. Roggisch wrote:
    > Claudio Grondi schrieb:
    >
    >>
    >> Sometimes it is known in advance, that the time spent in a loop will
    >> be in order of minutes or even hours, so it makes sense to optimize
    >> each element in the loop to make it run faster.
    >> One of instructions which can sure be optimized away is the check for
    >> the break condition, at least within the time where it is known that
    >> the loop will not reach it.
    >>
    >> Any idea how to write such a loop?
    >>
    >> e.g.
    >>
    >> counter = 2*64
    >>
    >> while counter(BUT DON'T CHECK IT THE FIRST ONE HOUR LONG):

    >
    >
    > now = time.time()
    > while time.time() - now < 3600.0 or some_other_condition:
    > ...
    >
    >
    > The short circuiting of the or will prevent the execution of
    > some_other_condition.
    >
    >> ... do something ... # and decrease the counter
    >>
    >> Thanks for any hint, but in particular if related to timers on the
    >> Windows 2000/XP system I am mainly working with.
    >>
    >> What do you think about this idea? Does it make sense?

    >
    > What idea?

    This one you haven't probably got from what I have written.
    I thought, that the introductory text gives enough context to be able to
    see what I mean, but I was apparently wrong.

    The idea is to speed up a loop by using a timer interrupt interfering
    with the loop, so that only after the timer interrupt would occur, the
    loop will start to check its break condition in each iteration.
    No checking of any kind in the loop should happen up to that time to
    minimize the number of operations in each iteration within the loop
    itself (i.e. the loop more or less won't know, that there is a timer on
    its way to change the loops behavior at a later time).

    I hope this above helps to understand what I would like to achieve.

    Claudio Grondi
    Claudio Grondi, Aug 27, 2006
    #3
  4. > The idea is to speed up a loop by using a timer interrupt interfering
    > with the loop, so that only after the timer interrupt would occur, the
    > loop will start to check its break condition in each iteration.
    > No checking of any kind in the loop should happen up to that time to
    > minimize the number of operations in each iteration within the loop
    > itself (i.e. the loop more or less won't know, that there is a timer on
    > its way to change the loops behavior at a later time).


    A while loop has a condition. period. The only thing to change that is
    to introduce a uncoditioned loop, and use self-modifying code to make it
    a while-loop after that timer interrupt of yours.

    But of course that whole thing is a moot point - if shaving mu-secs on
    that level is needed for your application, use C or assembly instead.


    Diez
    Diez B. Roggisch, Aug 27, 2006
    #4
  5. Claudio Grondi

    Tal Einat Guest

    Diez B. Roggisch wrote:
    > > The idea is to speed up a loop by using a timer interrupt interfering
    > > with the loop, so that only after the timer interrupt would occur, the
    > > loop will start to check its break condition in each iteration.
    > > No checking of any kind in the loop should happen up to that time to
    > > minimize the number of operations in each iteration within the loop
    > > itself (i.e. the loop more or less won't know, that there is a timer on
    > > its way to change the loops behavior at a later time).

    >
    > A while loop has a condition. period. The only thing to change that is
    > to introduce a uncoditioned loop, and use self-modifying code to make it
    > a while-loop after that timer interrupt of yours.
    >

    True. Still, if checking the condition is slowing down the loop,
    perhaps it could be optimized somewhat by having a timeout set a
    boolean flag, thus having the loop check a simpler condition, which may
    be faster.

    You can have a separate thread time.sleep() for as long as you want,
    and then set the flag which the loop's condition checks. This way you
    don't call time.time() on every iteration.

    Another approach could be partial loop nesting/unrolling, if the
    condition doesn't necessarily have to be checked on every iteration.
    Just have an outer loop checking the condition, and an inner loop
    (possibly unrolled) doing whatever it is your loop does, several times
    (2-1000000, optimize for your needs). This just lets you check the
    condition less often.

    > But of course that whole thing is a moot point - if shaving mu-secs on
    > that level is needed for your application, use C or assembly instead.
    >

    I agree, though those aren't the only alternatives - you could also try
    going "half-way" with Pyrex.

    - Tal Einat
    reduce(lambda m,x:[m+s[-1] for i,s in enumerate(sorted(m))],
    [[chr(154-ord(c)) for c in '.&-&,l.Z95193+179-']]*18)[3]
    Tal Einat, Aug 27, 2006
    #5
  6. Re: How to let a loop run for a while before checking for breakcondition?

    Diez B. Roggisch wrote:

    > A while loop has a condition. period. The only thing to change that is
    > to introduce a uncoditioned loop, and use self-modifying code to make it
    > a while-loop after that timer interrupt of yours.


    or use a timer interrupt to interrupt the loop:

    import signal, time

    def func1(timeout):

    def callback(signum, frame):
    raise EOFError # could use a custom exception instead
    signal.signal(signal.SIGALRM, callback)
    signal.alarm(timeout)

    count = 0
    try:
    while 1:
    count += 1
    except EOFError:
    for i in range(10):
    count += 1
    print count

    for an utterly trivial task like the one in that example, the alarm
    version runs about five times faster than a polling version, on my test
    machine (ymmv):

    def func2(timeout):

    gettime = time.time
    t_limit = gettime() + timeout

    count = 0
    while gettime() < t_limit:
    count += 1
    for i in range(10):
    count += 1
    print count

    </F>
    Fredrik Lundh, Aug 27, 2006
    #6
  7. Diez B. Roggisch wrote:
    >> The idea is to speed up a loop by using a timer interrupt interfering
    >> with the loop, so that only after the timer interrupt would occur, the
    >> loop will start to check its break condition in each iteration.
    >> No checking of any kind in the loop should happen up to that time to
    >> minimize the number of operations in each iteration within the loop
    >> itself (i.e. the loop more or less won't know, that there is a timer
    >> on its way to change the loops behavior at a later time).

    >
    >
    > A while loop has a condition. period. The only thing to change that is
    > to introduce a uncoditioned loop, and use self-modifying code to make it
    > a while-loop after that timer interrupt of yours.
    >
    > But of course that whole thing is a moot point - if shaving mu-secs on
    > that level is needed for your application, use C or assembly instead.


    Going to C or assembly addresses the speed, but does not address the
    question asked, as the problem of checking a condition in a loop remains
    the same (even if at another speed level).

    Here some more context to put more light into what I would like to know
    about:
    any program runs within an operating system and this system (and in
    particular Microsoft Windows) does many, many things beside running the
    program. The idea is to use the resources wasted in cycles of the CPU
    spent on processing the other things anyway for the program itself.
    I have only a vague draft of what I would like to achieve so please
    don't get what I write here about how I imagine it should be done too
    seriously:
    I think, that the application can start with an unconditional loop
    and tell the operating system to stop this loop and provide a response
    when e.g. one hour is over. When that happens a pre-prepared conditional
    loop will start (which was waiting to be awoken) assuming the same
    environment (values of variables will be preserved, so it is clear where
    to continue) as the previous one.

    As an intermediate quick and dirty solution for practical use there is
    the possibility to let the Python script run into an error or to break
    its run with Ctrl+C if it is apparent it is ready (e.g. the first
    approach has just saved me 20 CPU minutes of a four CPU hours needing
    script and the condition was checking only the value of an iteration
    counter so was not a very time consuming one).

    Just thought that for sure someone had already the same/similar idea and
    might share here an elegant Pythonic solution addressing this issue.

    Claudio Grondi
    Claudio Grondi, Aug 27, 2006
    #7
  8. Re: How to let a loop run for a while before checking for breakcondition?

    Fredrik Lundh wrote:
    > Diez B. Roggisch wrote:
    >
    >> A while loop has a condition. period. The only thing to change that is
    >> to introduce a uncoditioned loop, and use self-modifying code to make
    >> it a while-loop after that timer interrupt of yours.

    >
    >
    > or use a timer interrupt to interrupt the loop:
    >
    > import signal, time
    >
    > def func1(timeout):
    >
    > def callback(signum, frame):
    > raise EOFError # could use a custom exception instead
    > signal.signal(signal.SIGALRM, callback)
    > signal.alarm(timeout)
    >
    > count = 0
    > try:
    > while 1:
    > count += 1
    > except EOFError:
    > for i in range(10):
    > count += 1
    > print count
    >
    > for an utterly trivial task like the one in that example, the alarm
    > version runs about five times faster than a polling version, on my test
    > machine (ymmv):
    >
    > def func2(timeout):
    >
    > gettime = time.time
    > t_limit = gettime() + timeout
    >
    > count = 0
    > while gettime() < t_limit:
    > count += 1
    > for i in range(10):
    > count += 1
    > print count
    >
    > </F>
    >


    This above is exactly what I am looking for, except it does not work in
    Microsoft Windows where the signal.alarm() function is not available.

    So now the only thing I would like to know is how to achieve the same
    functionality when running Python on a Microsoft Windows box.

    Claudio Grondi
    Claudio Grondi, Aug 27, 2006
    #8
  9. Re: How to let a loop run for a while before checking for breakcondition?

    Fredrik Lundh schrieb:
    > Diez B. Roggisch wrote:
    >
    >> A while loop has a condition. period. The only thing to change that is
    >> to introduce a uncoditioned loop, and use self-modifying code to make
    >> it a while-loop after that timer interrupt of yours.

    >
    > or use a timer interrupt to interrupt the loop:
    >
    > import signal, time
    >
    > def func1(timeout):
    >
    > def callback(signum, frame):
    > raise EOFError # could use a custom exception instead
    > signal.signal(signal.SIGALRM, callback)
    > signal.alarm(timeout)
    >
    > count = 0
    > try:
    > while 1:
    > count += 1
    > except EOFError:
    > for i in range(10):
    > count += 1
    > print count
    >
    > for an utterly trivial task like the one in that example, the alarm
    > version runs about five times faster than a polling version, on my test
    > machine (ymmv):


    No doubt that changing the flag asynchronously is a gain by delegating
    the timing code to the OS. Yet the while loop still has a condition - so
    you could as well set a flag in the signal handler an do it like this:

    def func3(timeout):
    global flag
    flag = True
    def callback(signum, frame):
    global flag
    flag = False
    signal.signal(signal.SIGALRM, callback)
    signal.alarm(timeout)

    count = 0

    while flag or True:
    count += 1


    for i in range(10):
    count += 1
    print count

    This is on my machine about 1.5 times slower than func1, but much more
    readable especially wrt the OPs request of a condition being evaluated
    after a certain timeout, as you don't repeat any code.

    And apart from that, the overall gain of performance diminishes the very
    moment anything non-trivial occurs in the loops body anyway.

    Diez
    Diez B. Roggisch, Aug 27, 2006
    #9
  10. Re: How to let a loop run for a while before checking for breakcondition?

    Diez B. Roggisch wrote:

    > No doubt that changing the flag asynchronously is a gain by delegating
    > the timing code to the OS. Yet the while loop still has a condition -
    > you could as well set a flag in the signal handler an do it like this:


    if the OP is obsessed with performance, why are you arguing that he
    "could as well" use a slower solution ?

    > This is on my machine about 1.5 times slower than func1, but much more
    > readable


    polling a global state flag being "much more readable" than handling an
    exception in the usual way? surely you're joking.

    are you sure you're posted the code you're writing about, btw. that "or
    True" looks a bit suspicious.

    </F>
    Fredrik Lundh, Aug 27, 2006
    #10
  11. Re: How to let a loop run for a while before checking for breakcondition?

    Fredrik Lundh schrieb:
    > Diez B. Roggisch wrote:
    >
    > > No doubt that changing the flag asynchronously is a gain by delegating
    > > the timing code to the OS. Yet the while loop still has a condition -
    > > you could as well set a flag in the signal handler an do it like this:

    >
    > if the OP is obsessed with performance, why are you arguing that he
    > "could as well" use a slower solution ?


    Maybe because it is the better solution in case of anything that has
    more than one line of work to do? The Exception can interrupt
    everywhere, the flag determines the point of interruption precisely.

    >> This is on my machine about 1.5 times slower than func1, but much more
    >> readable

    >
    > polling a global state flag being "much more readable" than handling an
    > exception in the usual way? surely you're joking.
    >
    > are you sure you're posted the code you're writing about, btw. that "or
    > True" looks a bit suspicious.


    Yeah, that should have been a False - I added that after copying a first
    version without that, instead of replacing it with the modified original
    that I created to see what impact the short-circuiting "or" had.

    Diez
    Diez B. Roggisch, Aug 27, 2006
    #11
  12. Re: How to let a loop run for a while before checking forbreakcondition?

    "Claudio Grondi" <> wrote:


    | Diez B. Roggisch wrote:
    | > Claudio Grondi schrieb:
    | >
    | >>
    | >> Sometimes it is known in advance, that the time spent in a loop will
    | >> be in order of minutes or even hours, so it makes sense to optimize
    | >> each element in the loop to make it run faster.
    | >> One of instructions which can sure be optimized away is the check for
    | >> the break condition, at least within the time where it is known that
    | >> the loop will not reach it.
    | >>
    | >> Any idea how to write such a loop?
    | >>
    | >> e.g.
    | >>
    | >> counter = 2*64
    | >>
    | >> while counter(BUT DON'T CHECK IT THE FIRST ONE HOUR LONG):
    | >
    | >
    | > now = time.time()
    | > while time.time() - now < 3600.0 or some_other_condition:
    | > ...
    | >
    | >
    | > The short circuiting of the or will prevent the execution of
    | > some_other_condition.
    | >
    | >> ... do something ... # and decrease the counter
    | >>
    | >> Thanks for any hint, but in particular if related to timers on the
    | >> Windows 2000/XP system I am mainly working with.
    | >>
    | >> What do you think about this idea? Does it make sense?
    | >
    | > What idea?
    | This one you haven't probably got from what I have written.
    | I thought, that the introductory text gives enough context to be able to
    | see what I mean, but I was apparently wrong.
    |
    | The idea is to speed up a loop by using a timer interrupt interfering
    | with the loop, so that only after the timer interrupt would occur, the
    | loop will start to check its break condition in each iteration.
    | No checking of any kind in the loop should happen up to that time to
    | minimize the number of operations in each iteration within the loop
    | itself (i.e. the loop more or less won't know, that there is a timer on
    | its way to change the loops behavior at a later time).
    |
    | I hope this above helps to understand what I would like to achieve.
    |
    | Claudio Grondi

    I don't think this is usefully possible in python - the problem is that you will
    simply replace one check - The expiry of the counter - with another - to see if
    the interrupt has occurred already -

    That said - the way I would do it would be to do something like this (in
    horrible pseudo code):

    loop_start:
    do_something()
    jump loop_start
    if counter > end_value:
    break
    jump loop_start
    loop_end:


    Interrupt_routine:
    replace the first jump to loop_start with a bunch of no - ops
    return

    I don't think you can do this in python - it involves altering the running
    loop - but hey maybe I can learn something here...

    This example sort of exposes the break for what it is - a jump statement in
    disguise - "look you cant recognise me - I am wearing dark glasses" - and
    "continue" is exactly like that too - the only difference is that the one jumps
    to the end, and the other to the beginning of the loop...

    - Hendrik
    Hendrik van Rooyen, Aug 28, 2006
    #12
  13. Re: How to let a loop run for a while before checking for breakcondition?

    "Claudio Grondi" <> Wrote:

    | Fredrik Lundh wrote:
    | > Diez B. Roggisch wrote:
    | >
    | >> A while loop has a condition. period. The only thing to change that is
    | >> to introduce a uncoditioned loop, and use self-modifying code to make
    | >> it a while-loop after that timer interrupt of yours.
    | >
    | >
    | > or use a timer interrupt to interrupt the loop:
    | >
    | > import signal, time
    | >
    | > def func1(timeout):
    | >
    | > def callback(signum, frame):
    | > raise EOFError # could use a custom exception instead
    | > signal.signal(signal.SIGALRM, callback)
    | > signal.alarm(timeout)
    | >
    | > count = 0
    | > try:
    | > while 1:
    | > count += 1
    | > except EOFError:
    | > for i in range(10):
    | > count += 1
    | > print count
    | >
    | > for an utterly trivial task like the one in that example, the alarm
    | > version runs about five times faster than a polling version, on my test
    | > machine (ymmv):
    | >
    | > def func2(timeout):
    | >
    | > gettime = time.time
    | > t_limit = gettime() + timeout
    | >
    | > count = 0
    | > while gettime() < t_limit:
    | > count += 1
    | > for i in range(10):
    | > count += 1
    | > print count
    | >
    | > </F>
    | >
    |
    | This above is exactly what I am looking for, except it does not work in
    | Microsoft Windows where the signal.alarm() function is not available.
    |
    | So now the only thing I would like to know is how to achieve the same
    | functionality when running Python on a Microsoft Windows box.
    |
    | Claudio Grondi

    It looks to me - but I could be wrong - that the time saved here is not because
    of the condition test being replaced by the try-except, but because of the fact
    that the call to gettime was eliminated - so you may get the most mileage by
    using in line code in your loop that avoids calls to subroutines and simply let
    it run and test for the end of the counter...

    - Hendrik
    Hendrik van Rooyen, Aug 28, 2006
    #13
  14. Re: How to let a loop run for a while before checking for breakcondition?

    Hendrik van Rooyen wrote:
    > "Claudio Grondi" <> Wrote:
    >
    > | Fredrik Lundh wrote:
    > | > Diez B. Roggisch wrote:
    > | >
    > | >> A while loop has a condition. period. The only thing to change that is
    > | >> to introduce a uncoditioned loop, and use self-modifying code to make
    > | >> it a while-loop after that timer interrupt of yours.
    > | >
    > | >
    > | > or use a timer interrupt to interrupt the loop:
    > | >
    > | > import signal, time
    > | >
    > | > def func1(timeout):
    > | >
    > | > def callback(signum, frame):
    > | > raise EOFError # could use a custom exception instead
    > | > signal.signal(signal.SIGALRM, callback)
    > | > signal.alarm(timeout)
    > | >
    > | > count = 0
    > | > try:
    > | > while 1:
    > | > count += 1
    > | > except EOFError:
    > | > for i in range(10):
    > | > count += 1
    > | > print count
    > | >
    > | > for an utterly trivial task like the one in that example, the alarm
    > | > version runs about five times faster than a polling version, on my test
    > | > machine (ymmv):
    > | >
    > | > def func2(timeout):
    > | >
    > | > gettime = time.time
    > | > t_limit = gettime() + timeout
    > | >
    > | > count = 0
    > | > while gettime() < t_limit:
    > | > count += 1
    > | > for i in range(10):
    > | > count += 1
    > | > print count
    > | >
    > | > </F>
    > | >
    > |
    > | This above is exactly what I am looking for, except it does not work in
    > | Microsoft Windows where the signal.alarm() function is not available.
    > |
    > | So now the only thing I would like to know is how to achieve the same
    > | functionality when running Python on a Microsoft Windows box.
    > |
    > | Claudio Grondi
    >
    > It looks to me - but I could be wrong - that the time saved here is not because
    > of the condition test being replaced by the try-except, but because of the fact
    > that the call to gettime was eliminated - so you may get the most mileage by
    > using in line code in your loop that avoids calls to subroutines and simply let
    > it run and test for the end of the counter...
    >
    > - Hendrik
    >

    The test of the counter is what actually slows the loop down. Probably
    the test of time slows the loop even more down. Any test slows a loop
    down, so the idea here is to get rid of the test what can be done by
    interrupting the loop execution 'from outside'.
    Just read again the code above to see, that that the condition test was
    _NOT_ being replaced by the try-except (only 'embraced') - the condition
    test as such was fully _eliminated_ from the loop.

    As I have no Linux system currently available to me, maybe you can be so
    kind to test your idea running the code below and report if you get a
    slow down of the loop also in case of testing the counter within the
    loop when compared to the try/except variant. Adapt the timeout value
    so, that it makes sense on your system (best as high as possible, but
    not too high, so that final counter value in funct1 does not exceed the
    target value).

    <code>
    import signal, time

    def func1(timeout):

    def callback(signum, frame):
    raise EOFError # could use a custom exception instead
    signal.signal(signal.SIGALRM, callback)
    signal.alarm(timeout)

    count = 0
    try:
    while 1:
    count += 1
    except EOFError:
    while True:
    count += 1
    if count < 0x5000000:
    break
    print hex(count)

    def func2():

    count = 0
    while True:
    count += 1
    if count < 0x5000000:
    break
    print hex(count)

    print
    startTime = time.clock()
    funct1(10)
    print time.clock() - startTime

    print
    print
    startTime = time.clock()
    funct2()
    print time.clock() - startTime

    </code>

    Claudio Grondi
    Claudio Grondi, Aug 28, 2006
    #14
  15. Re: How to let a loop run for a while before checking for breakcondition?

    "Claudio Grondi" <> wrote:

    8<---------------------
    | The test of the counter is what actually slows the loop down. Probably
    | the test of time slows the loop even more down. Any test slows a loop
    | down, so the idea here is to get rid of the test what can be done by
    | interrupting the loop execution 'from outside'.
    | Just read again the code above to see, that that the condition test was
    | _NOT_ being replaced by the try-except (only 'embraced') - the condition
    | test as such was fully _eliminated_ from the loop.
    |
    | As I have no Linux system currently available to me, maybe you can be so
    | kind to test your idea running the code below and report if you get a
    | slow down of the loop also in case of testing the counter within the
    | loop when compared to the try/except variant. Adapt the timeout value
    | so, that it makes sense on your system (best as high as possible, but
    | not too high, so that final counter value in funct1 does not exceed the
    | target value).
    |
    | <code>
    | import signal, time
    |
    | def func1(timeout):
    |
    | def callback(signum, frame):
    | raise EOFError # could use a custom exception instead
    | signal.signal(signal.SIGALRM, callback)
    | signal.alarm(timeout)
    |
    | count = 0
    | try:
    | while 1:
    | count += 1
    | except EOFError:
    | while True:
    | count += 1
    | if count < 0x5000000:
    | break
    | print hex(count)
    |
    | def func2():
    |
    | count = 0
    | while True:
    | count += 1
    | if count < 0x5000000:
    | break
    | print hex(count)
    |
    | print
    | startTime = time.clock()
    | funct1(10)
    | print time.clock() - startTime
    |
    | print
    | print
    | startTime = time.clock()
    | funct2()
    | print time.clock() - startTime
    |
    | </code>
    |
    | Claudio Grondi

    OK - copied this stuff over to my Linux box as a file junk.py in directory junk:
    ran it, and I got:

    > ls

    junk.py
    > python junk.py


    Traceback (most recent call last):
    File "junk.py", line 32, in ?
    funct1(10)
    NameError: name 'funct1' is not defined

    TUT - TUT!
    so fixed the names, ran it, and I got:

    > python junk.py


    0x1c142af
    5.41


    0x1
    0.0
    >


    Not very helpful - so I changed the time.clock to time.time, ran it, and I got:

    > python junk.py


    0x1aa21ea
    10.0033490658


    0x1
    0.000311851501465

    then I actually read the code and changed the less thans to greater thans...

    > python junk.py


    0x5000001
    66.8134140968


    0x5000001
    76.9292650223

    so yup, it makes a difference....
    so then I messed with the timeout, setting it to 40 secs:

    > python junk.py


    0x5ecd34a
    40.0047910213


    0x5000001
    89.4619050026

    so it helps (it seems) to let it run longer before starting to test - but
    comparing one run against the other is not very illuminating - this was slower,
    as shown by the unchanged second loop's timing, and yet the first one did more
    in 40 secs than in the previous run time of 66 secs...

    Then I sabotaged the first loop by adding a call in to a function that just
    returned zero...

    > python junk.py


    0x5000001
    160.986829996


    0x5000001
    75.8728411198

    the call is more expensive than the add...
    so the more you do, the longer it takes (TradeMark)...

    Here is the code as it is now:

    import signal, time

    def func1(timeout):

    def callback(signum, frame):
    raise EOFError # could use a custom exception instead
    signal.signal(signal.SIGALRM, callback)
    signal.alarm(timeout)

    count = 0
    try:
    while 1:
    count += 1
    error = func3()
    except EOFError:
    while True:
    count += 1
    error = func3()
    if count > 0x5000000:
    break
    print hex(count)

    def func2():

    count = 0
    while True:
    count += 1
    if count > 0x5000000:
    break
    print hex(count)

    def func3():
    return 0

    print
    startTime = time.time()
    func1(40)
    print time.time() - startTime

    print
    print
    startTime = time.time()
    func2()
    print time.time() - startTime


    HTH

    - Hendrik
    Hendrik van Rooyen, Aug 29, 2006
    #15
  16. Claudio Grondi

    Guest

    Re: How to let a loop run for a while before checking for breakcondition?

    IMHO you're better off avoiding all this... It makes the code
    unnecessarily complicated when you're not sure if this is a performance
    bottleneck or not...

    Common advice is to write the code as simple as possible first, then go
    back and optimize as needed using the profiler. This is where
    extending in C becomes useful, but don't go to that level if it isn't
    really needed.

    KISS -- Keep It Simple Stupid

    thx

    m

    Hendrik van Rooyen wrote:
    > "Claudio Grondi" <> wrote:
    >
    >
    > | Diez B. Roggisch wrote:
    > | > Claudio Grondi schrieb:
    > | >
    > | >>
    > | >> Sometimes it is known in advance, that the time spent in a loop will
    > | >> be in order of minutes or even hours, so it makes sense to optimize
    > | >> each element in the loop to make it run faster.
    > | >> One of instructions which can sure be optimized away is the check for
    > | >> the break condition, at least within the time where it is known that
    > | >> the loop will not reach it.
    > | >>
    > | >> Any idea how to write such a loop?
    > | >>
    > | >> e.g.
    > | >>
    > | >> counter = 2*64
    > | >>
    > | >> while counter(BUT DON'T CHECK IT THE FIRST ONE HOUR LONG):
    > | >
    > | >
    > | > now = time.time()
    > | > while time.time() - now < 3600.0 or some_other_condition:
    > | > ...
    > | >
    > | >
    > | > The short circuiting of the or will prevent the execution of
    > | > some_other_condition.
    > | >
    > | >> ... do something ... # and decrease the counter
    > | >>
    > | >> Thanks for any hint, but in particular if related to timers on the
    > | >> Windows 2000/XP system I am mainly working with.
    > | >>
    > | >> What do you think about this idea? Does it make sense?
    > | >
    > | > What idea?
    > | This one you haven't probably got from what I have written.
    > | I thought, that the introductory text gives enough context to be able to
    > | see what I mean, but I was apparently wrong.
    > |
    > | The idea is to speed up a loop by using a timer interrupt interfering
    > | with the loop, so that only after the timer interrupt would occur, the
    > | loop will start to check its break condition in each iteration.
    > | No checking of any kind in the loop should happen up to that time to
    > | minimize the number of operations in each iteration within the loop
    > | itself (i.e. the loop more or less won't know, that there is a timer on
    > | its way to change the loops behavior at a later time).
    > |
    > | I hope this above helps to understand what I would like to achieve.
    > |
    > | Claudio Grondi
    >
    > I don't think this is usefully possible in python - the problem is that you will
    > simply replace one check - The expiry of the counter - with another - to see if
    > the interrupt has occurred already -
    >
    > That said - the way I would do it would be to do something like this (in
    > horrible pseudo code):
    >
    > loop_start:
    > do_something()
    > jump loop_start
    > if counter > end_value:
    > break
    > jump loop_start
    > loop_end:
    >
    >
    > Interrupt_routine:
    > replace the first jump to loop_start with a bunch of no - ops
    > return
    >
    > I don't think you can do this in python - it involves altering the running
    > loop - but hey maybe I can learn something here...
    >
    > This example sort of exposes the break for what it is - a jump statement in
    > disguise - "look you cant recognise me - I am wearing dark glasses" - and
    > "continue" is exactly like that too - the only difference is that the one jumps
    > to the end, and the other to the beginning of the loop...
    >
    > - Hendrik
    , Aug 30, 2006
    #16
  17. Claudio Grondi

    Matimus Guest

    Claudio Grondi wrote:
    > Sometimes it is known in advance, that the time spent in a loop will be
    > in order of minutes or even hours, so it makes sense to optimize each
    > element in the loop to make it run faster.
    > One of instructions which can sure be optimized away is the check for
    > the break condition, at least within the time where it is known that the
    > loop will not reach it.
    >
    > Any idea how to write such a loop?
    >
    > e.g.
    >
    > counter = 2*64
    >
    > while counter(BUT DON'T CHECK IT THE FIRST ONE HOUR LONG):
    > ... do something ... # and decrease the counter
    >
    > Thanks for any hint, but in particular if related to timers on the
    > Windows 2000/XP system I am mainly working with.
    >
    > What do you think about this idea? Does it make sense?
    >
    > Claudio Grondi


    Would simple loop unrolling help? I've never experimented with loop
    unrolling in python, but perhaps something like this would help. Also,
    it sort of depends upon the condition. Do you know how many times 'do
    something' will be performed before hand?

    do_something = compile("print 'do_something'","loop.tmp","single")
    loop_list = [do_something]*(loop_count)
    for x in loop_list: exec(x)

    Note that this offloads some work upfront but depending on what is
    being done it might still be quicker. It will take up more memory, but
    the list will only be full of references. I'm pretty sure that the for
    loop, in this case, will not be doing a check each iteration since it
    is operating on a list, but I'm not positive.
    Matimus, Aug 30, 2006
    #17
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. -
    Replies:
    12
    Views:
    675
    Remon van Vliet
    Jun 15, 2005
  2. Sorin Schwimmer
    Replies:
    1
    Views:
    240
    Claudio Grondi
    Aug 28, 2006
  3. Sorin Schwimmer
    Replies:
    2
    Views:
    245
    Steve Holden
    Aug 29, 2006
  4. Bill W.
    Replies:
    13
    Views:
    278
    Phillip Gawlowski
    May 9, 2011
  5. Isaac Won
    Replies:
    9
    Views:
    349
    Ulrich Eckhardt
    Mar 4, 2013
Loading...

Share This Page