Condition.wait(timeout) oddities

[Floris Bruynooghe]

Hi all

I'm a little confused about the corner cases of Condition.wait() with a timeout parameter in the threading module.

When looking at the code the first thing that I don't quite get is that thetimeout should never work as far as I understand it. .wait() always needsto return while holding the lock, therefore it does an .acquire() on the lock in a finally clause. Thus pretty much ignoring the timeout value.

The second issue is that while looking around for this I found two bug reports: http://bugs.python.org/issue1175933 and http://bugs.python.org/issue10218. Both are proposing to add a return value indicating whether the .wait() timed out or not similar to the other .wait() methods in threading. However the first was rejected after some (seemingly inconclusive) discussion.While the latter had minimal discussion and and was accepted without reference to the earlier attempt. Not sure if this was a process oversight or what, but it does leave the situation confusing.

But regardless I don't understand how the return value can be used currently: yes you did time out but you're still promised to hold the lock thanks to the .acquire() call on the lock in the finally block.

In my small brain I just can't figure out how Condition.wait() can both respect a timeout parameter and the promise to hold the lock on return. It seems to me that the only way to handle the timeout is to raise an exception rather then return a value because when you get an exception you can break the promise of holding the lock.

But maybe I'm missing something important or obvious, so I'd be happy to beenlightened!

Regards
Floris

 

[Chris Torek]

I'm a little confused about the corner cases of Condition.wait() with a
timeout parameter in the threading module.

When looking at the code the first thing that I don't quite get is that
the timeout should never work as far as I understand it. .wait() always
needs to return while holding the lock, therefore it does an .acquire()
on the lock in a finally clause. Thus pretty much ignoring the timeout
value.

It does not do a straight acquire, it uses self._acquire_restore(),
which for a condition variable, does instead:

self.__block.acquire()
self.__count = count
self.__owner = owner

(assuming that you did not override the lock argument or passed
in a threading.RLock() object as the lock), due to this bit of
code in _Condition.__init__():

# If the lock defines _release_save() and/or _acquire_restore(),
# these override the default implementations (which just call
# release() and acquire() on the lock). Ditto for _is_owned().
[snippage]
try:
self._acquire_restore = lock._acquire_restore
except AttributeError:
pass

That is, lock it holds is the one on the "blocking lock" (the
__block of the underlying RLock), which is the same one you had
to hold in the first place to call the .wait() function.

To put it another way, the lock that .wait() waits for is
a new lock allocated for the duration of the .wait() operation:

waiter = _allocate_lock()
waiter.acquire()
self.__waiters.append(waiter)
saved_state = self._release_save()
<here we wait for lock "waiter", with timeout>
self._acquire_restore(saved_state)
# the last stmt is the "finally" clause, I've just un-indented it

which is entirely different from the lock that .wait() re-acquires
(and which you held when you called .wait() initially) before it
returns.

The second issue is that while looking around for this I found two bug
reports: http://bugs.python.org/issue1175933 and
http://bugs.python.org/issue10218. Both are proposing to add a return
value indicating whether the .wait() timed out or not similar to the
other .wait() methods in threading. However the first was rejected
after some (seemingly inconclusive) discussion.

Tim Peters' reply seemed pretty conclusive to me.

While the latter had
minimal discussion and and was accepted without reference to the earlier
attempt. Not sure if this was a process oversight or what, but it does
leave the situation confusing.

But regardless I don't understand how the return value can be used
currently: yes you did time out but you're still promised to hold the
lock thanks to the .acquire() call on the lock in the finally block.

The return value is not generally useful for the reasons Tim Peters
noted originally. Those are all still true even in the second
discussion.

In my small brain I just can't figure out how Condition.wait() can both
respect a timeout parameter and the promise to hold the lock on return.

Remember, "two different locks". There is a lock on the state
of the condition variable itself, and then there is a lock on which
one actually waits. On both entry to and return from .wait(), you
(the caller) hold the lock on the state of the condition variable,
so you may inspect it and proceed based on the result. In between,
you give up that lock, so that other threads may obtain it and
change the state of the condition variable.

It seems to me that the only way to handle the timeout is to raise an
exception rather then return a value because when you get an exception
you can break the promise of holding the lock.

That *would* be a valid way to implement a timeout -- to return with
the condition variable lock itself no longer held -- but that would
require changing lots of other code structure.