no return value for threading.Condition.wait(timeout)?

[Gabriel Rossetti]

Hello everyone,

I am using threading.Condition.wait(timeout) and was surprised to see
that there is no return value nor an exception when wait() is used w/ a
timeout. How am I supposed to know if it was notified or if it timed out?

cheers,
Gabriel

 

[Carl Banks]

Hello everyone,

I am using threading.Condition.wait(timeout) and was surprised to see
that there is no return value nor an exception when wait() is used w/ a
timeout. How am I supposed to know if it was notified or if it timed out?

That's a good question. Condition.wait seems to violate an invariant
if it can return without having acquired the underlying lock. And if
it does the least it could do is to return a status so you know if you
have to skeddadle.

Workaround is to use the _is_owned() method of Condition to see if you
are the owner of the condition, but this method is undocumented.

cond = Condition()
....
cond.acquire()
while not ok_to_proceed():
cond.wait()
if not cond._is_owned():
# must've timed out
raise TimeOutException
operate()
cond.release()


Carl Banks

 

[Maxim Khitrov]

That's a good question.  Condition.wait seems to violate an invariant
if it can return without having acquired the underlying lock.  And if
it does the least it could do is to return a status so you know if you
have to skeddadle.

Workaround is to use the _is_owned() method of Condition to see if you
are the owner of the condition, but this method is undocumented.

cond = Condition()
...
cond.acquire()
while not ok_to_proceed():
   cond.wait()
   if not cond._is_owned():
       # must've timed out
       raise TimeOutException
operate()
cond.release()

You will always be the owner of the condition whether it times out or
your thread is notified. This doesn't solve the problem. As an aside,
the implementation of wait() is rather poor in any case; the sleep
function should not be used for the purpose of waiting for an event.

Depending on what platform you are using, I can offer two solutions.
The simplest one is to subclass Condition and override the wait
method. Just copy the code as it is, but modify the end of the
function to look like this:

if not gotit:
if __debug__:
self._note("%s.wait(%s): timed out", self, timeout)
try:
self.__waiters.remove(waiter)
except ValueError:
pass
return False
else:
if __debug__:
self._note("%s.wait(%s): got it", self, timeout)
return True

The two return statements will tell you whether you timed out ("if not
gotit" condition), or were notified ("else").

The better way to solve this and the other problem that I mentioned is
to use native OS events. This is what I had to do for a recent
project. I used pywin32 extension to completely rewrite Event,
Condition, and Timer classes to use a true event-based approach. In
the process of the rewrite I also modified some functionality, such as
returning True of False from wait().

Unfortunately, I'm not able to post this code right now, though I can
try and get permission to do so a bit later. If you are writing
software for Windows, then simply take a look at win32event module.
CreateEvent, SetEvent, ResetEvent, WaitForSingleObject are all the
Windows functions that you need to reimplement threading.Event. Using
the new event class you can implement a proper Condition.wait() method
(hint: use new events rather than locks to wait for notification).

If you are on Linux or BSD, I can't help you. I'm sure that there is
some equivalent functionality that you may be able to access using
ctypes. Exactly how to do it, however, is not something that I can
help you with right now.

- Max

 

[Piet van Oostrum]

GR> Hello everyone,
GR> I am using threading.Condition.wait(timeout) and was surprised to see that
GR> there is no return value nor an exception when wait() is used w/ a timeout.
GR> How am I supposed to know if it was notified or if it timed out?

Normally you wouldn't have to know. The logic of your program should be
such that you wait until a certain condition is satisfied. After each
wait you usually check that condition anyway, like:

while (some_condition_satisfied):
cond_var.wait()

The only exception could be if there is a 1-1 relation between a single
waiter and a single notifier. I that case you can usually use if instead
of while (but it does depend of the logic of the notifier).

So in case of using a timeout you can check the desired condition after
the wait and decide what to do. If the condition was not satisfied you
probably want to do something special, like error handling, or do
something else first and then retry the wait.

There could be situations where this doesn't work, however. For example
if you wait until a queue is not empty, and the notify is done when
something is inserted in the queue, another thread may sneak in while
you are waiting and snatch the inserted item just before your thread
continues after the wait. In that case you can't distinguish between a
timeout and a snatcher.

So it all depends on what your use case is.

(Java's wait doesn't return anything either.)

 

[Carl Banks]

You will always be the owner of the condition whether it times out or
your thread is notified. This doesn't solve the problem.

You are correct, I misread the code. So scratch what I said.


Carl Banks

 

[Aahz]

I am using threading.Condition.wait(timeout) and was surprised to see
that there is no return value nor an exception when wait() is used w/ a
timeout. How am I supposed to know if it was notified or if it timed out?

What are you trying to do? Maybe your purpose would be better suited to
using a Queue? (Yes, Queue is a hammer in search of a nail, but most
threading problems are easier if you treat them as nails.)

 

[Gabriel Rossetti]

You will always be the owner of the condition whether it times out or
your thread is notified. This doesn't solve the problem. As an aside,
the implementation of wait() is rather poor in any case; the sleep
function should not be used for the purpose of waiting for an event.

Depending on what platform you are using, I can offer two solutions.
The simplest one is to subclass Condition and override the wait
method. Just copy the code as it is, but modify the end of the
function to look like this:

if not gotit:
if __debug__:
self._note("%s.wait(%s): timed out", self, timeout)
try:
self.__waiters.remove(waiter)
except ValueError:
pass
return False
else:
if __debug__:
self._note("%s.wait(%s): got it", self, timeout)
return True

The two return statements will tell you whether you timed out ("if not
gotit" condition), or were notified ("else").

The better way to solve this and the other problem that I mentioned is
to use native OS events. This is what I had to do for a recent
project. I used pywin32 extension to completely rewrite Event,
Condition, and Timer classes to use a true event-based approach. In
the process of the rewrite I also modified some functionality, such as
returning True of False from wait().

Unfortunately, I'm not able to post this code right now, though I can
try and get permission to do so a bit later. If you are writing
software for Windows, then simply take a look at win32event module.
CreateEvent, SetEvent, ResetEvent, WaitForSingleObject are all the
Windows functions that you need to reimplement threading.Event. Using
the new event class you can implement a proper Condition.wait() method
(hint: use new events rather than locks to wait for notification).

If you are on Linux or BSD, I can't help you. I'm sure that there is
some equivalent functionality that you may be able to access using
ctypes. Exactly how to do it, however, is not something that I can
help you with right now.

- Max

Thank you, I will try that. I am coding for Linux, Mac & Windows (and
any POSIX platform, but right now I support only those three).

Gabriel

 

[Gabriel Rossetti]

Normally you wouldn't have to know. The logic of your program should be
such that you wait until a certain condition is satisfied. After each
wait you usually check that condition anyway, like:

while (some_condition_satisfied):
cond_var.wait()

The only exception could be if there is a 1-1 relation between a single
waiter and a single notifier. I that case you can usually use if instead
of while (but it does depend of the logic of the notifier).

I have a 1-1 relation, I have a thread reading msgs from a network
socket and a method waiting for an answer to arrive (see my thread
"Threading.Condition problem"). I would like to be able to have a
timeout as to not block forever, so the idea is to check if I returned
because of a timeout or not.

 

[Piet van Oostrum]

GR> I have a 1-1 relation, I have a thread reading msgs from a network socket
GR> and a method waiting for an answer to arrive (see my thread
GR> "Threading.Condition problem"). I would like to be able to have a timeout
GR> as to not block forever, so the idea is to check if I returned because of a
GR> timeout or not.

So that case is easy I think. After the wait just check if the answer
has arrived.

 

[Gabriel Rossetti]

So that case is easy I think. After the wait just check if the answer
has arrived.

The problem is other answers could arrive that are not the one I want.
Maybe I could check if I got the answer, if not then check how much time
has elapsed, and if it's grater or equal to the timeout consider it a
timeout. I still think a return value would be better though.

 

[Piet van Oostrum]

GR> I have a 1-1 relation, I have a thread reading msgs from a network socket
GR> and a method waiting for an answer to arrive (see my thread
GR> "Threading.Condition problem"). I would like to be able to have a timeout
GR> as to not block forever, so the idea is to check if I returned because of a
GR> timeout or not.
GR> The problem is other answers could arrive that are not the one I
GR> want. Maybe I could check if I got the answer, if not then check
GR> how much time has elapsed, and if it's grater or equal to the
GR> timeout consider it a timeout. I still think a return value would
GR> be better though.

The wait will not get a return value in Python. There was an issue
posted in the tracker to request this in 2005. Recently (after 4 years)
it was rejected by the BDFL. See http://bugs.python.org/issue1175933

OK, your use case might be a bit more difficult, but I still think it
can be solved without a return value. I can't tell exactly how because
of lack of information.

So (as I already mentioned in one of my previous messages) being
notified doesn't imply that everything is OK. Now suppose that your CV
is notified one microsecond before it would time out but the required
answer did not arrive. What's the difference with a real timeout except
for that microsecond?

Reimplementing Condition yourself is not to be recommended I think.
Using these synchronisation primitives is hard enough let alone
implementing them.

 

[Piet van Oostrum]

I should add that *if the wait times out* it still could be the case
that your answer has arrived but that the notify occured a microsecond
after the timeout. So knowing if the timeout occurred really doesn't
tell you much. It's just a way to prevent infinite waiting.