Threads and variable assignment

[Gregory Bond]

I've had a solid hunt through the (2.3) documentation but it seems
silent on this issue.

I have an problem that would naturally run as 2 threads: One monitors a
bunch of asyncrhonous external state and decides if things are "good" or
"bad". The second thread processes data, and the processing depends on
the "good" or "bad" state at the time the data is processed.

Sort of like this:

Thread 1:

global isgood
while 1:
wait_for_state_change()
if new_state_is_good():
isgood = 1
else:
isgood = 0

Thread 2:

s = socket(....)
s.connect(...)
f = s.makefile()
while 1:
l = f.readline()
if isgood:
print >> goodfile, l
else:
print >> badfile, l

What guarantees (if any!) does Python make about the thread safety of
this construct? Is it possible for thread 2 to get an undefined
variable if it somehow catches the microsecond when isgood is being
updated by thread 1?

 

[elbertlev]

Theoretically you have to use lock, while accesing the isgood instance,
but... practically noting bad can happen IN THIS PARTICULAR CASE, if
you don't. Python uses Global Interpreter Lock. In other words only
one thread is running at any particular moment. Thread scheduling is
preemptive, but "atomic" actions are not interrupted. Bollean asigment
is an atomic action.

WELCOME TO THE POWER OF MULTITHREADING

 

[David M. Cooke]

I've had a solid hunt through the (2.3) documentation but it seems
silent on this issue.

I have an problem that would naturally run as 2 threads: One monitors
a bunch of asyncrhonous external state and decides if things are
"good" or "bad". The second thread processes data, and the processing
depends on the "good" or "bad" state at the time the data is processed.

Sort of like this:

Thread 1:

global isgood
while 1:
wait_for_state_change()
if new_state_is_good():
isgood = 1
else:
isgood = 0

Thread 2:

s = socket(....)
s.connect(...)
f = s.makefile()
while 1:
l = f.readline()
if isgood:
print >> goodfile, l
else:
print >> badfile, l

You said that the processing depends on the good or bad state at the
time the data is processed: I don't know how finely-grained your state
changes will be in thread 1, but it doesn't seem that thread 2 would
notice at the right time. If the socket blocks reading a line, the
state could change i

What guarantees (if any!) does Python make about the thread safety of
this construct? Is it possible for thread 2 to get an undefined
variable if it somehow catches the microsecond when isgood is being
updated by thread 1?

It won't be undefined, but it's possible that (in thread 1)
between the "if new_state_is_good()" and the setting of isgood that
thread 2 will execute, so if new_state_is_good() was false, then it
could still write the line to the goodfile.

It really depends on how often you have state changes, how often you
get (full) lines on your socket, and how much you care that the
correct line be logged to the right file.

If you needed this to be robust, I'd either:

- Try to rewrite wait_for_status_change()/new_state_is_good() to be
asynchronous, particularly if wait_for_status_change() is blocking
on some file or socket object. This way you could hook it into
asynchat/asyncore or Twisted without any threads.

- Or, if you need to use threads, use a Queue.Queue object where
timestamps w/ state changes are pushed on in thread 1, and popped
off and analysed before logging in thread 2. (Or something; this
just popped in my head.)