Question regarding thread atomicity

[Jon Ribbens]

If I have the following line of Python code:

self.foo.bar += 1

where 'foo' is an instance of an object and 'bar' is an integer,
is that thread-safe? i.e. will the GIL mean that it is reliable,
or do I need to manually add locking around it?

 

[Peter Hansen]

If I have the following line of Python code:

self.foo.bar += 1

where 'foo' is an instance of an object and 'bar' is an integer,
is that thread-safe? i.e. will the GIL mean that it is reliable,
or do I need to manually add locking around it?

Provided another thread is not trying to change the binding
for "self.foo", and provided that there is no magic __setattr__
or __getattr__ dynamically finding the "bar" attribute (or the
"foo" one, for that matter), and depending on what you mean by
"thread-safe", then things should work safely.

Checking the output of "dis" is a good way to check this kind of
thing a little more:
.... def func(self):
.... self.foo.bar += 1
.... 3 0 LOAD_FAST 0 (self)
3 LOAD_ATTR 1 (foo)
6 DUP_TOP
7 LOAD_ATTR 2 (bar)
10 LOAD_CONST 1 (1)
13 INPLACE_ADD
14 ROT_TWO
15 STORE_ATTR 2 (bar)
18 LOAD_CONST 0 (None)
21 RETURN_VALUE

Note that the STORE_ATTR part is the key part that could make the
program crash if it were not atomic, but note that there are a
few different ways that you could get wrong results if you actually
had the above code executing from several threads simultaneously,
affecting the same object.

The basic failure mode would be that two threads both retrieve
the same current value for "bar" and add one to it "INPLACE", then
store it back again: the result is that although two attempts to
add have been made, the value increases by only one. Python won't
crash, which is why this all depends on your definition of "thread-safe",
but it likely won't do what you probably want it to do.

-Peter

 

[Jon Ribbens]

The basic failure mode would be that two threads both retrieve
the same current value for "bar" and add one to it "INPLACE", then
store it back again: the result is that although two attempts to
add have been made, the value increases by only one. Python won't
crash, which is why this all depends on your definition of "thread-safe",
but it likely won't do what you probably want it to do.

Ah. This violates my definition of "thread-safe" ;-) If it doesn't go
up by exactly one every time that line is executed, it isn't doing
what I want it to do. I guess I'll have to go and add a load of manual
locking :-(

 

[Paul Rubin]

Ah. This violates my definition of "thread-safe" ;-) If it doesn't go
up by exactly one every time that line is executed, it isn't doing
what I want it to do. I guess I'll have to go and add a load of manual
locking :-(

I have to agree, it looks like a pretty bad bug. The most natural way
I can think of to fix it is to add something like references to the
bytecode interpreter, so that

7 LOAD_ATTR 2 (bar)
10 LOAD_CONST 1 (1)
13 INPLACE_ADD
14 ROT_TWO
15 STORE_ATTR 2 (bar)

would become something like

LOAD_REF 2 (bar) # load a pointer to bar's location
LOAD_CONST 1
INPLACE_ADD_REF

However, apparently there's some implementation reasons to not want to
do it that way.