Immutable object thread-safety

[k3xji]

Hi all,

This will probably be a long question/short answer, sorry, but I have
wandered net about the subject and really feel cannot find just enough
information.I want to ask my question by giving an example to
explicitly express my understanding which may be also wrong:

So, let's have a string in a module like:
st = 'IAmAstring'
My understanding: Python allocates a string object and binds the st to
that string's reference.

and let's have two threads simply doing following operations on the st
string:

Thread A:
st += 'ThreadAWasHere'

Thread B
st = 'ThreadBWasHere'

Let's think about the following situation:
Thread A reads the st value, and it is currently binded to an object
which is holding the string 'IAmAString'.
So let Thread B starts execution at that point. Note that Thread A
does not add the new string to object, it only reads st object. So,
then Thread B just writes to st, and all of a sudden st is binded to
something which is ThreadBWasHere. And then let's continue Thread A,
it already readed st's reference which is a reference to 'IamAString'.

So, finally if the execution flow is like above, we will have
'IAmAStringThreadAWasHere'. I am assuming this should not be the case.
We should have 'ThreadBWasHereThreadAWasHere' in the final string,
because we have just consumed the code in ThreadB.

Also, the other question is the operation st = 'ThreadBWasHere' is
atomic? I mean, if Python does not guarantee if an immutable object
assignment is atomic, then how can we say that the object is thread-
safe? So, if it is an atomic operation, which operators are atomic,
means only assignment'='? How about other operators like +=, or -
=..etc?

I am confused about which data structure to rely on thread-safety, or
operator in Python?

Regards,

 

[Laszlo Nagy]

Also, the other question is the operation st = 'ThreadBWasHere' is
atomic?

I think this is the same question. And I believe it is not atomic,
because it is actually rebinding a name. Consider this:

a,b = b,a

This will rebind both a and b. In order to be correct, it MUST happen in
two phases: first calculate the right side, then do the rebind to the
names on the left side. The expression on the right side can be anything
that executes for a long time, and can even rebind 'a' and 'b' several
times, and will probably be paused by other threads etc. So the
assignment above cannot be atomic.

I strongly feel that if such an assignment is not atomic for "a,b = b,a"
then int wont be atomic for "a+=b" or eveb "a=b" or any other
assignment. However, I can be completely wrong.

I mean, if Python does not guarantee if an immutable object
assignment is atomic, then how can we say that the object is thread-
safe?

An assigment rebinds name to an object. It is assignment to a name.
Assigning immutable and mutable objects to names are not specially
distinguished.

So, if it is an atomic operation, which operators are atomic,
means only assignment'='?

I don't think that '=' operator is atomic, see above.

I am confused about which data structure to rely on thread-safety, or
operator in Python?

The immutable object itself will never change state, so it is thread
safe. The problem is not with the object, but the dictionary which holds
the name 'b' and 'a' and 'st' in your example. It is mutable, and so
must be protected in a threaded application.

Best,

Laszlo

 

[Laszlo Nagy]

This will rebind both a and b. In order to be correct, it MUST happen
in two phases: first calculate the right side, then do the rebind to
the names on the left side.

"rebind to the names" -> "rebind the names found on the left side, to
the objects calculated from the expressions on the right side".

I strongly feel that if such an assignment is not atomic for "a,b =
b,a" then int wont be atomic for "a+=b" or eveb "a=b" or any other
assignment.

"int" -> "it"
"eveb" -> "even"

The immutable object itself will never change state, so it is thread
safe. The problem is not with the object, but the dictionary which
holds the name 'b' and 'a' and 'st' in your example.

"The problem is not with the object" -> "The problem is not with THAT
object".

Wanted to answer too quickly. :-( Sorry.

 

[Alcari The Mad]

I think this is the same question. And I believe it is not atomic,
because it is actually rebinding a name. Consider this:

a,b = b,a

This will rebind both a and b. In order to be correct, it MUST happen in
two phases: first calculate the right side, then do the rebind to the
names on the left side. The expression on the right side can be anything
that executes for a long time, and can even rebind 'a' and 'b' several
times, and will probably be paused by other threads etc. So the
assignment above cannot be atomic.

You are correct.
a,b = b,a
yields the bytecode:

1 0 LOAD_NAME 0 (b)
3 LOAD_NAME 1 (a)
6 ROT_TWO
7 STORE_NAME 1 (a)
10 STORE_NAME 0 (b)

Which is most definitely not atomic.

I strongly feel that if such an assignment is not atomic for "a,b = b,a"
then int wont be atomic for "a+=b" or eveb "a=b" or any other
assignment. However, I can be completely wrong.

'a = b' requires two bytecodes:

1 0 LOAD_NAME 0 (b)
3 STORE_NAME 1 (a)

An assigment rebinds name to an object. It is assignment to a name.
Assigning immutable and mutable objects to names are not specially
distinguished.
I don't think that '=' operator is atomic, see above.
The immutable object itself will never change state, so it is thread
safe. The problem is not with the object, but the dictionary which holds
the name 'b' and 'a' and 'st' in your example. It is mutable, and so
must be protected in a threaded application.

I would suggest reading up on using the 'with' statement and
thread.allocate_lock().

 

[Gabriel Genellina]

En Sun, 26 Oct 2008 23:25:09 -0200, Alcari The Mad

All of the builtin functions(which are implemented in C, like len()) are
atomic(but assigning their output to a value may not be).

You can't count on the builtins being atomic. len(x) executes
type(x).__len__ if such method exists, which may execute arbitrary Python
code, even trigger the garbage collector and run absolutely unrelated
things.
See this effbot page for discussion [1] - but in general, since the
language reference doesn't specify whether an operation is atomic or not,
you should not count on it. Use a lock when required.

[1]
http://effbot.org/pyfaq/what-kinds-of-global-value-mutation-are-thread-safe.htm