Writing a thread-safe class

[Timothy Madden]

Hello

I would like to write a class with methods that can be accessed by many
threads at the same time.

For this I have a lock attribute in my class obtained with
threading.Lock(), in the constructor, and every method begins by
acquiring the lock and ends by releasing it

My problem is that the lock is still an attribute of the class, and the
very expression self.lock in the statement self.lock.acquire() is
performed before locking, thus not being thread-safe.

If I am correct self is a dictionary of object attributes, and if
another thread has the lock and creates a new attribute, or deletes one,
in the same time with my lookup for self.lock, than the lookup is
compromised.

How do people create thread-safe classes in python ?

Thank you,
Timothy Madden

 

[Carl Banks]

Hello

I would like to write a class with methods that can be accessed by many
threads at the same time.

For this I have a lock attribute in my class obtained with
threading.Lock(), in the constructor, and every method begins by
acquiring the lock and ends by releasing it

My problem is that the lock is still an attribute of the class, and the
very expression self.lock in the statement self.lock.acquire() is
performed before locking, thus not being thread-safe.

If I am correct self is a dictionary of object attributes, and if
another thread has the lock and creates a new attribute, or deletes one,
in the same time with my lookup for self.lock, than the lookup is
compromised.

You are not correct. Dictionary operation (like getting and setting
items) are atomic and limited to one thread at a time, thus thread-
safe.

(In fact, in CPython only one thread can execute Python code at a
time, in most cases. This is called the Global Interpreter Lock, or
GIL. If you search this newsgroup for information you will find a LOT
of discission about it. Other Python implementations such as Jython
may not use the GIL, but I am quite sure dictionary access is thread-
safe on those platforms also.)

How do people create thread-safe classes in python ?

I think exactly the way you are doing it. (Some people might use
threading.RLock instead as a minor safety measure, but threading.Lock
will suffice.)


Carl Banks

 

[Timothy Madden]

Carl Banks wrote:
[...]

You are not correct. Dictionary operation (like getting and setting
items) are atomic and limited to one thread at a time, thus thread-
safe.
>
(In fact, in CPython only one thread can execute Python code at a
time, in most cases. This is called the Global Interpreter Lock, or

[...]

I find that hard to believe, but I will look into it.

Thank you,
Timothy Madden

 

[sturlamolden]

I find that hard to believe, but I will look into it.

Carl Banks is correct.

There is a mutex called "the global interpreter lock" that takes care
of this. You can have multiple threads running, but access to the
Python interpreter is serialized.

 

[News123]

So what's recommended way for multicore machines?
Threads will probably only accelerate if the used C libraries are
releasing the GIL, right?

What's for example about PIL (Python Imaging library)?



Assuming, that the C library calls don't releas the GIL


Shoud I directly use use fork() and some kind of IPC? or are there some
special well established, recommendable commodity modules aiming for
rmultiprocessor job distribution?

So far I have just a single-core machine, but I'll be using a multicore
machine in the next weeks.

Then I'll probably find out

bye

N

 

[MRAB]

So what's recommended way for multicore machines?
Threads will probably only accelerate if the used C libraries are
releasing the GIL, right?

What's for example about PIL (Python Imaging library)?



Assuming, that the C library calls don't releas the GIL


Shoud I directly use use fork() and some kind of IPC? or are there some
special well established, recommendable commodity modules aiming for
rmultiprocessor job distribution?

So far I have just a single-core machine, but I'll be using a multicore
machine in the next weeks.

Then I'll probably find out

Have a look at the multiprocessing module.