Writing a thread-safe class

Discussion in 'Python' started by Timothy Madden, Sep 12, 2009.

  1. Hello

    I would like to write a class with methods that can be accessed by many
    threads at the same time.

    For this I have a lock attribute in my class obtained with
    threading.Lock(), in the constructor, and every method begins by
    acquiring the lock and ends by releasing it

    My problem is that the lock is still an attribute of the class, and the
    very expression self.lock in the statement self.lock.acquire() is
    performed before locking, thus not being thread-safe.

    If I am correct self is a dictionary of object attributes, and if
    another thread has the lock and creates a new attribute, or deletes one,
    in the same time with my lookup for self.lock, than the lookup is
    compromised.

    How do people create thread-safe classes in python ?

    Thank you,
    Timothy Madden
     
    Timothy Madden, Sep 12, 2009
    #1
    1. Advertising

  2. Timothy Madden

    Carl Banks Guest

    On Sep 11, 4:26 pm, Timothy Madden <> wrote:
    > Hello
    >
    > I would like to write a class with methods that can be accessed by many
    > threads at the same time.
    >
    > For this I have a lock attribute in my class obtained with
    > threading.Lock(), in the constructor, and every method begins by
    > acquiring the lock and ends by releasing it
    >
    > My problem is that the lock is still an attribute of the class, and the
    > very expression self.lock in the statement self.lock.acquire() is
    > performed before locking, thus not being thread-safe.
    >
    > If I am correct self is a dictionary of object attributes, and if
    > another thread has the lock and creates a new attribute, or deletes one,
    > in the same time with my lookup for self.lock, than the lookup is
    > compromised.


    You are not correct. Dictionary operation (like getting and setting
    items) are atomic and limited to one thread at a time, thus thread-
    safe.

    (In fact, in CPython only one thread can execute Python code at a
    time, in most cases. This is called the Global Interpreter Lock, or
    GIL. If you search this newsgroup for information you will find a LOT
    of discission about it. Other Python implementations such as Jython
    may not use the GIL, but I am quite sure dictionary access is thread-
    safe on those platforms also.)


    > How do people create thread-safe classes in python ?


    I think exactly the way you are doing it. (Some people might use
    threading.RLock instead as a minor safety measure, but threading.Lock
    will suffice.)


    Carl Banks
     
    Carl Banks, Sep 12, 2009
    #2
    1. Advertising

  3. Carl Banks wrote:
    [...]
    >
    > You are not correct. Dictionary operation (like getting and setting
    > items) are atomic and limited to one thread at a time, thus thread-
    > safe.
    >
    > (In fact, in CPython only one thread can execute Python code at a
    > time, in most cases. This is called the Global Interpreter Lock, or

    [...]

    I find that hard to believe, but I will look into it.

    Thank you,
    Timothy Madden
     
    Timothy Madden, Sep 12, 2009
    #3
  4. Timothy Madden

    sturlamolden Guest

    On 12 Sep, 15:54, Timothy Madden <> wrote:

    > I find that hard to believe, but I will look into it.


    Carl Banks is correct.

    There is a mutex called "the global interpreter lock" that takes care
    of this. You can have multiple threads running, but access to the
    Python interpreter is serialized.
     
    sturlamolden, Sep 12, 2009
    #4
  5. Timothy Madden

    News123 Guest

    So what's recommended way for multicore machines?
    Threads will probably only accelerate if the used C libraries are
    releasing the GIL, right?

    What's for example about PIL (Python Imaging library)?



    Assuming, that the C library calls don't releas the GIL


    Shoud I directly use use fork() and some kind of IPC? or are there some
    special well established, recommendable commodity modules aiming for
    rmultiprocessor job distribution?

    So far I have just a single-core machine, but I'll be using a multicore
    machine in the next weeks.

    Then I'll probably find out

    bye

    N




    sturlamolden wrote:
    > On 12 Sep, 15:54, Timothy Madden <> wrote:
    >
    >> I find that hard to believe, but I will look into it.

    >
    > Carl Banks is correct.
    >
    > There is a mutex called "the global interpreter lock" that takes care
    > of this. You can have multiple threads running, but access to the
    > Python interpreter is serialized.
    >
    >
    >
     
    News123, Sep 14, 2009
    #5
  6. Timothy Madden

    MRAB Guest

    News123 wrote:
    > So what's recommended way for multicore machines?
    > Threads will probably only accelerate if the used C libraries are
    > releasing the GIL, right?
    >
    > What's for example about PIL (Python Imaging library)?
    >
    >
    >
    > Assuming, that the C library calls don't releas the GIL
    >
    >
    > Shoud I directly use use fork() and some kind of IPC? or are there some
    > special well established, recommendable commodity modules aiming for
    > rmultiprocessor job distribution?
    >
    > So far I have just a single-core machine, but I'll be using a multicore
    > machine in the next weeks.
    >
    > Then I'll probably find out
    >

    Have a look at the multiprocessing module.

    > bye
    >
    > N
    >
    >
    >
    >
    > sturlamolden wrote:
    >> On 12 Sep, 15:54, Timothy Madden <> wrote:
    >>
    >>> I find that hard to believe, but I will look into it.

    >> Carl Banks is correct.
    >>
    >> There is a mutex called "the global interpreter lock" that takes care
    >> of this. You can have multiple threads running, but access to the
    >> Python interpreter is serialized.
    >>
    >>
    >>
     
    MRAB, Sep 14, 2009
    #6
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Peter Blatt
    Replies:
    0
    Views:
    301
    Peter Blatt
    Aug 11, 2003
  2. Marc Twain
    Replies:
    2
    Views:
    971
    Marc Twain
    Nov 14, 2003
  3. Frank LaRosa
    Replies:
    1
    Views:
    774
    Thomas Weidenfeller
    Dec 9, 2003
  4. Gabriel Rossetti
    Replies:
    0
    Views:
    1,338
    Gabriel Rossetti
    Aug 29, 2008
  5. John Nagle
    Replies:
    5
    Views:
    477
    John Nagle
    Mar 12, 2012
Loading...

Share This Page