Make a small function thread safe

Discussion in 'Python' started by Brad Tilley, Dec 16, 2011.

  1. Brad Tilley

    Brad Tilley Guest

    Hey guys,

    I have a C++ function that I'd like to replicate (as closely as
    possible) in Python. Here's an example:

    107 void increment_counter( unsigned int& counter )
    108 {
    109 boost::mutex::scoped_lock lock( counter_lock );
    110 ++counter;
    111 }

    A thread locks the function on entrance and then releases it on exit.
    What is the equivalent way to do this in Python?

    Many thanks!

    Brad
     
    Brad Tilley, Dec 16, 2011
    #1
    1. Advertising

  2. Brad Tilley

    Tim Wintle Guest

    On Fri, 2011-12-16 at 05:21 -0800, Brad Tilley wrote:
    > 107 void increment_counter( unsigned int& counter )
    > 108 {
    > 109 boost::mutex::scoped_lock lock( counter_lock );
    > 110 ++counter;
    > 111 }



    with counter_lock:
    counter += 1


    .... where counter_lock is a threading.Lock instance.

    (see docs for the threading module)

    Tim
     
    Tim Wintle, Dec 16, 2011
    #2
    1. Advertising

  3. Brad Tilley

    Tim Wintle Guest

    On Fri, 2011-12-16 at 09:24 -0500, Brad Tilley wrote:
    > So something like this then:
    >
    > import threading
    >
    > shared_container = []
    > lock = threading.Lock()
    >
    > class thread_example( threading.Thread ):
    >
    > def __init__( self ):
    > threading.Thread.__init__ (self)
    >
    > def run(t):
    > lock
    > shared_container.append(t.name)


    should be:
    def run(t):
    with lock:
    shared_container.append(t.name)

    (or lock.acquire() and lock.release() as you mentioned)

    > # main
    >
    > threads = []
    > for i in xrange(10):
    > thread = thread_example()
    > threads.append(thread)
    >
    > for thread in threads:
    > thread.start()


    you'll either need to lock again here, or join each thread:

    for thread in threads:
    thread.join()

    > for item in shared_container:
    > print item


    Tim
     
    Tim Wintle, Dec 16, 2011
    #3
  4. Brad Tilley

    Brad Tilley Guest

    On Dec 16, 9:36 am, Tim Wintle <> wrote:

    > should be:
    >       def run(t):
    >           with lock:
    >               shared_container.append(t.name)
    >
    > (or lock.acquire() and lock.release() as you mentioned)



    Thanks Tim. The with statement is closer to the C++ code (IMO) more so
    than the explicit acquire() and release() so I'll use that approach. I
    appreciate your advice.

    Brad
     
    Brad Tilley, Dec 16, 2011
    #4
  5. Brad Tilley

    Lie Ryan Guest

    On 12/17/2011 01:30 AM, Brad Tilley wrote:
    > Or perhaps run should look like this instead:
    >
    > def run(t):
    > lock.acquire()
    > shared_container.append(t.name <http://t.name>)
    > lock.release()
    >
    > That seems a bit barbaric to me, not sure.


    change that to:

    def run(t):
    with lock:
    shared_container.append(t.name <http://t.name>)


    the `with-statement` will call lock.acquire() and lock.release().
     
    Lie Ryan, Dec 16, 2011
    #5
  6. Brad Tilley

    John Nagle Guest

    On 12/16/2011 2:08 PM, Lie Ryan wrote:
    > On 12/17/2011 01:30 AM, Brad Tilley wrote:
    >> Or perhaps run should look like this instead:
    >>
    >> def run(t):
    >> lock.acquire()
    >> shared_container.append(t.name <http://t.name>)
    >> lock.release()
    >>
    >> That seems a bit barbaric to me, not sure.

    >
    > change that to:
    >
    > def run(t):
    > with lock:
    > shared_container.append(t.name <http://t.name>)
    >
    >
    > the `with-statement` will call lock.acquire() and lock.release().
    >


    And, importantly, if you leave the "with" via an exception, the
    lock will be unlocked.

    John Nagle
     
    John Nagle, Dec 17, 2011
    #6
  7. Brad Tilley

    RangerElf Guest

    Which is why the original .acquire() ... .release() idiom was wrong, this would better express the intent:

    try:
    lock.acquire()
    shared_container.append(...)
    finally:
    lock.release()

    But like everyone mentions, the with statement takes care of all that in a more readable and compact fashion.
     
    RangerElf, Dec 18, 2011
    #7
  8. Brad Tilley

    Ian Kelly Guest

    On Sun, Dec 18, 2011 at 6:27 PM, Tim Delaney
    <> wrote:
    > On 18 December 2011 19:52, RangerElf <> wrote:
    >>
    >> Which is why the original .acquire() ... .release() idiom was wrong, this
    >> would better express the intent:
    >>
    >> try:
    >>  lock.acquire()
    >>  shared_container.append(...)
    >> finally:
    >>  lock.release()

    >
    >
    > No - this is very bad. The lock must be acquired outside the try: -
    > otherwise if an exception is thrown while acquiring, you will try to release
    > a lock that you have not acquired.
    >
    > Which again is why using with is a much better option - you can't make this
    > kind of mistake.


    Well, not unless you make the same mistake in the context manager itself.

    from contextlib import contextmanager

    @contextmanager
    def bad_context_manager(lock):
    try:
    lock.acquire()
    yield
    finally:
    lock.release()

    class Lock(object):
    def __init__(self):
    self.is_locked = False
    def acquire(self):
    assert False
    def release(self):
    assert self.is_locked, "Tried to release lock without acquiring it"

    with bad_context_manager(Lock()):
    pass

    Traceback (most recent call last):
    File "<stdin>", line 1, in <module>
    File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__
    return self.gen.next()
    File "<stdin>", line 7, in bad_context_manager
    File "<stdin>", line 7, in release
    AssertionError: Tried to release lock without acquiring it


    Perhaps a (small) reason to avoid the contextmanager decorator and
    implement __enter__ and __exit__ instead.
     
    Ian Kelly, Dec 19, 2011
    #8
  9. Brad Tilley

    Guest

    On Dec 16, 8:21 am, Brad Tilley <> wrote:
    > A thread locks the function on entrance and then releases it on exit.
    > What is the equivalent way to do this in Python?


    I'm not sure if this applies in your case, but much of the time, you
    can use thread local storage, rather thread locking, in order to make
    your code thread safe. You tend to run into far less bottleneck and
    race condition issues by using thread local storage rather than thread
    locking, whenever possible.
    --
    // T.Hsu
     
    , Dec 19, 2011
    #9
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Francis Reed
    Replies:
    2
    Views:
    727
    Erik Funkenbusch
    Apr 8, 2006
  2. Gabriel Rossetti
    Replies:
    0
    Views:
    1,326
    Gabriel Rossetti
    Aug 29, 2008
  3. Ian Collins

    Re: How to Easily Make a Function Thread-Safe

    Ian Collins, Jan 12, 2009, in forum: C Programming
    Replies:
    4
    Views:
    885
    Phil Carmody
    Jan 13, 2009
  4. CBFalconer

    Re: How to Easily Make a Function Thread-Safe

    CBFalconer, Jan 12, 2009, in forum: C Programming
    Replies:
    14
    Views:
    637
    Antoninus Twink
    Jan 13, 2009
  5. John Nagle
    Replies:
    5
    Views:
    473
    John Nagle
    Mar 12, 2012
Loading...

Share This Page