Make a small function thread safe

[Brad Tilley]

Hey guys,

I have a C++ function that I'd like to replicate (as closely as
possible) in Python. Here's an example:

107 void increment_counter( unsigned int& counter )
108 {
109 boost::mutex::scoped_lock lock( counter_lock );
110 ++counter;
111 }

A thread locks the function on entrance and then releases it on exit.
What is the equivalent way to do this in Python?

Many thanks!

Brad

 

[Tim Wintle]

107 void increment_counter( unsigned int& counter )
108 {
109 boost::mutex::scoped_lock lock( counter_lock );
110 ++counter;
111 }

with counter_lock:
counter += 1


.... where counter_lock is a threading.Lock instance.

(see docs for the threading module)

Tim

 

[Tim Wintle]

So something like this then:

import threading

shared_container = []
lock = threading.Lock()

class thread_example( threading.Thread ):

def __init__( self ):
threading.Thread.__init__ (self)

def run(t):
lock
shared_container.append(t.name)

should be:
def run(t):
with lock:
shared_container.append(t.name)

(or lock.acquire() and lock.release() as you mentioned)

# main

threads = []
for i in xrange(10):
thread = thread_example()
threads.append(thread)

for thread in threads:
thread.start()

you'll either need to lock again here, or join each thread:

for thread in threads:
thread.join()

for item in shared_container:
print item

Tim

 

[Brad Tilley]

should be:
      def run(t):
          with lock:
              shared_container.append(t.name)

(or lock.acquire() and lock.release() as you mentioned)

Thanks Tim. The with statement is closer to the C++ code (IMO) more so
than the explicit acquire() and release() so I'll use that approach. I
appreciate your advice.

Brad

 

[Lie Ryan]

Or perhaps run should look like this instead:

def run(t):
lock.acquire()
shared_container.append(t.name <http://t.name>)
lock.release()

That seems a bit barbaric to me, not sure.

change that to:

def run(t):
with lock:
shared_container.append(t.name <http://t.name>)


the `with-statement` will call lock.acquire() and lock.release().

 

[John Nagle]

change that to:

def run(t):
with lock:
shared_container.append(t.name <http://t.name>)


the `with-statement` will call lock.acquire() and lock.release().

And, importantly, if you leave the "with" via an exception, the
lock will be unlocked.

John Nagle

 

[RangerElf]

Which is why the original .acquire() ... .release() idiom was wrong, this would better express the intent:

try:
lock.acquire()
shared_container.append(...)
finally:
lock.release()

But like everyone mentions, the with statement takes care of all that in a more readable and compact fashion.

 

[Ian Kelly]

No - this is very bad. The lock must be acquired outside the try: -
otherwise if an exception is thrown while acquiring, you will try to release
a lock that you have not acquired.

Which again is why using with is a much better option - you can't make this
kind of mistake.

Well, not unless you make the same mistake in the context manager itself.

from contextlib import contextmanager

@contextmanager
def bad_context_manager(lock):
try:
lock.acquire()
yield
finally:
lock.release()

class Lock(object):
def __init__(self):
self.is_locked = False
def acquire(self):
assert False
def release(self):
assert self.is_locked, "Tried to release lock without acquiring it"

with bad_context_manager(Lock()):
pass

Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__
return self.gen.next()
File "<stdin>", line 7, in bad_context_manager
File "<stdin>", line 7, in release
AssertionError: Tried to release lock without acquiring it


Perhaps a (small) reason to avoid the contextmanager decorator and
implement __enter__ and __exit__ instead.

 

[ting]

A thread locks the function on entrance and then releases it on exit.
What is the equivalent way to do this in Python?

I'm not sure if this applies in your case, but much of the time, you
can use thread local storage, rather thread locking, in order to make
your code thread safe. You tend to run into far less bottleneck and
race condition issues by using thread local storage rather than thread
locking, whenever possible.