Synchronization methodology

[Chris Ashurst]

Hi, I'm coming in from a despised Java background, and I'm having some
trouble wrapping my head around sharing an object between multiple
instances of a single class (in simpler terms, I would say imagine a
simple chat server that has to share a list of connected users to each
instance of a connected user).

Usually, I would have a synchronized list instantiated inside each
instance of a client class, which would do the trick, but since there's
no synchronization in Python, I'm stuck staring at little tests
involving a standalone non-threaded class instance that holds the list
of users, and each connected user being passed this instance to be
"synchronized".

Now, whilst this *works*, it just doesn't feel right, and I would
appreciate it if anyone has any more Pythonic expressions of this paradigm.

Note that I have looked at Twisted (and I use it for other stuff), but
I'm wanting to do things at a lower level (because I'm masochistic), and
I feel like all the fun has been sucked out of programming by basically
telling Twisted what I want and have it do it for me invisibly.

Thanks!


~Chris

 

[Marc 'BlackJack' Rintsch]

Hi, I'm coming in from a despised Java background, and I'm having some
trouble wrapping my head around sharing an object between multiple
instances of a single class (in simpler terms, I would say imagine a
simple chat server that has to share a list of connected users to each
instance of a connected user).

Usually, I would have a synchronized list instantiated inside each
instance of a client class, which would do the trick, but since there's
no synchronization in Python, I'm stuck staring at little tests
involving a standalone non-threaded class instance that holds the list
of users, and each connected user being passed this instance to be
"synchronized".

There's no ``synchronized`` keyword, but of course you can synchronize
methods. You have to attach an `threading.RLock` to the objects you want
to synchronize on and use its `aquire()` and `release()` methods in places
where you used ``synchronized (someObject) { ... }`` in Java.

synchronized foo() { /* code */ }

is the same as

foo() {
synchronized (this) { /* code */ }
}

in Java. And that's basically:

def foo(self):
self.lock.aquire()
# code
self.lock.release()

You may want to make sure the lock will be released in case of an
exception:

def foo(self):
self.lock.aquire()
try:
pass # code
finally:
self.lock.release()

Ciao,
Marc 'BlackJack' Rintsch

 

[sjdevnull]

Hi, I'm coming in from a despised Java background

Consider strongly the fact that Python supports multiple process
solutions well, so you're not stuck having to use multithreaded
solutions in every circumstance (but can still use them when necessary).

 

[Paul Rubin]

Hi, I'm coming in from a despised Java background, and I'm having some
trouble wrapping my head around sharing an object between multiple
instances of a single class (in simpler terms, I would say imagine a
simple chat server that has to share a list of connected users to each
instance of a connected user).

If you're doing this with threads, one common Pythonic style is
generally to have the threads communicate through queues (see docs for
the Queue module). This is sort of a cheap version of the CSP
(communicating sequential processes) approach. You might have a
service thread that takes care of those shared objects, reading
requests from a queue (untested):

# All client threads write their requests to this queue, and
# the service loop reads from it
service_queue = Queue()

# make one of these in each client thread
class Request(Queue.Queue):
def __call__(self, reply_queue, func, *args, **kwargs):
service_queue.put((self, func, args, kwargs))
return self.get()

def service_loop():
while True:
client, func, args, kwargs = service_queue.get()
client.put (func(*args, **kwargs))

Threading.Thread(target=service_loop).start()

Then in your connected user thread you could say:

# set up a reply queue for this user's thread
request = Request()
...

# add the user to the connected user list, with timestamp and remote IP
status = request(user_list.append, self, time(), remote_ip=whatever)

The last line would drop a request on the service queue to perform the
function call

user_list.append(time(), remote_ip=whatever)

The service queue thread would retrieve the request, call the
function, and pass the result back through the reply queue. The idea
is you'd handle all operations on user_list in the service thread, so
access is completely serialized using the queues. You can see that
you can easily construct any function call that then gets executed in
the other thread, without having to use a bunch of boilerplate at each
request. You have to be a bit careful to not use long-running
requests that could block other threads from getting their requests
serviced. Use additional threads if you need such requests.

You sometimes end up creating more threads than necessary this way but
you tend to not have many synchronization problems if you stick with
this style.

 

[Paul Rubin]

# add the user to the connected user list, with timestamp and remote IP
status = request(user_list.append, self, time(), remote_ip=whatever)

Editing error, ignore the "self," arg up there. Of course there may be
other mistakes too, I didn't test any of that code ;-)

 

[Duncan Booth]

You may want to make sure the lock will be released in case of an
exception:

def foo(self):
self.lock.aquire()
try:
pass # code
finally:
self.lock.release()

In Python 2.5 this can also be written more succintly using the with
statement so foo becomes:

from __future__ import with_statement
....

def foo(self):
with self.lock:
pass # insert your code here


or write a decorator.

 

[Jeremy Dillworth]

Hi Chris,

Have you looked at the mutex module?

Jeremy