dictionaries and threads

[Gary Robinson]

Hi,

I know the Global Interpreter Lock ensures that only one python thread
has access to the interpreter at a time, which prevents a lot of
situations where one thread might step on another's toes.

But I'd like to ask about a specific situation just to be sure I
understand things relative to some code I'm writing.

I've got a dictionary which is accessed by several threads at the same
time (that is, to the extent that the GIL allows). The thing is,
however, no two threads will ever be accessing the same dictionary
items at the same time. In fact the thread's ID from thread.get_ident()
is the key to the dictionary; a thread only modifies items
corresponding to its own thread ID. A thread will be adding an item
with its ID when it's created, and deleting it before it exits, and
modifying the item's value in the meantime.

As far as I can tell, if the Python bytecodes that cause dictionary
modifications are atomic, then there should be no problem. But I don't
know that they are because I haven't looked at the bytecodes.

Any feedback on this would be appreciated. For various reasons, we're
still using Python 2.3 for the time being.

Gary

--

Gary Robinson
CTO
Emergent Music, LLC
(e-mail address removed)
207-942-3463
Company: http://www.goombah.com
Blog: http://www.garyrobinson.net

 

[Paul Rubin]

As far as I can tell, if the Python bytecodes that cause dictionary
modifications are atomic, then there should be no problem. But I don't
know that they are because I haven't looked at the bytecodes.

Depending on behavior like that is asking for trouble. If it doesn't
kill your app's performance, put some kind of locks around the
dictionary and/or direct all access to the directory through a single
thread. The favorite Python idiom for that seems to be the Queue
module; you'd set up a work queue and a result queue to communicate
with the thread controlling the dictionary.

If your app absolutely can't stand that, look to a more fundamental
solution, maybe something like POSH (poshmodule.sf.net).