Re: changing dicts in a threaded environment ?

Discussion in 'Python' started by Terry Reedy, Nov 27, 2012.

  1. Terry Reedy

    Terry Reedy Guest

    On 11/27/2012 7:53 AM, Bart Thate wrote:

    [Answers based on reading without thread experience.]

    > i use python3 now and i need to be able to remove elements from a dict
    > in a thread safe manner.

    Essentially no change from py2.

    > kinda like a Queue.Queue thing but then in a dict, so i can pass arond
    > my dict based objects as parameters arond without having to wonder if
    > it gets properly locked.

    As I understand it, dicts do not get locked unless you do it.

    > So not only do i need to get this solved:
    > Issue #14417 <>: Mutating a dict during
    > lookup now restarts the lookup instead of raising a RuntimeError (undoes
    > issue #14205 <>).

    As I understand #14417, you should be explicitly locking dicts. The
    issue in #14205 was that people doing mutations in just one thread and
    lookups in others usually got away without locking becuase of recursive
    retries, but occasionally crashed the interpreter because of them. The
    first fix was to prevent crashes by removing retries. But that would
    break programs that naively worked because of them. The second fix was
    to do iterative retries instead, so a thread might occasionally hang,
    but without crashing.

    As I understand it, it is better to not be a naive user.

    > i also need to lock all the actual modifying underlying "real" stuff as
    > well not just the iterator or view or whatever i don't know yet ;]

    I am not sure what you are asking.
    > So my question is kinda like, is a dict possible in the same way the
    > Queue.Queue is now made truely thread safe ?

    A custom dict might be but the builtin is not. Queue is somewhat unique
    as a builtin designed for threads.

    > Also is there something like a for queues ?

    Have you searched?

    > I want to pass a dict to a thead and then have a watcher on the dicts
    > state if result have arrived.

    Builtin dicts do not have an 'I have changed flag. You would need to
    subclass or probably better, wrap a dict using the appropriate pattern.
    Perhaps done already, but you want a wrapper that does both locking and
    watcher notification.

    Terry Jan Reedy
    Terry Reedy, Nov 27, 2012
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Rick
    Michael Borgwardt
    Nov 17, 2003
  2. nukleus
    Tom Hawtin
    Jan 19, 2007
  3. Klaus
    Richard Bos
    Dec 16, 2004
  4. Kenneth Dombrowski

    cPickle segfault with nested dicts in threaded env

    Kenneth Dombrowski, Sep 8, 2010, in forum: Python
    Kenneth Dombrowski
    Sep 9, 2010
  5. bruce
    Jan 10, 2012

Share This Page