Inconsistency in dictionary behaviour: dict(dict) not calling __setitem__

Discussion in 'Python' started by Almad, Dec 12, 2006.

  1. Almad

    Almad Guest

    Hello,

    I discovered this behaviour in dictionary which I find confusing. In
    SneakyLang, I've tried to extend dictionary so it visits another class
    after something is added:

    class RegisterMap(dict):
    def __setitem__(self, k, v):
    dict.__setitem__(self, k,v)
    self[k].visit_register_map(self)


    However, when constructing dictionary with dictionary in constructor
    like d = RegisterMap({'k':'v'}), __setitem__ is not called, so
    workaround is needed:

    class RegisterMap(dict):
    def __init__(self, *args, **kwargs):
    dict.__init__(self, *args, **kwargs)
    for k in self:
    self.__after_add(k)

    def __after_add(self, k):
    self[k].visit_register_map(self)

    def __setitem__(self, k, v):
    dict.__setitem__(self, k,v)
    self.__after_add(k)


    What is the reason for this behavior? Am I doing something wrong and
    better approach is needed? Or should this be considered as minor bug in
    Python? (tried this only in 2.4 so far)

    Thank You,

    Almad
    Almad, Dec 12, 2006
    #1
    1. Advertising

  2. Re: Inconsistency in dictionary behaviour: dict(dict) not calling__setitem__

    Almad wrote:

    > However, when constructing dictionary with dictionary in constructor
    > like d = RegisterMap({'k':'v'}), __setitem__ is not called


    why should it do that? dict() is a concrete implementation, not a
    template class for the creation of dict-like objects.

    > Or should this be considered as minor bug in Python?


    no.

    </F>
    Fredrik Lundh, Dec 12, 2006
    #2
    1. Advertising

  3. Re: Inconsistency in dictionary behaviour: dict(dict) not calling__setitem__

    Fredrik Lundh wrote:
    > Almad wrote:
    >
    >> However, when constructing dictionary with dictionary in constructor
    >> like d = RegisterMap({'k':'v'}), __setitem__ is not called

    >
    > why should it do that? dict() is a concrete implementation, not a
    > template class for the creation of dict-like objects.


    I think what was unexpected for the OP is that dict.__init__
    does not use __setitem__ to create its internal structures.
    This is independent of dict being a concrete implementation
    or not:

    >>> class H:

    .... def __init__(self): self.hello()
    .... def hello(self): print 'oi oi'
    >>> class K(H):

    .... def hello(self): print 'hello'
    >>> K()

    hello
    <__main__.K instance at 0x00AC9878>

    At least, I know it surprised me when I first met this
    behavior. Or is my reasoning incorrect?

    Further to the OP: This is due to optimizations -
    initialization is much faster if it's done by modifying
    dict's internal state directly and thus without calling a
    (possibly custom-made) method for each item to be inserted.
    What I find an even nastier surprise is that dict.update
    behaves this way as well:

    >>> class D(dict):

    .... def __setitem__(self,k,v):
    .... print 'setting',k,'to',v
    .... dict.__setitem__(self,k,v)
    >>> d=D({1:'a', 2:int})

    # No output here - OK, half-expected
    >>> d.update({3:'s',6:23})

    # No output here, either...

    The docstring is, at best, misguiding on this particular point:

    >>> print d.update.__doc__

    D.update(E, **F) -> None. Update D from E and F: for k in
    E: D[k] = E[k]
    (if E has keys else: for (k, v) in E: D[k] = v) then: for k
    in F: D[k] = F[k]


    It's not a big issue, but an almost sure one-time surprise
    for everyone, I suppose...
    Mitja Trampus, Dec 12, 2006
    #3
  4. Re: Inconsistency in dictionary behaviour: dict(dict) not calling__setitem__

    Mitja Trampus wrote:

    > I think what was unexpected for the OP is that dict.__init__
    > does not use __setitem__ to create its internal structures.


    you can implement most methods on core objects in terms of other
    methods. picking one by random, and then complaining that other
    methods don't use the one you picked, strikes me as a rather
    naive view of how object-oriented design works in practice.

    </F>
    Fredrik Lundh, Dec 12, 2006
    #4
  5. Re: Inconsistency in dictionary behaviour: dict(dict) not calling__setitem__

    Mitja Trampus wrote:

    ....
    > At least, I know it surprised me when I first met this behavior. Or is
    > my reasoning incorrect?


    Why len() doesn't call iteritems() ? :)

    Kidding apart for example it would be ok for __setitem__
    to call either an internal "insert_new_item" or
    "update_existing_item" depending on if the key is
    already present in the dictionary.
    In this case I suppose you agree it would make a lot
    of sense to go directly for "insert_new_item" in the
    constructor from a dict instead of calling the public
    __setitem__...

    The key point is that you're not authorized to assume
    constructing a dictionary from a dictionary will use
    __setitem__ unless this is explicitly stated in the
    interface.

    ....

    > What I find an even nastier surprise is that dict.update behaves this
    > way as well:

    ....
    > The docstring is, at best, misguiding on this particular point:
    >
    > >>> print d.update.__doc__

    > D.update(E, **F) -> None. Update D from E and F: for k in E: D[k] = E[k]
    > (if E has keys else: for (k, v) in E: D[k] = v) then: for k in F: D[k] =
    > F[k]


    I cannot understand this doc string at all.

    The explanation in the manual however just talks about
    "updating", with no reference to assignments. The manual
    of 2.3 instead was using a code example and I'd say this
    would qualify as a binding to actually implement calls
    to __setitem__. This kind of error (i.e. over-specifying
    by providing actual code that implies specific side-effects)
    was also present in the C++ standard, and in at least
    one case an implementation would have to be very
    inefficient to comply on the issue (this fortunately is
    not what happened, the standard was "fixed" instead).

    If there is a bug in this case is IMO a docstring bug.

    Andrea
    Andrea Griffini, Dec 13, 2006
    #5
  6. Almad

    Terry Reedy Guest

    Re: Inconsistency in dictionary behaviour: dict(dict) notcalling__setitem__

    "Almad" <> wrote in message
    news:...
    > Hello,
    >
    > I discovered this behaviour in dictionary which I find confusing

    ....
    > However, when constructing dictionary with dictionary in constructor
    > like d = RegisterMap({'k':'v'}), __setitem__ is not called,


    d.__setitem__(k,v) is the internal translation of d[k] = v (when such is
    explicitly written in this code). d.__init__(otherdict) need not go thru
    that interface if a more direct means is available.

    >so workaround is needed:


    Your 'workaround' seems to be the proper solution.

    > Am I doing something wrong


    Denigrating a solution because it does not meet your expectation.

    Your are not the first to do this ;-).

    tjr
    Terry Reedy, Dec 13, 2006
    #6
  7. [Almad]
    > I discovered this behaviour in dictionary which I find confusing. In
    > SneakyLang, I've tried to extend dictionary so it visits another class
    > after something is added:
    >
    > class RegisterMap(dict):
    > def __setitem__(self, k, v):
    > dict.__setitem__(self, k,v)
    > self[k].visit_register_map(self)
    >
    >
    > However, when constructing dictionary with dictionary in constructor
    > like d = RegisterMap({'k':'v'}), __setitem__ is not called,


    Try subclassing from UserDict.DictMixin.


    Raymond
    Raymond Hettinger, Dec 13, 2006
    #7
  8. Almad

    Almad Guest

    Thanks to everybody for replies, I'm now satisfied ^_^

    Almad
    Almad, Dec 14, 2006
    #8
  9. Almad

    Terry Reedy Guest

    Re: Inconsistency in dictionary behaviour: dict(dict) notcalling__setitem__

    Final note: one of the developers ran into a similar issue with dict and
    has opened a discussion on pydev about how the C implementation might be
    changed to have derived classes act more consistently without imposing a
    time penalty on the normal use of dict. There might possibly be a change
    by 2.6 but I am not following the details.

    tjr
    Terry Reedy, Dec 14, 2006
    #9
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Ron Garret

    Setdefault bypasses __setitem__

    Ron Garret, Oct 13, 2005, in forum: Python
    Replies:
    9
    Views:
    461
    Fredrik Lundh
    Oct 14, 2005
  2. Tor Erik Soenvisen

    str and __setitem__

    Tor Erik Soenvisen, Jan 25, 2007, in forum: Python
    Replies:
    3
    Views:
    267
  3. jeremito
    Replies:
    12
    Views:
    542
    Bruno Desthuilliers
    Feb 7, 2007
  4. Ravi
    Replies:
    3
    Views:
    263
  5. Ravi
    Replies:
    4
    Views:
    421
    sturlamolden
    Mar 7, 2011
Loading...

Share This Page