why does UserDict.DictMixin use keys instead of __iter__?

Discussion in 'Python' started by Steven Bethard, Jan 4, 2005.

  1. Sorry if this is a repost -- it didn't appear for me the first time.


    So I was looking at the Language Reference's discussion about emulating
    container types[1], and nowhere in it does it mention that .keys() is
    part of the container protocol. Because of this, I would assume that to
    use UserDict.DictMixin correctly, a class would only need to define
    __getitem__, __setitem__, __delitem__ and __iter__. So why does
    UserDict.DictMixin require keys() to be defined?

    py> class D(object, UserDict.DictMixin):
    .... """Simple dict wrapper that implements container protocol"""
    .... def __init__(self, dict): self.dict = dict
    .... def __len__(self, key): return len(self.dict)
    .... def __getitem__(self, key): return self.dict[key]
    .... def __setitem__(self, key, value): self.dict[key] = value
    .... def __delitem__(self, key): del self.dict[key]
    .... def __iter__(self): return iter(self.dict)
    .... def __contains__(self, key): return key in self.dict
    ....
    py> d = D(dict(a=1, b=2))
    py> d.clear()
    Traceback (most recent call last):
    File "<interactive input>", line 1, in ?
    File "C:\Program Files\Python\lib\UserDict.py", line 114, in clear
    for key in self.keys():
    AttributeError: 'D' object has no attribute 'keys'
    py> d.keys()
    Traceback (most recent call last):
    File "<interactive input>", line 1, in ?
    AttributeError: 'D' object has no attribute 'keys'


    I thought about submitting a patch, but I couldn't think of a way that
    didn't raise backwards compatibility concerns...


    Steve

    [1]http://docs.python.org/ref/sequence-types.html
    Steven Bethard, Jan 4, 2005
    #1
    1. Advertising

  2. Steven Bethard

    Nick Coghlan Guest

    Steven Bethard wrote:
    > Sorry if this is a repost -- it didn't appear for me the first time.
    >
    >
    > So I was looking at the Language Reference's discussion about emulating
    > container types[1], and nowhere in it does it mention that .keys() is
    > part of the container protocol. Because of this, I would assume that to
    > use UserDict.DictMixin correctly, a class would only need to define
    > __getitem__, __setitem__, __delitem__ and __iter__. So why does
    > UserDict.DictMixin require keys() to be defined?


    Because it's a DictMixin, not a ContainerMixin?

    ..keys() is definitely part of the standard dictionary interface, and not
    something the mixin can derive from the generic container methods.

    Cheers,
    Nick.

    --
    Nick Coghlan | | Brisbane, Australia
    ---------------------------------------------------------------
    http://boredomandlaziness.skystorm.net
    Nick Coghlan, Jan 4, 2005
    #2
    1. Advertising

  3. Steven Bethard

    John Machin Guest

    Steven Bethard wrote:
    > Sorry if this is a repost -- it didn't appear for me the first time.
    >
    >
    > So I was looking at the Language Reference's discussion about

    emulating
    > container types[1], and nowhere in it does it mention that .keys() is
    > part of the container protocol.


    I don't see any reference to a "container protocol". What I do see is
    (1) """It is also recommended that mappings provide the methods keys(),
    ...."""
    (2) """The UserDict module provides a DictMixin class to help create
    those methods from a base set of __getitem__(), __setitem__(),
    __delitem__(), and keys(). """

    > Because of this, I would assume that to
    > use UserDict.DictMixin correctly, a class would only need to define
    > __getitem__, __setitem__, __delitem__ and __iter__.


    So I can't see why would you assume that, given that the docs say in
    effect "you supply get/set/del + keys as the building blocks, the
    DictMixin class will provide the remainder". This message is reinforced
    in the docs for UserDict itself.

    > So why does
    > UserDict.DictMixin require keys() to be defined?


    Because it was a reasonable, documented, design?

    In any case, isn't UserDict past history? Why are you mucking about
    with it?
    John Machin, Jan 4, 2005
    #3
  4. Nick Coghlan wrote:
    > Steven Bethard wrote:
    >
    >> Sorry if this is a repost -- it didn't appear for me the first time.
    >>
    >>
    >> So I was looking at the Language Reference's discussion about emulating
    >> container types[1], and nowhere in it does it mention that .keys() is
    >> part of the container protocol. Because of this, I would assume that to
    >> use UserDict.DictMixin correctly, a class would only need to define
    >> __getitem__, __setitem__, __delitem__ and __iter__. So why does
    >> UserDict.DictMixin require keys() to be defined?

    >
    >
    > Because it's a DictMixin, not a ContainerMixin?


    "Containers usually are sequences (such as lists or tuples) or mappings
    (like dictionaries)".

    > .keys() is definitely part of the standard dictionary interface, and not
    > something the mixin can derive from the generic container methods.


    Why is that? Isn't keys derivable as:

    def keys(self):
    return list(self)

    if __iter__ is defined?

    Steve
    Steven Bethard, Jan 4, 2005
    #4
  5. John Machin wrote:
    > Steven Bethard wrote:
    >
    >>So I was looking at the Language Reference's discussion about
    >>emulating container types[1], and nowhere in it does it mention that
    >> .keys() is part of the container protocol.

    >
    > I don't see any reference to a "container protocol".


    Sorry, I extrapolated "container protocol" from this statement:

    "Containers usually are sequences (such as lists or tuples) or mappings
    (like dictionaries), but can represent other containers as well. The
    first set of methods is used either to emulate a sequence or to emulate
    a mapping"

    and the fact that there is a "sequence protocol" and a "mapping protocol".

    But all I was really reading from this statement was that the "first set
    of methods" (__len__, __getitem__, __setitem__, __delitem__ and
    __iter__) were more integral than the second set of methods (keys(),
    values(), ...).


    > What I do see is
    > (1) """It is also recommended that mappings provide the methods keys(),
    > ..."""


    You skipped the remaining 13 methods in this list:

    "It is also recommended that mappings provide the methods keys(),
    values(), items(), has_key(), get(), clear(), setdefault(), iterkeys(),
    itervalues(), iteritems(), pop(), popitem(), copy(), and update()
    behaving similar to those for Python's standard dictionary objects."

    This is the "second set of methods" I mentioned above. I don't
    understand why the creators of UserDict.DictMixin decided that keys(),
    from the second list, is more important than __iter__, from the first list.


    >>Because of this, I would assume that to
    >>use UserDict.DictMixin correctly, a class would only need to define
    >>__getitem__, __setitem__, __delitem__ and __iter__.

    >
    >
    > So I can't see why would you assume that, given that the docs say in
    > effect "you supply get/set/del + keys as the building blocks, the
    > DictMixin class will provide the remainder". This message is reinforced
    > in the docs for UserDict itself.


    Sorry, my intent was not to say that I didn't know from the docs that
    UserDict.DictMixin required keys(). Clearly it's documented. My
    question was *why* does it use keys()? Why use keys() when keys() can
    be derived from __iter__, and __iter__ IMHO looks to be a more basic
    part of the mapping protocol.

    > In any case, isn't UserDict past history? Why are you mucking about
    > with it?


    UserDict is past history, but DictMixin isn't. As you note, DictMixin
    is even mentioned in the section of the Language Reference that we're
    discussing:

    "The UserDict module provides a DictMixin class to help create those
    methods from a base set of __getitem__(), __setitem__(), __delitem__(),
    and keys()."


    Steve
    Steven Bethard, Jan 4, 2005
    #5
  6. Steven Bethard

    John Machin Guest

    Steven Bethard wrote:
    > John Machin wrote:
    > > Steven Bethard wrote:
    > >
    > >>So I was looking at the Language Reference's discussion about
    > >>emulating container types[1], and nowhere in it does it mention

    that
    > >> .keys() is part of the container protocol.

    > >
    > > I don't see any reference to a "container protocol".

    >
    > Sorry, I extrapolated "container protocol" from this statement:
    >
    > "Containers usually are sequences (such as lists or tuples) or

    mappings
    > (like dictionaries), but can represent other containers as well. The
    > first set of methods is used either to emulate a sequence or to

    emulate
    > a mapping"
    >
    > and the fact that there is a "sequence protocol" and a "mapping

    protocol".
    >
    > But all I was really reading from this statement was that the "first

    set
    > of methods" (__len__, __getitem__, __setitem__, __delitem__ and
    > __iter__) were more integral than the second set of methods (keys(),
    > values(), ...).
    >
    >
    > > What I do see is
    > > (1) """It is also recommended that mappings provide the methods

    keys(),
    > > ..."""

    >
    > You skipped the remaining 13 methods in this list:
    >
    > "It is also recommended that mappings provide the methods keys(),
    > values(), items(), has_key(), get(), clear(), setdefault(),

    iterkeys(),
    > itervalues(), iteritems(), pop(), popitem(), copy(), and update()
    > behaving similar to those for Python's standard dictionary objects."
    >
    > This is the "second set of methods" I mentioned above. I don't
    > understand why the creators of UserDict.DictMixin decided that

    keys(),
    > from the second list, is more important than __iter__, from the first

    list.
    >
    >
    > >>Because of this, I would assume that to
    > >>use UserDict.DictMixin correctly, a class would only need to define
    > >>__getitem__, __setitem__, __delitem__ and __iter__.

    > >
    > >
    > > So I can't see why would you assume that, given that the docs say

    in
    > > effect "you supply get/set/del + keys as the building blocks, the
    > > DictMixin class will provide the remainder". This message is

    reinforced
    > > in the docs for UserDict itself.

    >
    > Sorry, my intent was not to say that I didn't know from the docs that


    > UserDict.DictMixin required keys(). Clearly it's documented.


    Sorry, the combination of (a) "assume X where not(X) is documented" and
    (b) posting of tracebacks that demonstrated behaviour that is both
    expected and documented lead to my making an unwarranted assumption :)

    > My
    > question was *why* does it use keys()? Why use keys() when keys()

    can
    > be derived from __iter__, and __iter__ IMHO looks to be a more basic
    > part of the mapping protocol.


    Now that I understand your question: Hmmm, good question. __iter__
    arrived (2.2) before DictMixin (2.3), so primacy is not the reason.
    Ease of implementation by the user of DictMixin: probably not, "yield
    akey" vs "alist.append(akey)" -- not much in it in Python, different
    story in C, but a C extension wouldn't be using DictMixin anyway.
    >
    > > In any case, isn't UserDict past history? Why are you mucking about
    > > with it?

    >
    > UserDict is past history, but DictMixin isn't.


    OK, I'll rephrase: what is your interest in DictMixin?

    My interest: I'm into mappings that provide an approximate match
    capability, and have a few different data structures that I'd like to
    implement as C types in a unified manner. The plot includes a base type
    that, similarly to DictMixin, provides all the non-basic methods.
    John Machin, Jan 4, 2005
    #6
  7. [Steven Bethard]
    > Sorry, my intent was not to say that I didn't know from the docs that
    > UserDict.DictMixin required keys(). Clearly it's documented. My
    > question was *why* does it use keys()? Why use keys() when keys() can
    > be derived from __iter__, and __iter__ IMHO looks to be a more basic
    > part of the mapping protocol.


    Viewed from the present, __iter__() may seem more basic. However, it is a
    recent innovation. The keys() method, on the other hand, goes back to the
    beginning. There were no shortage of mapping-like classes defining keys() but
    not __iter__().

    Still, if __iter__() is provided, UserDict.DictMixin will take advantage of it.
    The same is also true for __contains__(), and iteritems().


    Raymond Hettinger
    Raymond Hettinger, Jan 4, 2005
    #7
  8. John Machin wrote:
    > OK, I'll rephrase: what is your interest in DictMixin?
    >
    > My interest: I'm into mappings that provide an approximate match
    > capability, and have a few different data structures that I'd like to
    > implement as C types in a unified manner. The plot includes a base type
    > that, similarly to DictMixin, provides all the non-basic methods.


    I was recently trying to prototype a simple mapping type that implements
    the suggestion "Improved default value logic for Dictionaries" from
    http://www.python.org/moin/Python3_2e0Suggestions
    You can't just inherit from dict and override dict.__getitem__ because
    dict.__getitem__ isn't always called:

    py> class D(dict):
    .... def __init__(*args, **kwds):
    .... self = args[0]
    .... self.function, self.args, self.kwds = None, None, None
    .... super(D, self).__init__(*args[1:], **kwds)
    .... def setdefault(self, function, *args, **kwds):
    .... self.function, self.args, self.kwds = function, args, kwds
    .... def __getitem__(self, key):
    .... if key not in self:
    .... super(D, self).__setitem__(
    .... key, self.function(*self.args, **self.kwds))
    .... return super(D, self).__getitem__(key)
    ....
    py> d = D()
    py> d.setdefault(list)
    py> d['c'].append(2)
    py> d
    {'c': [2]}
    py> print d.get('d') # should print []
    None

    This, of course, is exactly the kind of thing that DictMixin is designed
    for. =)

    Of course, it's no trouble for me to implement keys(). I was just
    wondering why that design decision was made when it seems like __iter__
    is more integral to the mapping protocol. And if you want efficient
    iteration over your mapping type, you're going to have to define
    __iter__ too...

    Steve
    Steven Bethard, Jan 4, 2005
    #8
  9. Steven Bethard

    Nick Coghlan Guest

    Steven Bethard wrote:
    > Nick Coghlan wrote:
    >> .keys() is definitely part of the standard dictionary interface, and
    >> not something the mixin can derive from the generic container methods.

    >
    >
    > Why is that? Isn't keys derivable as:
    >
    > def keys(self):
    > return list(self)
    >
    > if __iter__ is defined?


    As you may have guessed, I completely forgot about __iter__. . .

    Cheers,
    Nick.

    --
    Nick Coghlan | | Brisbane, Australia
    ---------------------------------------------------------------
    http://boredomandlaziness.skystorm.net
    Nick Coghlan, Jan 5, 2005
    #9
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Andreas Kuntzagk
    Replies:
    1
    Views:
    338
    Steven Taschuk
    Jul 22, 2003
  2. Alex Martelli

    UserDict question

    Alex Martelli, Nov 4, 2003, in forum: Python
    Replies:
    2
    Views:
    300
    Guyon Morée
    Nov 4, 2003
  3. John Lenton

    misguiding docs in 2.3's UserDict ?

    John Lenton, Jul 6, 2004, in forum: Python
    Replies:
    0
    Views:
    294
    John Lenton
    Jul 6, 2004
  4. Mr. SweatyFinger
    Replies:
    2
    Views:
    1,762
    Smokey Grindel
    Dec 2, 2006
  5. Replies:
    10
    Views:
    708
    Daniel T.
    Feb 3, 2006
Loading...

Share This Page