why does UserDict.DictMixin use keys instead of __iter__?

S

Steven Bethard

Sorry if this is a repost -- it didn't appear for me the first time.


So I was looking at the Language Reference's discussion about emulating
container types[1], and nowhere in it does it mention that .keys() is
part of the container protocol. Because of this, I would assume that to
use UserDict.DictMixin correctly, a class would only need to define
__getitem__, __setitem__, __delitem__ and __iter__. So why does
UserDict.DictMixin require keys() to be defined?

py> class D(object, UserDict.DictMixin):
.... """Simple dict wrapper that implements container protocol"""
.... def __init__(self, dict): self.dict = dict
.... def __len__(self, key): return len(self.dict)
.... def __getitem__(self, key): return self.dict[key]
.... def __setitem__(self, key, value): self.dict[key] = value
.... def __delitem__(self, key): del self.dict[key]
.... def __iter__(self): return iter(self.dict)
.... def __contains__(self, key): return key in self.dict
....
py> d = D(dict(a=1, b=2))
py> d.clear()
Traceback (most recent call last):
File "<interactive input>", line 1, in ?
File "C:\Program Files\Python\lib\UserDict.py", line 114, in clear
for key in self.keys():
AttributeError: 'D' object has no attribute 'keys'
py> d.keys()
Traceback (most recent call last):
File "<interactive input>", line 1, in ?
AttributeError: 'D' object has no attribute 'keys'


I thought about submitting a patch, but I couldn't think of a way that
didn't raise backwards compatibility concerns...


Steve

[1]http://docs.python.org/ref/sequence-types.html
 
N

Nick Coghlan

Steven said:
Sorry if this is a repost -- it didn't appear for me the first time.


So I was looking at the Language Reference's discussion about emulating
container types[1], and nowhere in it does it mention that .keys() is
part of the container protocol. Because of this, I would assume that to
use UserDict.DictMixin correctly, a class would only need to define
__getitem__, __setitem__, __delitem__ and __iter__. So why does
UserDict.DictMixin require keys() to be defined?

Because it's a DictMixin, not a ContainerMixin?

..keys() is definitely part of the standard dictionary interface, and not
something the mixin can derive from the generic container methods.

Cheers,
Nick.
 
J

John Machin

Steven said:
Sorry if this is a repost -- it didn't appear for me the first time.


So I was looking at the Language Reference's discussion about emulating
container types[1], and nowhere in it does it mention that .keys() is
part of the container protocol.

I don't see any reference to a "container protocol". What I do see is
(1) """It is also recommended that mappings provide the methods keys(),
...."""
(2) """The UserDict module provides a DictMixin class to help create
those methods from a base set of __getitem__(), __setitem__(),
__delitem__(), and keys(). """
Because of this, I would assume that to
use UserDict.DictMixin correctly, a class would only need to define
__getitem__, __setitem__, __delitem__ and __iter__.

So I can't see why would you assume that, given that the docs say in
effect "you supply get/set/del + keys as the building blocks, the
DictMixin class will provide the remainder". This message is reinforced
in the docs for UserDict itself.
So why does
UserDict.DictMixin require keys() to be defined?

Because it was a reasonable, documented, design?

In any case, isn't UserDict past history? Why are you mucking about
with it?
 
S

Steven Bethard

Nick said:
Steven said:
Sorry if this is a repost -- it didn't appear for me the first time.


So I was looking at the Language Reference's discussion about emulating
container types[1], and nowhere in it does it mention that .keys() is
part of the container protocol. Because of this, I would assume that to
use UserDict.DictMixin correctly, a class would only need to define
__getitem__, __setitem__, __delitem__ and __iter__. So why does
UserDict.DictMixin require keys() to be defined?


Because it's a DictMixin, not a ContainerMixin?

"Containers usually are sequences (such as lists or tuples) or mappings
(like dictionaries)".
.keys() is definitely part of the standard dictionary interface, and not
something the mixin can derive from the generic container methods.

Why is that? Isn't keys derivable as:

def keys(self):
return list(self)

if __iter__ is defined?

Steve
 
S

Steven Bethard

John said:
Steven said:
So I was looking at the Language Reference's discussion about
emulating container types[1], and nowhere in it does it mention that
.keys() is part of the container protocol.

I don't see any reference to a "container protocol".

Sorry, I extrapolated "container protocol" from this statement:

"Containers usually are sequences (such as lists or tuples) or mappings
(like dictionaries), but can represent other containers as well. The
first set of methods is used either to emulate a sequence or to emulate
a mapping"

and the fact that there is a "sequence protocol" and a "mapping protocol".

But all I was really reading from this statement was that the "first set
of methods" (__len__, __getitem__, __setitem__, __delitem__ and
__iter__) were more integral than the second set of methods (keys(),
values(), ...).

What I do see is
(1) """It is also recommended that mappings provide the methods keys(),
..."""

You skipped the remaining 13 methods in this list:

"It is also recommended that mappings provide the methods keys(),
values(), items(), has_key(), get(), clear(), setdefault(), iterkeys(),
itervalues(), iteritems(), pop(), popitem(), copy(), and update()
behaving similar to those for Python's standard dictionary objects."

This is the "second set of methods" I mentioned above. I don't
understand why the creators of UserDict.DictMixin decided that keys(),
from the second list, is more important than __iter__, from the first list.

So I can't see why would you assume that, given that the docs say in
effect "you supply get/set/del + keys as the building blocks, the
DictMixin class will provide the remainder". This message is reinforced
in the docs for UserDict itself.

Sorry, my intent was not to say that I didn't know from the docs that
UserDict.DictMixin required keys(). Clearly it's documented. My
question was *why* does it use keys()? Why use keys() when keys() can
be derived from __iter__, and __iter__ IMHO looks to be a more basic
part of the mapping protocol.
In any case, isn't UserDict past history? Why are you mucking about
with it?

UserDict is past history, but DictMixin isn't. As you note, DictMixin
is even mentioned in the section of the Language Reference that we're
discussing:

"The UserDict module provides a DictMixin class to help create those
methods from a base set of __getitem__(), __setitem__(), __delitem__(),
and keys()."


Steve
 
J

John Machin

Steven said:
John said:
Steven said:
So I was looking at the Language Reference's discussion about
emulating container types[1], and nowhere in it does it mention that
.keys() is part of the container protocol.

I don't see any reference to a "container protocol".

Sorry, I extrapolated "container protocol" from this statement:

"Containers usually are sequences (such as lists or tuples) or mappings
(like dictionaries), but can represent other containers as well. The
first set of methods is used either to emulate a sequence or to emulate
a mapping"

and the fact that there is a "sequence protocol" and a "mapping protocol".

But all I was really reading from this statement was that the "first set
of methods" (__len__, __getitem__, __setitem__, __delitem__ and
__iter__) were more integral than the second set of methods (keys(),
values(), ...).

What I do see is
(1) """It is also recommended that mappings provide the methods keys(),
..."""

You skipped the remaining 13 methods in this list:

"It is also recommended that mappings provide the methods keys(),
values(), items(), has_key(), get(), clear(), setdefault(), iterkeys(),
itervalues(), iteritems(), pop(), popitem(), copy(), and update()
behaving similar to those for Python's standard dictionary objects."

This is the "second set of methods" I mentioned above. I don't
understand why the creators of UserDict.DictMixin decided that keys(),
from the second list, is more important than __iter__, from the first list.
So I can't see why would you assume that, given that the docs say in
effect "you supply get/set/del + keys as the building blocks, the
DictMixin class will provide the remainder". This message is reinforced
in the docs for UserDict itself.

Sorry, my intent was not to say that I didn't know from the docs that
UserDict.DictMixin required keys(). Clearly it's documented.

Sorry, the combination of (a) "assume X where not(X) is documented" and
(b) posting of tracebacks that demonstrated behaviour that is both
expected and documented lead to my making an unwarranted assumption :)
My
question was *why* does it use keys()? Why use keys() when keys() can
be derived from __iter__, and __iter__ IMHO looks to be a more basic
part of the mapping protocol.

Now that I understand your question: Hmmm, good question. __iter__
arrived (2.2) before DictMixin (2.3), so primacy is not the reason.
Ease of implementation by the user of DictMixin: probably not, "yield
akey" vs "alist.append(akey)" -- not much in it in Python, different
story in C, but a C extension wouldn't be using DictMixin anyway.
UserDict is past history, but DictMixin isn't.

OK, I'll rephrase: what is your interest in DictMixin?

My interest: I'm into mappings that provide an approximate match
capability, and have a few different data structures that I'd like to
implement as C types in a unified manner. The plot includes a base type
that, similarly to DictMixin, provides all the non-basic methods.
 
R

Raymond Hettinger

[Steven Bethard]
Sorry, my intent was not to say that I didn't know from the docs that
UserDict.DictMixin required keys(). Clearly it's documented. My
question was *why* does it use keys()? Why use keys() when keys() can
be derived from __iter__, and __iter__ IMHO looks to be a more basic
part of the mapping protocol.

Viewed from the present, __iter__() may seem more basic. However, it is a
recent innovation. The keys() method, on the other hand, goes back to the
beginning. There were no shortage of mapping-like classes defining keys() but
not __iter__().

Still, if __iter__() is provided, UserDict.DictMixin will take advantage of it.
The same is also true for __contains__(), and iteritems().


Raymond Hettinger
 
S

Steven Bethard

John said:
OK, I'll rephrase: what is your interest in DictMixin?

My interest: I'm into mappings that provide an approximate match
capability, and have a few different data structures that I'd like to
implement as C types in a unified manner. The plot includes a base type
that, similarly to DictMixin, provides all the non-basic methods.

I was recently trying to prototype a simple mapping type that implements
the suggestion "Improved default value logic for Dictionaries" from
http://www.python.org/moin/Python3_2e0Suggestions
You can't just inherit from dict and override dict.__getitem__ because
dict.__getitem__ isn't always called:

py> class D(dict):
.... def __init__(*args, **kwds):
.... self = args[0]
.... self.function, self.args, self.kwds = None, None, None
.... super(D, self).__init__(*args[1:], **kwds)
.... def setdefault(self, function, *args, **kwds):
.... self.function, self.args, self.kwds = function, args, kwds
.... def __getitem__(self, key):
.... if key not in self:
.... super(D, self).__setitem__(
.... key, self.function(*self.args, **self.kwds))
.... return super(D, self).__getitem__(key)
....
py> d = D()
py> d.setdefault(list)
py> d['c'].append(2)
py> d
{'c': [2]}
py> print d.get('d') # should print []
None

This, of course, is exactly the kind of thing that DictMixin is designed
for. =)

Of course, it's no trouble for me to implement keys(). I was just
wondering why that design decision was made when it seems like __iter__
is more integral to the mapping protocol. And if you want efficient
iteration over your mapping type, you're going to have to define
__iter__ too...

Steve
 
N

Nick Coghlan

Steven said:
Why is that? Isn't keys derivable as:

def keys(self):
return list(self)

if __iter__ is defined?

As you may have guessed, I completely forgot about __iter__. . .

Cheers,
Nick.
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

Forum statistics

Threads
473,744
Messages
2,569,484
Members
44,903
Latest member
orderPeak8CBDGummies

Latest Threads

Top