Why IterableUserDict?

[Steven D'Aprano]

I was writing some tests for a mapping class I have made, and I decided
to run those same tests over dict and UserDict. The built-in dict passed
all the tests, but UserDict failed one:

class SimpleMappingTest(unittest.TestCase):
type2test = UserDict.UserDict
def test_iter(self):
k, v = [0, 1, 2, 3], 'abcd'
m = self.type2test(zip(k, v))
it = iter(m)
self.assert_(iter(it) is it)
self.assertEquals(sorted(it), k) # This line fails.
# many more tests

To cut a long story short, the problem is that UserDict doesn't support
the modern iteration protocol. Python falls back on the old-fashioned
__getitem__ iteration protocol, but since it expects IndexError rather
than KeyError, iteration fails for UserDict once it hits key=4.

If I look at the source code for the UserDict module, I discover that
there's a second mapping class, IterableUserDict, specifically to make
UserDict iterable. To do this requires a single one-line method:

class IterableUserDict(UserDict):
def __iter__(self):
return iter(self.data)

This class was apparently added to the module in 2.2 -- it doesn't exist
in 2.1.

Now that boggles my brain. Why was support for iteration added as a
subclass, instead of simply adding the __iter__ method to UserDict?
UserDict is supposed to be a drop-in replacement for dict (although the
use-cases for it are much fewer now that we can inherit from dict), so it
doesn't make sense to me to have a non-iterable UserDict plus a subclass
which is iterable.

Can anyone shed any light on this apparently bizarre design decision?

 

[Peter Otten]

I was writing some tests for a mapping class I have made, and I decided
to run those same tests over dict and UserDict. The built-in dict passed
all the tests, but UserDict failed one:

class SimpleMappingTest(unittest.TestCase):
type2test = UserDict.UserDict
def test_iter(self):
k, v = [0, 1, 2, 3], 'abcd'
m = self.type2test(zip(k, v))
it = iter(m)
self.assert_(iter(it) is it)
self.assertEquals(sorted(it), k) # This line fails.
# many more tests

To cut a long story short, the problem is that UserDict doesn't support
the modern iteration protocol. Python falls back on the old-fashioned
__getitem__ iteration protocol, but since it expects IndexError rather
than KeyError, iteration fails for UserDict once it hits key=4.

If I look at the source code for the UserDict module, I discover that
there's a second mapping class, IterableUserDict, specifically to make
UserDict iterable. To do this requires a single one-line method:

class IterableUserDict(UserDict):
def __iter__(self):
return iter(self.data)

This class was apparently added to the module in 2.2 -- it doesn't exist
in 2.1.

Now that boggles my brain. Why was support for iteration added as a
subclass, instead of simply adding the __iter__ method to UserDict?
UserDict is supposed to be a drop-in replacement for dict (although the
use-cases for it are much fewer now that we can inherit from dict), so it
doesn't make sense to me to have a non-iterable UserDict plus a subclass
which is iterable.

Can anyone shed any light on this apparently bizarre design decision?

I looked it up for you:

http://svn.python.org/view/python/trunk/Lib/UserDict.py?view=log

"""
Revision 22248 - (view) (download) (as text) (annotate) - [select for diffs]
Modified Tue Aug 7 17:40:42 2001 UTC (9 years, 1 month ago) by gvanrossum
File length: 1913 byte(s)
Diff to previous 21167
Remove the __iter__ method from the UserDict class -- it can silently
break old code (in extreme cases). See SF bug #448153.

Add a new subclass IterableUserDict that has the __iter__ method.

Note that for new projects, unless backwards compatibility with
pre-2.2 Python is required, subclassing 'dictionary' is recommended;
UserDict might become deprecated.
"""

The bug report is also interesting:

http://bugs.python.org/issue448153

Peter

 

[Terry Reedy]

I was writing some tests for a mapping class I have made, and I decided
to run those same tests over dict and UserDict. The built-in dict passed
all the tests, but UserDict failed one:

You forgot to specify Python version ;-).

class SimpleMappingTest(unittest.TestCase):
type2test = UserDict.UserDict

In 3.x, collections.UserDict

def test_iter(self):
k, v = [0, 1, 2, 3], 'abcd'
m = self.type2test(zip(k, v))
it = iter(m)
self.assert_(iter(it) is it)
self.assertEquals(sorted(it), k) # This line fails.

Not in 3.x

import collections
k, v = [0, 1, 2, 3], 'abcd'
m = collections.UserDict(zip(k, v))
it = iter(m)
assert iter(it) is it
assert sorted(it) == k

runs clean.

If I look at the source code for the UserDict module, I discover that
there's a second mapping class, IterableUserDict,

Not any more. One of numerous 3.x cleanups made possible by dropping
obsessive back compatibility, which, as Peter explained, wan the reason
for the hack.

 

[Steven D'Aprano]

On Sat, 18 Sep 2010 09:42:14 +0200, Peter Otten wrote:

[...]

Now that boggles my brain. Why was support for iteration added as a
subclass, instead of simply adding the __iter__ method to UserDict?
UserDict is supposed to be a drop-in replacement for dict (although the
use-cases for it are much fewer now that we can inherit from dict), so
it doesn't make sense to me to have a non-iterable UserDict plus a
subclass which is iterable.

Can anyone shed any light on this apparently bizarre design decision?

I looked it up for you:

http://svn.python.org/view/python/trunk/Lib/UserDict.py?view=log [...]
The bug report is also interesting:

http://bugs.python.org/issue448153

Oh my.

Guido is more forgiving than me. I would have left __iter__ in UserDict
and treated the bug report as a "will not fix".


Thanks for that.