Overriding dict constructor

S

Steven D'Aprano

I have a dict subclass that associates extra data with each value of the
key/value items:

class MyDict(dict):
def __setitem__(self, key, value):
super(MyDict, self).__setitem__(key, (value, "extra_data"))
def __getitem__(self, key):
return super(MyDict, self).__getitem__(key)[0]
# plus extra methods


This works fine for item access, updates, etc:
d = MyDict()
d[0] = 'a'; d[1] = 'b'
d[1]
'b'


But if I try to create a regular dict from this, dict() doesn't call my
__getitem__ method:
{0: ('a', 'extra_data'), 1: ('b', 'extra_data')}


instead of {0: 'a', 1: 'b'} as I expected.

How can I fix this?
 
S

Steven D'Aprano

I was going to suggest overriding items() (or iteritems() for Python
2.x), but while that is another hole that your values leak out it isn't
the hole used by the dict constructor.

Yes, I already override items(), keys(), values(), their iter...
versions, and just about every damn method that dicts have :/

I suspect that the dict() constructor is just grabbing the key/value
pairs directly from the underlying hash table. If that's the case, my
choices are to not inherit from dict at all, or take your suggestion and
keep an auxiliary dict alongside the main one.
 
H

Hrvoje Niksic

Christian Heimes said:
Am 20.09.2010 13:11, schrieb Steven D'Aprano:
I have a dict subclass that associates extra data with each value of the
key/value items: [...]
How can I fix this?

Since the dict class is crucial to the overall performance of Python,
the dict class behaves bit different than other classes. I don't know if
this is documented somewhere. Dict methods call the PyDict_GetItem
function directly instead of going through the type's struct.

Normally where this kind of optimization is necessary Python is careful
to use PyFoo_CheckExact to find out if it is dealing with an instance of
the class or a subclass, and only goes through the fast path for the
former case. That PyDict_Merge (called by dict_init) doesn't do this
could be considered a bug because it constrains dict subclass in a way
that is hard to work around, and without a clear gain in performance
compared to using an exact check.

dict_fromkeys is an example in the same file that uses
PyDict_CheckExact.
 
R

Raymond Hettinger

[Steven D'Aprano]
But if I try to create a regular dict from this, dict() doesn't call my
__getitem__ method:


{0: ('a', 'extra_data'), 1: ('b', 'extra_data')}

instead of {0: 'a', 1: 'b'} as I expected.

How can I fix this?

Try using dict(d.items()).


Raymond
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

No members online now.

Forum statistics

Threads
473,744
Messages
2,569,484
Members
44,903
Latest member
orderPeak8CBDGummies

Latest Threads

Top