Unexpected behavior with dictionary keys containment and auser-defined class

[Rob Clewley]

Hi, the short version of my question is: when is a dictionary's
__contains__ method behavior different to using the 'in' idiom?
(because I have an example of a difference in my code).

Longer version: I have a user-defined class with a few overrides of
special methods, particularly __eq__ and __ne__. I also have a
dictionary keyed by instances of these classes, and I'm confused about
the unexpected behavior trying to test whether an instance is in the
dictionary's keys. The instance is i and the dictionary is d. I have
been using the idiom

i in d

which I understood to be the pythonic way to test the keys, but it
doesn't work. However, when I debug my code I see the instance in the
list of keys, and in fact

i in d.keys() and d.keys()[10] == i

both return True. But

d.__contains__(i)
d.has_key(i)
d.keys()[10] is i

return False. I put a print statement in my class's __eq__ method and
it is being called. It tests equality of some of my class instance's
attributes. I didn't realize there was any situation where you could
expect different results from i in d versus i in d.keys() --
am I misunderstanding something?

I'm not sure what other details to provide! Thanks a lot,
Rob

 

[James Stroud]

Hi, the short version of my question is: when is a dictionary's
__contains__ method behavior different to using the 'in' idiom?
(because I have an example of a difference in my code).
Never.

Longer version: I have a user-defined class with a few overrides of
special methods, particularly __eq__ and __ne__. I also have a
dictionary keyed by instances of these classes, and I'm confused about
the unexpected behavior trying to test whether an instance is in the
dictionary's keys. The instance is i and the dictionary is d. I have
been using the idiom

i in d
>
which I understood to be the pythonic way to test the keys, but it
doesn't work. However, when I debug my code I see the instance in the
list of keys, and in fact

i in d.keys() and d.keys()[10] == i

both return True. But

d.__contains__(i)
d.has_key(i)
d.keys()[10] is i

return False. I put a print statement in my class's __eq__ method and
it is being called. It tests equality of some of my class instance's
attributes. I didn't realize there was any situation where you could
expect different results from i in d versus i in d.keys() --
am I misunderstanding something?

Well, the only conclusion is that dict uses the hash of an object to
test containment while lists use id.

James

 

[Steven D'Aprano]

Hi, the short version of my question is: when is a dictionary's
__contains__ method behavior different to using the 'in' idiom? (because
I have an example of a difference in my code).
[...]

i in d.keys() and d.keys()[10] == i

both return True. But

d.__contains__(i)
d.has_key(i)
d.keys()[10] is i

The instance you are testing for isn't the same instance as the one in
the dictionary. It might be *equal*, but it isn't identical, and by
default, hashing of classes goes by identity. You need to give your class
a hash function so that whenever x==y hash(x)==hash(y) as well.

That means you need to over-ride __hash__(self) in the class.

return False. I put a print statement in my class's __eq__ method and it
is being called. It tests equality of some of my class instance's
attributes. I didn't realize there was any situation where you could
expect different results from i in d versus i in d.keys() -- am
I misunderstanding something?

i in d.keys() does an item-by-item equality test, returning the first
time i is equal to an item.

i in d hashes i, then looks up a table to see whether there is an item in
that spot. If there is, it compares that item to i for equality.

 

[Rob Clewley]

Hi, the short version of my question is: when is a dictionary's

Never.

Yes, sorry, I managed to summarize the long version incorrectly

Well, the only conclusion is that dict uses the hash of an object to test
containment while lists use id.

Great, thanks. I think a copy of my object has been made so that it
doesn't hash to the same thing. I'll write a __hash__ as suggested in
the other post. I forgot all about dictionary hashing for some
reason...

-Rob