Unexpected behavior with dictionary keys containment and auser-defined class

Discussion in 'Python' started by Rob Clewley, Jan 13, 2009.

  1. Rob Clewley

    Rob Clewley Guest

    Hi, the short version of my question is: when is a dictionary's
    __contains__ method behavior different to using the 'in' idiom?
    (because I have an example of a difference in my code).

    Longer version: I have a user-defined class with a few overrides of
    special methods, particularly __eq__ and __ne__. I also have a
    dictionary keyed by instances of these classes, and I'm confused about
    the unexpected behavior trying to test whether an instance is in the
    dictionary's keys. The instance is i and the dictionary is d. I have
    been using the idiom

    i in d

    which I understood to be the pythonic way to test the keys, but it
    doesn't work. However, when I debug my code I see the instance in the
    list of keys, and in fact

    i in d.keys() and d.keys()[10] == i

    both return True. But

    d.__contains__(i)
    d.has_key(i)
    d.keys()[10] is i

    return False. I put a print statement in my class's __eq__ method and
    it is being called. It tests equality of some of my class instance's
    attributes. I didn't realize there was any situation where you could
    expect different results from i in d versus i in d.keys() --
    am I misunderstanding something?

    I'm not sure what other details to provide! Thanks a lot,
    Rob
     
    Rob Clewley, Jan 13, 2009
    #1
    1. Advertising

  2. Rob Clewley

    James Stroud Guest

    Re: Unexpected behavior with dictionary keys containment and a user-definedclass

    Rob Clewley wrote:
    > Hi, the short version of my question is: when is a dictionary's
    > __contains__ method behavior different to using the 'in' idiom?
    > (because I have an example of a difference in my code).


    Never.

    > Longer version: I have a user-defined class with a few overrides of
    > special methods, particularly __eq__ and __ne__. I also have a
    > dictionary keyed by instances of these classes, and I'm confused about
    > the unexpected behavior trying to test whether an instance is in the
    > dictionary's keys. The instance is i and the dictionary is d. I have
    > been using the idiom
    >
    > i in d
    >
    > which I understood to be the pythonic way to test the keys, but it
    > doesn't work. However, when I debug my code I see the instance in the
    > list of keys, and in fact
    >
    > i in d.keys() and d.keys()[10] == i
    >
    > both return True. But
    >
    > d.__contains__(i)
    > d.has_key(i)
    > d.keys()[10] is i
    >
    > return False. I put a print statement in my class's __eq__ method and
    > it is being called. It tests equality of some of my class instance's
    > attributes. I didn't realize there was any situation where you could
    > expect different results from i in d versus i in d.keys() --
    > am I misunderstanding something?


    Well, the only conclusion is that dict uses the hash of an object to
    test containment while lists use id.

    James
     
    James Stroud, Jan 13, 2009
    #2
    1. Advertising

  3. On Mon, 12 Jan 2009 22:41:00 -0500, Rob Clewley wrote:

    > Hi, the short version of my question is: when is a dictionary's
    > __contains__ method behavior different to using the 'in' idiom? (because
    > I have an example of a difference in my code).


    [...]

    > i in d.keys() and d.keys()[10] == i
    >
    > both return True. But
    >
    > d.__contains__(i)
    > d.has_key(i)
    > d.keys()[10] is i


    The instance you are testing for isn't the same instance as the one in
    the dictionary. It might be *equal*, but it isn't identical, and by
    default, hashing of classes goes by identity. You need to give your class
    a hash function so that whenever x==y hash(x)==hash(y) as well.

    That means you need to over-ride __hash__(self) in the class.


    > return False. I put a print statement in my class's __eq__ method and it
    > is being called. It tests equality of some of my class instance's
    > attributes. I didn't realize there was any situation where you could
    > expect different results from i in d versus i in d.keys() -- am
    > I misunderstanding something?


    i in d.keys() does an item-by-item equality test, returning the first
    time i is equal to an item.

    i in d hashes i, then looks up a table to see whether there is an item in
    that spot. If there is, it compares that item to i for equality.



    --
    Steven
     
    Steven D'Aprano, Jan 13, 2009
    #3
  4. Rob Clewley

    Rob Clewley Guest

    >> Hi, the short version of my question is: when is a dictionary's
    >> __contains__ method behavior different to using the 'in' idiom?
    >> (because I have an example of a difference in my code).

    >
    > Never.


    Yes, sorry, I managed to summarize the long version incorrectly :)

    > Well, the only conclusion is that dict uses the hash of an object to test
    > containment while lists use id.


    Great, thanks. I think a copy of my object has been made so that it
    doesn't hash to the same thing. I'll write a __hash__ as suggested in
    the other post. I forgot all about dictionary hashing for some
    reason...

    -Rob
     
    Rob Clewley, Jan 13, 2009
    #4
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Christopher Deckers
    Replies:
    0
    Views:
    469
    Christopher Deckers
    Oct 15, 2003
  2. Chandrashekara Adiga

    Containment classes

    Chandrashekara Adiga, Aug 11, 2003, in forum: C++
    Replies:
    2
    Views:
    573
    Rolf Magnus
    Aug 11, 2003
  3. Oodini
    Replies:
    1
    Views:
    1,788
    Keith Thompson
    Sep 27, 2005
  4. galathaea
    Replies:
    6
    Views:
    307
    James Kanze
    Jan 8, 2009
  5. Replies:
    9
    Views:
    390
    James Kanze
    Apr 8, 2009
Loading...

Share This Page