__eq__() inconvenience when subclassing set

Discussion in 'Python' started by Jess Austin, Oct 29, 2009.

  1. Jess Austin

    Jess Austin Guest

    I'm subclassing set, and redefining __eq__(). I'd appreciate any
    relevant advice.

    >>> class mySet(set):

    .... def __eq__(self, other):
    .... print "called mySet.__eq__()!"
    .... if isinstance(other, (set, frozenset)):
    .... return True
    .... return set.__eq__(self, other)
    ....

    I stipulate that this is a weird thing to do, but this is a toy class
    to avoid the lengthy definition of the class I actually want to
    write. Now I want the builtin set and frozenset types to use the new
    __eq__() with mySet symmetrically.

    >>> mySet() == set([1])

    called mySet.__eq__()!
    True
    >>> mySet() == frozenset([1])

    called mySet.__eq__()!
    True
    >>> set([1]) == mySet()

    called mySet.__eq__()!
    True
    >>> frozenset([1]) == mySet()

    False

    frozenset doesn't use mySet.__eq__() because mySet is not a subclass
    of frozenset as it is for set. I've tried a number of techniques to
    mitigate this issue. If I multiple-inherit from both set and
    frozenset, I get the instance lay-out conflict error. I have similar
    problems setting mySet.__bases__ directly, and hacking mro() in a
    metaclass. So far nothing has worked. If it matters, I'm using 2.6,
    but I can change versions if it will help.

    Should I give up on this, or is there something else I can try? Keep
    in mind, I must redefine __eq__(), and I'd like to be able to compare
    instances of the class to both set and frozenset instances.

    cheers,
    Jess
     
    Jess Austin, Oct 29, 2009
    #1
    1. Advertising

  2. Jess Austin schrieb:
    > >>> frozenset([1]) == mySet()

    > False
    >
    > frozenset doesn't use mySet.__eq__() because mySet is not a subclass
    > of frozenset as it is for set.


    You could just overwrite set and frozenset:

    class eqmixin(object):
    def __eq__(self, other):
    print "called %s.__eq__()" % self.__class__
    if isinstance(other, (set, frozenset)):
    return True
    return super(eqmixin, self).__eq__(other)

    class set(eqmixin, set):
    pass
    class frozenset(eqmixin, frozenset):
    pass
    class MySet(set):
    pass


    Regards,
    Mick.
     
    Mick Krippendorf, Oct 29, 2009
    #2
    1. Advertising

  3. Jess Austin

    Jess Austin Guest

    On Oct 28, 10:07 pm, Mick Krippendorf <> wrote:
    > You could just overwrite set and frozenset:
    >
    > class eqmixin(object):
    >     def __eq__(self, other):
    >         print "called %s.__eq__()" % self.__class__
    >         if isinstance(other, (set, frozenset)):
    >             return True
    >         return super(eqmixin, self).__eq__(other)
    >
    > class frozenset(eqmixin, frozenset):
    >     pass


    That's nice, but it means that everyone who imports my class will have
    to import the monkeypatch of frozenset, as well. I'm not sure I want
    that. More ruby than python, ne?

    thanks,
    Jess
     
    Jess Austin, Oct 29, 2009
    #3
  4. Jess Austin wrote:
    > That's nice, but it means that everyone who imports my class will have
    > to import the monkeypatch of frozenset, as well. I'm not sure I want
    > that. More ruby than python, ne?


    I thought it was only a toy class?

    Mick.
     
    Mick Krippendorf, Oct 29, 2009
    #4
  5. Jess Austin

    Jess Austin Guest

    On Oct 29, 3:54 pm, Mick Krippendorf <> wrote:
    > Jess Austin wrote:
    > > That's nice, but it means that everyone who imports my class will have
    > > to import the monkeypatch of frozenset, as well.  I'm not sure I want
    > > that.  More ruby than python, ne?

    >
    > I thought it was only a toy class?


    Well, I posted a toy, but it's a stand-in for something else more
    complicated. Trying to conserve bytes, you know.
     
    Jess Austin, Oct 29, 2009
    #5
  6. En Wed, 28 Oct 2009 23:12:53 -0300, Jess Austin <>
    escribió:

    >>>> class mySet(set):

    > ... def __eq__(self, other):
    > ... print "called mySet.__eq__()!"
    > ... if isinstance(other, (set, frozenset)):
    > ... return True
    > ... return set.__eq__(self, other)
    > ...
    >
    > Now I want the builtin set and frozenset types to use the new
    > __eq__() with mySet symmetrically.
    >
    >>>> mySet() == set([1])

    > called mySet.__eq__()!
    > True
    >>>> mySet() == frozenset([1])

    > called mySet.__eq__()!
    > True
    >>>> set([1]) == mySet()

    > called mySet.__eq__()!
    > True
    >>>> frozenset([1]) == mySet()

    > False
    >
    > frozenset doesn't use mySet.__eq__() because mySet is not a subclass
    > of frozenset as it is for set. [...failed attempts to inherit from both
    > set and frozenset...]
    > I must redefine __eq__(), and I'd like to be able to compare
    > instances of the class to both set and frozenset instances.


    We know the last test fails because the == logic fails to recognize mySet
    (on the right side) as a "more specialized" object than frozenset (on the
    left side), because set and frozenset don't have a common base type
    (although they share a lot of implementation)

    I think the only way would require modifying tp_richcompare of
    set/frozenset objects, so it is aware of subclasses on the right side.
    Currently, frozenset() == mySet() effectively ignores the fact that mySet
    is a subclass of set.

    --
    Gabriel Genellina
     
    Gabriel Genellina, Oct 30, 2009
    #6
  7. Jess Austin

    Jess Austin Guest

    On Oct 29, 10:41 pm, "Gabriel Genellina" <>
    wrote:
    > We know the last test fails because the == logic fails to recognize mySet  
    > (on the right side) as a "more specialized" object than frozenset (on the  
    > left side), because set and frozenset don't have a common base type  
    > (although they share a lot of implementation)
    >
    > I think the only way would require modifying tp_richcompare of  
    > set/frozenset objects, so it is aware of subclasses on the right side.  
    > Currently, frozenset() == mySet() effectively ignores the fact that mySet  
    > is a subclass of set.


    I don't think even that would work. By the time set_richcompare() is
    called (incidentally, it's used for both set and frozenset), it's too
    late. That function is not responsible for calling the subclass's
    method. It does call PyAnySet_Check(), but only to short-circuit
    equality and inequality for non-set objects. I believe that something
    higher-level in the interpreter decides to call the right-side type's
    method because it's a subclass of the left-side type, but I'm not
    familiar enough with the code to know where that happens. It may be
    best not to sully such generalized code with a special case for
    this.

    I may do some experiments with bytes, str, and unicode, since that
    seems to be an analogous case. There is a basestring type, but at
    this point I don't know that it really helps with anything.

    cheers,
    Jess
     
    Jess Austin, Oct 30, 2009
    #7
  8. En Fri, 30 Oct 2009 17:55:27 -0300, Jess Austin <>
    escribió:

    > On Oct 29, 10:41 pm, "Gabriel Genellina" <>
    > wrote:
    >> We know the last test fails because the == logic fails to recognize
    >> mySet (on the right side) as a "more specialized" object than frozenset
    >> (on the left side), because set and frozenset don't have a common base
    >> type (although they share a lot of implementation)
    >>
    >> I think the only way would require modifying tp_richcompare of
    >> set/frozenset objects, so it is aware of subclasses on the right side.
    >> Currently, frozenset() == mySet() effectively ignores the fact that
    >> mySet is a subclass of set.

    >
    > I don't think even that would work. By the time set_richcompare() is
    > called (incidentally, it's used for both set and frozenset), it's too
    > late. That function is not responsible for calling the subclass's
    > method. It does call PyAnySet_Check(), but only to short-circuit
    > equality and inequality for non-set objects. I believe that something
    > higher-level in the interpreter decides to call the right-side type's
    > method because it's a subclass of the left-side type, but I'm not
    > familiar enough with the code to know where that happens. It may be
    > best not to sully such generalized code with a special case for
    > this.
    >
    > I may do some experiments with bytes, str, and unicode, since that
    > seems to be an analogous case. There is a basestring type, but at
    > this point I don't know that it really helps with anything.


    Looks like in 3.1 this can be done with bytes+str and viceversa, even if
    bytes and str don't have a common ancestor (other than object; basestring
    doesn't exist in 3.x):

    p3> Base = bytes
    p3> Other = str
    p3>
    p3> class Derived(Base):
    .... def __eq__(self, other):
    .... print('Derived.__eq__')
    .... return True
    ....
    p3> Derived()==Base()
    Derived.__eq__
    True
    p3> Base()==Derived()
    Derived.__eq__
    True
    p3> Derived()==Other()
    Derived.__eq__
    True
    p3> Other()==Derived()
    Derived.__eq__ # !!!
    True
    p3> Base.mro()
    [<class 'bytes'>, <class 'object'>]
    p3> Other.mro()
    [<class 'str'>, <class 'object'>]

    The same example with set+frozenset (the one you're actually interested
    in) doesn't work, unfortunately.
    After further analysis, this works for bytes and str because both types
    refuse to guess and compare to each other; they return NotImplemented when
    the right-side operand is not of the same type. And this gives that other
    operand the chance of being called.

    set and frozenset, on the other hand, are promiscuous: their
    tp_richcompare slot happily accepts any set of any kind, derived or not,
    and compares their contents. I think it should be a bit more strict: if
    the right hand side is not of the same type, and its tp_richcompare slot
    is not the default one, it should return NotImplemented. This way the
    other type has a chance to be called.

    --
    Gabriel Genellina
     
    Gabriel Genellina, Nov 1, 2009
    #8
  9. Jess Austin

    Jess Austin Guest

    On Nov 1, 1:13 am, "Gabriel Genellina" <> wrote:
    > Looks like in 3.1 this can be done with bytes+str and viceversa, even if  
    > bytes and str don't have a common ancestor (other than object; basestring  
    > doesn't exist in 3.x):
    >
    > p3> Base = bytes
    > p3> Other = str
    > p3>
    > p3> class Derived(Base):
    > ...   def __eq__(self, other):
    > ...     print('Derived.__eq__')
    > ...     return True
    > ...
    > p3> Derived()==Base()
    > Derived.__eq__
    > True
    > p3> Base()==Derived()
    > Derived.__eq__
    > True
    > p3> Derived()==Other()
    > Derived.__eq__
    > True
    > p3> Other()==Derived()
    > Derived.__eq__            # !!!
    > True
    > p3> Base.mro()
    > [<class 'bytes'>, <class 'object'>]
    > p3> Other.mro()
    > [<class 'str'>, <class 'object'>]
    >
    > The same example with set+frozenset (the one you're actually interested  
    > in) doesn't work, unfortunately.
    > After further analysis, this works for bytes and str because both types  
    > refuse to guess and compare to each other; they return NotImplemented when  
    > the right-side operand is not of the same type. And this gives that other  
    > operand the chance of being called.
    >
    > set and frozenset, on the other hand, are promiscuous: their  
    > tp_richcompare slot happily accepts any set of any kind, derived or not,  
    > and compares their contents. I think it should be a bit more strict: if  
    > the right hand side is not of the same type, and its tp_richcompare slot  
    > is not the default one, it should return NotImplemented. This way the  
    > other type has a chance to be called.


    Thanks for this, Gabriel! There seems to be a difference between the
    two cases, however:

    >>> str() == bytes()

    False
    >>> set() == frozenset()

    True

    I doubt that either of these invariants is amenable to modification,
    even for purposes of "consistency". I'm not sure how to resolve this,
    but you've definitely helped me here. Perhaps the test in
    set_richcompare can return NotImplemented in particular cases but not
    in others? I'll think about this; let me know if you come up with
    anything more.

    thanks,
    Jess
     
    Jess Austin, Nov 3, 2009
    #9
  10. En Mon, 02 Nov 2009 22:05:42 -0300, Jess Austin <>
    escribió:
    > On Nov 1, 1:13 am, "Gabriel Genellina" <> wrote:


    >> Looks like in 3.1 this can be done with bytes+str and viceversa, even
    >> if bytes and str don't have a common ancestor (other than object;
    >> basestring doesn't exist in 3.x):
    >> The same example with set+frozenset (the one you're actually interested
    >> in) doesn't work, unfortunately.


    > Thanks for this, Gabriel! There seems to be a difference between the
    > two cases, however:
    >
    >>>> str() == bytes()

    > False
    >>>> set() == frozenset()

    > True
    >
    > I doubt that either of these invariants is amenable to modification,
    > even for purposes of "consistency". I'm not sure how to resolve this,
    > but you've definitely helped me here. Perhaps the test in
    > set_richcompare can return NotImplemented in particular cases but not
    > in others? I'll think about this; let me know if you come up with
    > anything more.


    I think it should return NotImplemented only when the right-hand side
    operand has overriden tp_richcompare. That way, set()==frozenset() would
    still be True. Only when one inherits from set/frozenset AND overrides
    __eq__, set_richcompare should step aside and let the more specific __eq__
    be called (by just returning NotImplemented).

    What is your goal when overriding __eq__ for your new set class? It may
    help building a case for this change; a concrete use case is much better
    than an abstract request.

    --
    Gabriel Genellina
     
    Gabriel Genellina, Nov 3, 2009
    #10
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Shane Hathaway

    __eq__ and __ne__

    Shane Hathaway, Jul 8, 2003, in forum: Python
    Replies:
    0
    Views:
    380
    Shane Hathaway
    Jul 8, 2003
  2. Matteo Dell'Amico

    sets.Set doesn't honour __eq__

    Matteo Dell'Amico, Jul 7, 2004, in forum: Python
    Replies:
    1
    Views:
    342
    David Vaughan
    Jul 22, 2006
  3. www
    Replies:
    2
    Views:
    323
  4. VenturaPOR

    inconvenience latch

    VenturaPOR, Jul 1, 2008, in forum: VHDL
    Replies:
    1
    Views:
    478
    jeppe
    Jul 2, 2008
  5. Warren Tang
    Replies:
    0
    Views:
    213
    Warren Tang
    Oct 24, 2011
Loading...

Share This Page