Python 3 __cmp__ semantic change?

S

Steven D'Aprano

That's not surprising. You're measuring the wrong things. If you read
what I wrote, you'll see that I'm talking about Fraction.__gt__ being
slower (as it is defined in terms of Fraction.__eq__ and
Fraction.__lt__) using when my 'totally_ordered' decorator.

I haven't done any tests but as Fraction.__gt__ calls *both*
Fraction.__eq__ and Fraction.__lt__ it is obvious that it is going to be
roughly twice as slow.


What's obvious to you and what's obvious to the Python VM are not
necessarily the same thing. I believe you are worrying about the wrong
thing. (BTW, I think your earlier decorator had a bug, in that it failed
to define __ne__ but then called "self != other".) My tests suggest that
relying on __cmp__ is nearly three times *slower* than your decorated
class, and around four times slower than defining all the rich
comparisons directly:

$ python comparisons.py
Testing FractionCmp... 37.4376080036
Testing FractionRichCmpDirect... 9.83379387856
Testing FractionRichCmpIndirect... 16.152534008
Testing FractionDecoratored... 13.2626030445

Test code follows. If I've made an error, please let me know.


[comparisons.py]
from __future__ import division

def totally_ordered(cls):
# Define total ordering in terms of __eq__ and __lt__ only.
if not hasattr(cls, '__ne__'):
def ne(self, other):
return not self == other
cls.__ne__ = ne
if not hasattr(cls, '__gt__'):
def gt(self, other):
return not (self < other or self == other)
cls.__gt__ = gt
if not hasattr(cls, '__ge__'):
def ge(self, other):
return not (self < other)
cls.__ge__ = ge
if not hasattr(cls, '__le__'):
def le(self, other):
return (self < other or self == other)
cls.__le__ = le
return cls


class AbstractFraction:
def __init__(self, num, den=1):
if self.__class__ is AbstractFraction:
raise TypeError("abstract base class, do not instantiate")
assert den > 0, "denomintator must be > 0"
self.num = num
self.den = den
def __float__(self):
return self.num/self.den

class FractionCmp(AbstractFraction):
def __cmp__(self, other):
return cmp(self.num*other.den, self.den*other.num)

class FractionRichCmpDirect(AbstractFraction):
def __eq__(self, other):
return (self.num*other.den) == (self.den*other.num)
def __ne__(self, other):
return (self.num*other.den) != (self.den*other.num)
def __lt__(self, other):
return (self.num*other.den) < (self.den*other.num)
def __le__(self, other):
return (self.num*other.den) <= (self.den*other.num)
def __gt__(self, other):
return (self.num*other.den) > (self.den*other.num)
def __ge__(self, other):
return (self.num*other.den) >= (self.den*other.num)

class FractionRichCmpIndirect(AbstractFraction):
def __cmp__(self, other):
return cmp(self.num*other.den, self.den*other.num)
def __eq__(self, other):
return self.__cmp__(other) == 0
def __ne__(self, other):
return self.__cmp__(other) != 0
def __lt__(self, other):
return self.__cmp__(other) < 0
def __le__(self, other):
return self.__cmp__(other) <= 0
def __gt__(self, other):
return self.__cmp__(other) > 0
def __ge__(self, other):
return self.__cmp__(other) >= 0


class FractionDecoratored(AbstractFraction):
def __eq__(self, other):
return self.num*other.den == self.den*other.num
def __lt__(self, other):
return self.num*other.den < self.den*other.num

FractionDecoratored = totally_ordered(FractionDecoratored)

def test_suite(small, big):
assert small < big
assert small <= big
assert not (small > big)
assert not (small >= big)
assert small != big
assert not (small == big)

from timeit import Timer

test = 'test_suite(p, q)'
setup = '''from __main__ import %s as frac
from __main__ import test_suite
p = frac(1, 2)
q = frac(4, 5)
assert float(p) < float(q)'''

for cls in [FractionCmp, FractionRichCmpDirect, FractionRichCmpIndirect,
FractionDecoratored]:
t = Timer(test, setup % cls.__name__)
print "Testing %s..." % cls.__name__,
best = min(t.repeat())
print best
[end comparisons.py]
 
A

Arnaud Delobelle

Steven D'Aprano said:
What's obvious to you and what's obvious to the Python VM are not
necessarily the same thing. I believe you are worrying about the wrong
thing.

All I was asserting was that using my decorator, Fraction.__gt__ would
be roughly twice as slow as Fraction.__eq__ or Fraction.__lt__. I was
not worried about it at all! Your tests below, although very
interesting, don't shed any light on this.
(BTW, I think your earlier decorator had a bug, in that it failed to
define __ne__ but then called "self != other".)

That would be true for Python 2.x but I'm explicitly writing code for
Python 3 here, which, IIRC, infers != correctly when you define ==. I
can't refer you to the docs because my internet access to some US sites
seems to be partly broken ATM, but here's a simple example:

Python 3.0rc1+ (py3k:66521, Sep 21 2008, 07:58:29)
[GCC 4.0.1 (Apple Inc. build 5465)] on darwin
Type "help", "copyright", "credits" or "license" for more information..... def __init__(self):
.... self
.... .... def __init__(self, x):
.... self.x = x
.... def __eq__(self, other):
.... return self.x == other.x
....
My tests suggest that relying on __cmp__ is nearly three times
*slower* than your decorated class, and around four times slower than
defining all the rich comparisons directly:

$ python comparisons.py
Testing FractionCmp... 37.4376080036
Testing FractionRichCmpDirect... 9.83379387856
Testing FractionRichCmpIndirect... 16.152534008
Testing FractionDecoratored... 13.2626030445

Test code follows. If I've made an error, please let me know.

[snip test code]

If anything these tests make a retroactive case for getting rid of
__cmp__ as it seems really slow. Even FractionRichCmpIndirect (which is
the same as applying my second totally_ordered decorator to FractionCmp)
is almost twice as fast as FractionCmp. I would guess it is because
when doing e.g.

a < b

The VM will first look for

a.__lt__(b)

and fail, wasting some time in the process. Then it would look for

a.__cmp__(b)

Whereas when __lt__ is explicitely defined, the first step always
succeeds.
 
A

Arnaud Delobelle

Arnaud Delobelle said:
(BTW, I think your earlier decorator had a bug, in that it failed to
define __ne__ but then called "self != other".)

That would be true for Python 2.x but I'm explicitly writing code for
Python 3 here, which, IIRC, infers != correctly when you define ==. I
can't refer you to the docs because my internet access to some US sites
seems to be partly broken ATM, but here's a simple example:

Python 3.0rc1+ (py3k:66521, Sep 21 2008, 07:58:29)
[GCC 4.0.1 (Apple Inc. build 5465)] on darwin
Type "help", "copyright", "credits" or "license" for more information.... def __init__(self):
... self
... ... def __init__(self, x):
... self.x = x
... def __eq__(self, other):
... return self.x == other.x
...

Getting increasingly frustrated by my inability to reach python.org,
I've compiled the html docs for python 3 in order to find the bit that
explains the behaviour above. But the docs say exactly the opposite:

There are no implied relationships among the comparison
operators. The truth of x==y does not imply that x!=y is
false. Accordingly, when defining __eq__(), one should also define
__ne__() so that the operators will behave as expected. See the
paragraph on __hash__() for some important notes on creating
hashable objects which support custom comparison operations and are
usable as dictionary keys.

So how did I get it into my head that defining __eq__ would create the
correct behaviour for __ne__ automatically? And more puzzlingly, how
come it is what actually happens? Which should I believe: the
documentation or the implementation?
 
A

Aahz

I can't refer you to the docs because my internet access to some US
sites seems to be partly broken ATM, but here's a simple example:

BTW, python.org is hosted at XS4ALL in the Netherlands, so if you can't
reach it, it has nothing to do with your US access.
 
A

Arnaud Delobelle

Terry Reedy said:
http://bugs.python.org/issue4395

ps to Arnaud: upgrade to rc3, which has bug fixes and many doc changes.

This occured to me after I posted the example so I did update before
building the docs. When I saw the inconsistency between the docs and
python, I also rebuilt python but it behaved the same.

I didn't try checking bugs.python.org because I can't access python.org
at the moment (thanks Aahz for pointing out it's hosted in the
Netherlands - as there are a number of US hosted sites that I can't
access I just assumed Python was one of them). I guess the bug report
is about updating the docs!
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

Forum statistics

Threads
473,792
Messages
2,569,639
Members
45,353
Latest member
RogerDoger

Latest Threads

Top