Python 3 __cmp__ semantic change?

J

Johannes Bauer

Hello group,

I'm porting some code of mine to Python 3. One class has the __cmp__
operator overloaded, but comparison doesn't seem to work anymore with that:

Traceback (most recent call last):
File "./parse", line 25, in <module>
print(x < y)
TypeError: unorderable types: IP() < IP()

Was there some kind of semantic change?

Kind regards,
Johannes
 
I

Inyeol.Lee

Hello group,

I'm porting some code of mine to Python 3. One class has the __cmp__
operator overloaded, but comparison doesn't seem to work anymore with that:

Traceback (most recent call last):
  File "./parse", line 25, in <module>
    print(x < y)
TypeError: unorderable types: IP() < IP()

Was there some kind of semantic change?

Overload __lt__ method.

Inyeol
 
S

Steve Holden

Johannes said:
Well, of course I could do that, but the python doc says:

"Called by comparison operations if rich comparison (see above) is not
defined."
http://www.python.org/doc/2.5.2/ref/customization.html

And my code works just fine with 2.5 - only on 3.0 it doesn't work
anymore. Why is that?

Well the Python 2.5 documentation can't be regarded as a reliable guide
to what to expect in 3.0 ...

You will observe that __cmp__ no longer appears in the index:

http://docs.python.org/dev/3.0/genindex-_.html

regards
Steve
 
S

Steve Holden

Ben said:
I searched in vain for an official description of this changed
behaviour. Where can we find an official description of how
comparisons are different in Python 3.0?
If it's not present then it would be worth reporting it as a 3.0 bug -
there's still time to get it in, as the release isn't due until early
December.

regards
Steve
 
J

Johannes Bauer

Steve said:
If it's not present then it would be worth reporting it as a 3.0 bug -
there's still time to get it in, as the release isn't due until early
December.

Seems it was removed on purpose - I'm sure there was a good reason for
that, but may I ask why? Instead of the sleek __cmp__ function I had
earlier, I now have code like:


def __lt__(self, other):
return self.__cmp__(other) < 0

def __le__(self, other):
return self.__cmp__(other) < 0

def __gt__(self, other):
return self.__cmp__(other) > 0

def __ge__(self, other):
return self.__cmp__(other) >= 0

Does anyone know the reason why __cmp__ was discarded?

Kind regards,
Johannes
 
T

Terry Reedy

Johannes said:
Seems it was removed on purpose - I'm sure there was a good reason for
that, but may I ask why? Instead of the sleek __cmp__ function I had
earlier, I now have code like:


def __lt__(self, other):
return self.__cmp__(other) < 0

def __le__(self, other):
return self.__cmp__(other) < 0

def __gt__(self, other):
return self.__cmp__(other) > 0

def __ge__(self, other):
return self.__cmp__(other) >= 0

Does anyone know the reason why __cmp__ was discarded?

See previous threads, including recent one about sorting.
 
G

George Sakkis

    Johannes> Seems it was removed on purpose - I'm sure there was a good
    Johannes> reason for that, but may I ask why?

Start here:

    http://www.mail-archive.com/[email protected]/msg11474.html

Also, a comment to this blog post suggests creating a CmpMixin:

   http://oakwinter.com/code/porting-setuptools-to-py3k/

Skip

Dropping __cmp__ without providing implicit or at least easy explicit
[1] total ordering is (was?) a mistake; it opens the door to subtle
bugs or redundant boilerplate code.

[1] E.g. http://code.activestate.com/recipes/576529/
 
H

Hyuga

I hope you actually have <= here.






I think it was because __cmp__ was the backward compatible fallback for
the newer rich comparison methods and Python 3 cleans up a lot of stuff
left in just for backward compatibility. In this case it is a cleanup
too far as in most cases (i.e. those cases where you don't need the full
complexity of the rich comparisons) __cmp__ is a much simpler solution.

Seehttp://mail.python.org/pipermail/python-dev/2003-March/034073.html
for Guido's original thoughts. Also, once upon a time pep-3000 referred
to the removal of __cmp__ but I can't find it in any of the current
peps. Seehttp://mail.python.org/pipermail/python-checkins/2004-August/042959.html
andhttp://mail.python.org/pipermail/python-checkins/2004-August/042972.html
where the reference to removal of __cmp__ became "Comparisons other than
``==`` and ``!=`` between disparate types will raise an exception unless
explicitly supported by the type" and the reference to Guido's email
about removing __cmp__ was also removed.

Guido's primary argument for removing it seems to be that the code for
supporting both __cmp__ and the rich comparisons is "hairy" and that
it felt really satisfying to remove. I don't think that's a good
enough argument. It was hairy because there are a lot of cases to
check, but I wouldn't say it was crufty. It made sense, and the way
it worked seemed logical enough. I never ran into any problems with
it. And by and far the most common case is to implement some total
ordering for a class.

Now, as has been pointed out, all you really need to define total
ordering, at least for sorting, is __eq__ and __lt__, which isn't too
bad. But you still lose the ability to make any other sort of
comparison without implementing all the other comparison operators
too.

Perhaps the code could be made somewhat simpler like this: If rich
comparisons are defined, use those and *only* those operators that are
defined, and don't try to fall back on __cmp__ otherwise. If no rich
comparisons are defined, just look for __cmp__.
 
A

Arnaud Delobelle

Hyuga said:
Guido's primary argument for removing it seems to be that the code for
supporting both __cmp__ and the rich comparisons is "hairy" and that
it felt really satisfying to remove. I don't think that's a good
enough argument. It was hairy because there are a lot of cases to
check, but I wouldn't say it was crufty. It made sense, and the way
it worked seemed logical enough. I never ran into any problems with
it. And by and far the most common case is to implement some total
ordering for a class.

Now, as has been pointed out, all you really need to define total
ordering, at least for sorting, is __eq__ and __lt__, which isn't too
bad. But you still lose the ability to make any other sort of
comparison without implementing all the other comparison operators
too.

As classes can be decorated in Python 3, you can write a decorator to
make a class totally ordered. Here is a very simplified proof of
concept such decorator:

def totally_ordered(cls):
if not hasattr(cls, '__gt__'):
def gt(self, other):
return self != other and not self < other
cls.__gt__ = gt
# Do the same with __le__, __ge__
return cls


@totally_ordered
class Fraction:
def __init__(self, num, den=1):
assert den > 0, "denomintator must be > 0"
self.num = num
self.den = den
def __eq__(self, other):
return self.num*other.den == self.den*other.num
def __lt__(self, other):
return self.num*other.den < self.den*other.num
False

Granted it's not as efficient as a __cmp__ function.
 
S

Steven D'Aprano

On Fri, 21 Nov 2008 17:26:21 +0000, Arnaud Delobelle wrote:

[...]
As classes can be decorated in Python 3, you can write a decorator to
make a class totally ordered. Here is a very simplified proof of
concept such decorator:

def totally_ordered(cls):
if not hasattr(cls, '__gt__'):
def gt(self, other):
return self != other and not self < other
cls.__gt__ = gt
# Do the same with __le__, __ge__
return cls


@totally_ordered
class Fraction:
def __init__(self, num, den=1):
assert den > 0, "denomintator must be > 0" self.num = num
self.den = den
def __eq__(self, other):
return self.num*other.den == self.den*other.num
def __lt__(self, other):
return self.num*other.den < self.den*other.num

False

Granted it's not as efficient as a __cmp__ function.

What makes you say that? What do you mean by "efficient"? Are you talking
about memory footprint, runtime speed, disk-space, programmer efficiency,
algorithmic complexity, or something else?

As I see it, a __cmp__ method would be written something like this:

def __cmp__(self, other):
return cmp(self.num*other.den, self.den*other.num)

which presumably would save you a trivial amount of source code (and
hence memory footprint, disk-space and programmer efficiency), but the
algorithmic complexity is identical and the runtime speed might even be
trivially slower due to the extra function call.

If your major concern is to reduce the amount of repeated code in the
methods, then there's no reason why you can't write a __cmp__ method as
above and then call it from your rich comparisons:

def __eq__(self, other):
return self.__cmp__(other) == 0
def __lt__(self, other):
return self.__cmp__(other) < 0

and if you really want to be concise:

__gt__ = lambda s, o: s.__cmp__(o) > 0
__ge__ = lambda s, o: s.__cmp__(o) >= 0
__le__ = lambda s, o: s.__cmp__(o) <= 0
 
A

Arnaud Delobelle

Steven D'Aprano said:
On Fri, 21 Nov 2008 17:26:21 +0000, Arnaud Delobelle wrote:

[...]
As classes can be decorated in Python 3, you can write a decorator to
make a class totally ordered. Here is a very simplified proof of
concept such decorator:

def totally_ordered(cls):
if not hasattr(cls, '__gt__'):
def gt(self, other):
return self != other and not self < other
cls.__gt__ = gt
# Do the same with __le__, __ge__
return cls


@totally_ordered
class Fraction:
def __init__(self, num, den=1):
assert den > 0, "denomintator must be > 0" self.num = num
self.den = den
def __eq__(self, other):
return self.num*other.den == self.den*other.num
def __lt__(self, other):
return self.num*other.den < self.den*other.num

False

Granted it's not as efficient as a __cmp__ function.

What makes you say that? What do you mean by "efficient"? Are you talking
about memory footprint, runtime speed, disk-space, programmer efficiency,
algorithmic complexity, or something else?

What I'm talking about is very simple - and explained below, with the
help of your __cmp__ method.
As I see it, a __cmp__ method would be written something like this:

def __cmp__(self, other):
return cmp(self.num*other.den, self.den*other.num)

I'm talking about runtime speed (*not* asymptotic complexity). My code
makes Fraction.__gt__ about twice as slow as Fraction.__lt__ or
Fraction.__eq__ even though with __cmp__ they would all be equally fast.
 
S

Steven D'Aprano

What I'm talking about is very simple - and explained below, with the
help of your __cmp__ method.

I'm talking about runtime speed (*not* asymptotic complexity). My code
makes Fraction.__gt__ about twice as slow as Fraction.__lt__ or
Fraction.__eq__ even though with __cmp__ they would all be equally fast.


Sounds like a premature micro-optimization to me. On my machine, running
Python 2.5, the speed difference is nothing like twice as slow.

.... def __init__(self, num, den=1):
.... self.num = num
.... self.den = den
.... def __cmp__(self, other):
.... return cmp(self.num*other.den, self.den*other.num)
........ __lt__ = lambda self, other: self.__cmp__(other) < 0
........ 'from __main__ import UseRichCmp;'
.... 'x=UseRichCmp(3, 5); y=UseRichCmp(1, 2)')
t1.repeat() [3.3418200016021729, 2.4046459197998047, 2.2295808792114258]
t2.repeat()
[3.8954730033874512, 3.0240590572357178, 3.5528950691223145]

There's a slight speed difference, around 35% slower. But the random
variation in speed is almost 50%, so I would conclude from this trial
that there is no *significant* speed difference between the methods.
 
A

Arnaud Delobelle

Steven D'Aprano said:
What I'm talking about is very simple - and explained below, with the
help of your __cmp__ method.

I'm talking about runtime speed (*not* asymptotic complexity). My code
makes Fraction.__gt__ about twice as slow as Fraction.__lt__ or
Fraction.__eq__ even though with __cmp__ they would all be equally fast.


Sounds like a premature micro-optimization to me. On my machine, running
Python 2.5, the speed difference is nothing like twice as slow.

... def __init__(self, num, den=1):
... self.num = num
... self.den = den
... def __cmp__(self, other):
... return cmp(self.num*other.den, self.den*other.num)
...... __lt__ = lambda self, other: self.__cmp__(other) < 0
...... 'from __main__ import UseRichCmp;'
... 'x=UseRichCmp(3, 5); y=UseRichCmp(1, 2)')
t1.repeat() [3.3418200016021729, 2.4046459197998047, 2.2295808792114258]
t2.repeat()
[3.8954730033874512, 3.0240590572357178, 3.5528950691223145]

There's a slight speed difference, around 35% slower. But the random
variation in speed is almost 50%, so I would conclude from this trial
that there is no *significant* speed difference between the methods.

That's not surprising. You're measuring the wrong things. If you read
what I wrote, you'll see that I'm talking about Fraction.__gt__ being
slower (as it is defined in terms of Fraction.__eq__ and
Fraction.__lt__) using when my 'totally_ordered' decorator.

I haven't done any tests but as Fraction.__gt__ calls *both*
Fraction.__eq__ and Fraction.__lt__ it is obvious that it is going to be
roughly twice as slow.
 
A

Arnaud Delobelle

Arnaud Delobelle said:
I haven't done any tests but as Fraction.__gt__ calls *both*
Fraction.__eq__ and Fraction.__lt__ it is obvious that it is going to be
roughly twice as slow.

There's a very simple way of emulating Fraction.__cmp__ in Python 3:

def totally_ordered(cls):
def __lt__(self, other): return self.cmp(other) < 0
def __eq__(self, other): return self.cmp(other) == 0
def __gt__(self, other): return self.cmp(other) > 0
cls.__lt__ = __lt__
cls.__eq__ = __eq__
cls.__gt__ = __gt__
# and same with __le__, __ge__
return cls

@totally_ordered
class Fraction:
def __init__(self, num, den=1):
assert den > 0, "denomintator must be > 0"
self.num = num
self.den = den
def cmp(self, other):
return self.num*other.den - self.den*other.num


It doesn't suffer the speed penalty incurred when defining comparison
operators from __eq__ and __lt__.
 
A

Aahz

There's a very simple way of emulating Fraction.__cmp__ in Python 3:

def totally_ordered(cls):
def __lt__(self, other): return self.cmp(other) < 0
def __eq__(self, other): return self.cmp(other) == 0
def __gt__(self, other): return self.cmp(other) > 0
cls.__lt__ = __lt__
cls.__eq__ = __eq__
cls.__gt__ = __gt__
# and same with __le__, __ge__
return cls

@totally_ordered
class Fraction:
def __init__(self, num, den=1):
assert den > 0, "denomintator must be > 0"
self.num = num
self.den = den
def cmp(self, other):
return self.num*other.den - self.den*other.num


It doesn't suffer the speed penalty incurred when defining comparison
operators from __eq__ and __lt__.

That's true IIF __sub__() is not substantially slower than __cmp__();
however, your basic technique is sound and one can easily rewrite your
cmp() to use some other algorithm.
 
C

Colin J. Williams

Johannes said:
Seems it was removed on purpose - I'm sure there was a good reason for
that, but may I ask why? Instead of the sleek __cmp__ function I had
earlier, I now have code like:


def __lt__(self, other):
return self.__cmp__(other) < 0

def __le__(self, other):
return self.__cmp__(other) < 0

def __gt__(self, other):
return self.__cmp__(other) > 0

def __ge__(self, other):
return self.__cmp__(other) >= 0

Does anyone know the reason why __cmp__ was discarded?

Kind regards,
Johannes

Johannes,

Isn't the problem with your original
post that x and y are either of
different types or, is of the same type
that the values of that type are not
strictly comparable? It seems that the
old approach was to make a recursive
comparison.

Colin W.
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

No members online now.

Forum statistics

Threads
473,777
Messages
2,569,604
Members
45,234
Latest member
SkyeWeems

Latest Threads

Top