Enumeration idioms: Values from different enumerations

[Ben Finney]

[quoting private email with permission]

I just downloaded your enum module for python [from the Cheeseshop]
and played a bit with it. IMO some of the behaviour makes it less
usefull.

Feedback is appreciated. I'm hoping to provide a "one obvious way" to
do enumerations in Python.

Traceback (most recent call last):
File "<stdin>", line 1, in ?
File "enum.py", line 100, in __cmp__
raise EnumValueCompareError(self, other)
enum.EnumValueCompareError: Not values from the same enumeration:

type(day.mon) == type(col.red) True
type(day.mon) is type(col.red) True
lst= [day.mon, col.red]
day.fri in lst

As can be seen from the above, you raise an exception when one wants
to compare Enums from different enumarations, but it seems a bit
strange that different enumerations belong to the same type.

This does seem a point that could be confusing. Would it be better if
every Enum instance had its own unique subclass of EnumValue, that was
used to instantiate values for that enumeration?

I also think it would be more usefull if enums from different
enumerations just tested unequal.

My rationale for this is: The only purpose of an enumeration is to
have a set of arbitrary unique values within that enumeration. The
values make no sense in the context of a different enumeration, so I
see three different comparison results:
[...]
enum.EnumValueCompareError: Not values from the same enumeration

However, I'm aware that this is not how other Python types behave:
False

Is there a semantic difference between these cases? Is that difference
enough to want different behaviour?

Is there some behaviour other than "evaluate to False" or "raise an
exception", that could indicate "not comparable"?

Because as it is you can't put elements from different enumerations
in a list and check with the in operator if a specific enum value is
in the list.

It could also cause problems for dictionaries, since keys in
dictionaries can be tested against a key giving in a subscription.

These are valid concerns. I can't see how to reconcile these against
the desire for values from different enums to fail comparison.

Am I alone in my tri-state view of enumeration value comparisons? Can
anyone else defend why values from different enumerations should not
compare?

 

[Peter Hansen]

These are valid concerns. I can't see how to reconcile these against
the desire for values from different enums to fail comparison.

Am I alone in my tri-state view of enumeration value comparisons? Can
anyone else defend why values from different enumerations should not
compare?

While I'm largely a disinterested bystander, it occurs to me that if you
look at the reasons behind wanting enumerations in the first place, you
might find a good reason to defend this view.

For example, if enumerations are intended to reduce the likelihood of
certain types of errors (where the use of typical XXXX=3 "constants"
might be more prone to errors), then perhaps this suggests that passing
errors silently is bad. That is, trying to compare enumerations that
should not be compared *is* an error (raising an exception) *because*
the whole point of enumerations is to avoid errors in such cases.

Or perhaps I'm off the mark as to why people want enumerations. Is it
just a cosmetic consideration? (Yes, I've used and appreciated them
extensively in C and might even pick up Enum for my Python code... I'm
just sort of playing devil's advocate here.)

-Peter

 

[Mike Meyer]

For example, if enumerations are intended to reduce the likelihood of
certain types of errors (where the use of typical XXXX=3 "constants"
might be more prone to errors), then perhaps this suggests that
passing errors silently is bad. That is, trying to compare
enumerations that should not be compared *is* an error (raising an
exception) *because* the whole point of enumerations is to avoid
errors in such cases.

Except it might not be an error. For instance, if I've got a list of
enum objects taken from various types (say I've got one set of enums
for days of the week, another for months of the year, and so on, and
which I use depends on whether the user wants to select days of the
week, months of the year, etc), it makes perfect sense to want to know
if a specific enum value is in the list, and the obvious way to check
it is with "my_value in enum_list". That won't work if you raise an
exception - it takes a relatively convoluted bit of code to make this
test.

Python generally uses '==' to mean "is the same value". To do that, a
simple true/false return is enough. In raising an exception, you're
making '==' carry an extra meaning (I'm not sure *what* that is,
though). Do any python builtins behave that way? How about anything in
the python standard library?

<mike

 

[Antoon Pardon]

[quoting private email with permission]

I just downloaded your enum module for python [from the Cheeseshop]
and played a bit with it. IMO some of the behaviour makes it less
usefull.

Feedback is appreciated. I'm hoping to provide a "one obvious way" to
do enumerations in Python.

from enum import Enum
day = Enum('mon', 'tue', 'wed', 'thu', 'fri', 'sat', 'sun')
col = Enum('red', 'green', 'blue')
day.wed == col.blue

Traceback (most recent call last):
File "<stdin>", line 1, in ?
File "enum.py", line 100, in __cmp__
raise EnumValueCompareError(self, other)
enum.EnumValueCompareError: Not values from the same enumeration:

type(day.mon) == type(col.red) True
type(day.mon) is type(col.red) True
lst= [day.mon, col.red]
day.fri in lst

As can be seen from the above, you raise an exception when one wants
to compare Enums from different enumarations, but it seems a bit
strange that different enumerations belong to the same type.

This does seem a point that could be confusing. Would it be better if
every Enum instance had its own unique subclass of EnumValue, that was
used to instantiate values for that enumeration?

If you decide on keeping the current behaviour when comparing
values of different enumerations, I would definitely answer
yes.

I also think it would be more usefull if enums from different
enumerations just tested unequal.

My rationale for this is: The only purpose of an enumeration is to
have a set of arbitrary unique values within that enumeration. The
values make no sense in the context of a different enumeration, so I
see three different comparison results:
[...]
enum.EnumValueCompareError: Not values from the same enumeration

However, I'm aware that this is not how other Python types behave:
False

Is there a semantic difference between these cases? Is that difference
enough to want different behaviour?

Is there some behaviour other than "evaluate to False" or "raise an
exception", that could indicate "not comparable"?

This is a difficult question, because AFAIU python doesn't have clear
guidelines for this. I also have the impression that comparisons are
used for two different reasons.

The first is compare things in a mathematical kind of way in which
how things are compared is important.

The second is more an ordering mechanisme. Whether a string is greater
than an int or vice versa, is not imporant what is important is that
this order is consistent and can be used in binary searches, trees
and other structures to organise data.

Now python has only one collection of compare operators which can
cause conflicts. For instance, one could impose an order on sets
to use for such structure but that goes against the normal order
on sets which is based on subsets.

The only advise I have here is look at PEP 3000 that states:

Comparisons other than == and != between disparate types will raise
an exception unless explicitly supported by the type.

Which seem to imply that == and != should not raise an exception
but >, >=, <, <= should.

 

[Ben Finney]

That is, [perhaps] trying to compare enumerations that should not
be compared *is* an error (raising an exception) *because* the
whole point of enumerations is to avoid errors in such cases.

Except it might not be an error. For instance, if I've got a list of
enum objects taken from various types (say I've got one set of enums
for days of the week, another for months of the year, and so on, and
which I use depends on whether the user wants to select days of the
week, months of the year, etc), it makes perfect sense to want to know
if a specific enum value is in the list, and the obvious way to check
it is with "my_value in enum_list". That won't work if you raise an
exception - it takes a relatively convoluted bit of code to make this
test.

The 'enum' package in Cheeseshop doesn't do that. Enumerations are
sequences (of unique arbitrary values), that can be iterated and tested
for membership.

What's being discussed here is what happens when comparing the *values*
from the enumeration.

Python generally uses '==' to mean "is the same value". To do that,
a simple true/false return is enough. In raising an exception,
you're making '==' carry an extra meaning (I'm not sure *what* that
is, though).

The problem with "is the same value" as an explanation for '==' is
that it doesn't help in cases such as::

What should be the result of this comparison::

Are they "the same value"? They're both "small" (and they both coerce
to the same string value, and in this case the same integer value).

If not, is 'False' the right way to indicate that?

Or is it an error to even try comparing them?

Do any python builtins behave that way? How about anything in the
python standard library?

No to both; I believe this may be a defining property of
enumerations. Am I wrong?

 

[Antoon Pardon]

That is, [perhaps] trying to compare enumerations that should not
be compared *is* an error (raising an exception) *because* the
whole point of enumerations is to avoid errors in such cases.

Except it might not be an error. For instance, if I've got a list of
enum objects taken from various types (say I've got one set of enums
for days of the week, another for months of the year, and so on, and
which I use depends on whether the user wants to select days of the
week, months of the year, etc), it makes perfect sense to want to know
if a specific enum value is in the list, and the obvious way to check
it is with "my_value in enum_list". That won't work if you raise an
exception - it takes a relatively convoluted bit of code to make this
test.

The 'enum' package in Cheeseshop doesn't do that. Enumerations are
sequences (of unique arbitrary values), that can be iterated and tested
for membership.

Sure but we do have this:

from enum import Enum
day = Enum('mon', 'tue', 'wed', 'thu', 'fri', 'sat', 'sun')
col = Enum('red', 'green', 'blue')
lst= [day.mon, col.red]
col.blue in lst

Traceback (most recent call last):
File "<stdin>", line 1, in ?
File "enum.py", line 100, in __cmp__
raise EnumValueCompareError(self, other)
enum.EnumValueCompareError: Not values from the same enumeration:

What's being discussed here is what happens when comparing the *values*
from the enumeration.


The problem with "is the same value" as an explanation for '==' is
that it doesn't help in cases such as::


What should be the result of this comparison::


Are they "the same value"? They're both "small" (and they both coerce
to the same string value, and in this case the same integer value).

If not, is 'False' the right way to indicate that?

I would agree 'False' is the right answer here.

 

[Ben Sizer]

The problem with "is the same value" as an explanation for '==' is
that it doesn't help in cases such as::


What should be the result of this comparison::


Are they "the same value"? They're both "small" (and they both coerce
to the same string value, and in this case the same integer value).

Is it possible to make it have the following sort of behaviour? :
False

It works for comparing a boolean (True) vs. an integer (1), so it has
some sort of precedent. (Especially if you make the tenuous assumption
that True,False are language-supported 'enums' for 0 and 1.)

 

[Antoon Pardon]

Is it possible to make it have the following sort of behaviour? :

False

It works for comparing a boolean (True) vs. an integer (1), so it has
some sort of precedent. (Especially if you make the tenuous assumption
that True,False are language-supported 'enums' for 0 and 1.)

I'm against it. I don't like the following returning True:

ShirtSize.small in [ShirtSize.Medium, AppleSize.small]


I also think it may cause problems with other comparisons.

Supose the following:

col = Enum('red', 'green', 'blue')
paint = Enum('violet' , 'blue', 'red')

Then we get the following situation:

col.red == paint.red and col.blue == paint.blue

but

col.red < col.blue and paint.blue < paint.red

 

[Ben Finney]

The 'enum' package in Cheeseshop [defines enumerations as]
sequences (of unique arbitrary values), that can be iterated and
tested for membership.

Sure but we do have this:

from enum import Enum
day = Enum('mon', 'tue', 'wed', 'thu', 'fri', 'sat', 'sun')
col = Enum('red', 'green', 'blue')
lst= [day.mon, col.red]
col.blue in lst

[...]
enum.EnumValueCompareError: Not values from the same enumeration:

Yes, that looks like a case in favour of allowing at least equal-value
and not-equal-value comparisons.

I would agree 'False' is the right answer here.

If you decide on keeping the current behaviour when comparing
values of different enumerations, I would definitely answer
yes.

If values from different enums were to be compared for equality (and
not raise an exception), would it still be good to have those values
be of different types?

This is a difficult question, because AFAIU python doesn't have
clear guidelines for [comparisons between differing types]. I also
have the impression that comparisons are used for two different
reasons.

The first is compare things in a mathematical kind of way in which
how things are compared is important.

The second is more an ordering mechanisme. Whether a string is
greater than an int or vice versa, is not imporant what is important
is that this order is consistent and can be used in binary searches,
trees and other structures to organise data.

Now python has only one collection of compare operators which can
cause conflicts. For instance, one could impose an order on sets to
use for such structure but that goes against the normal order on
sets which is based on subsets.

The only advise I have here is look at PEP 3000 that states:

Comparisons other than == and != between disparate types will raise
an exception unless explicitly supported by the type.

Which seem to imply that == and != should not raise an exception but

This seems like a good guide for a policy on enumerations.

How about this proposal:

Enumerations (instances of Enum) contain a fixed sequence of unique,
arbitrary values. The values within an enumeration are instances of a
type (subclass of EnumValue), unique to that enumeration.

Values from an enumeration can be compared for equality (__eq__,
__ne__) with any other value, and will only return True for the same
value from the same enumeration, otherwise False. Other comparison
operations can be performed only with values from the same
enumeration, otherwise an exception (EnumValueCompareError, subclass
of ValueError) is raised.

This might result in the following simplistic implementation::

class EnumValue(object):
# ...

def __eq__(self, other):
result = False
if self is other:
result = True
return result

def __cmp__(self, other):
if isinstance(other, EnumValue):
if is_same_enumeration(self, other):
result = cmp(self.index, other.index)
else:
raise EnumValueCompareError(self, other)
return result

This gives meaning to the "equal value" comparisons, but ensures that
other comparisons are errors.

Comments so far?

 

[Paul Rubin]

This gives meaning to the "equal value" comparisons, but ensures that
other comparisons are errors.

Comments so far?

What does copy.copy of an enumeration value do? What happens if you
have a list with some enumeration values inside, and you make a
deepcopy of it? What happens if you pickle an enum, then unpickle the
pickle and compare what comes out to the other enum?

All in all, comparing by object identity doesn't sound too good.
Maybe you want to have the enum contain its own name internally, and
do a string comparison.

 

[Antoon Pardon]

If values from different enums were to be compared for equality (and
not raise an exception), would it still be good to have those values
be of different types?

Yes, because people who would need a quick and dirty ordering
could then use the type of the object as a first key.

This seems like a good guide for a policy on enumerations.

How about this proposal:

Enumerations (instances of Enum) contain a fixed sequence of unique,
arbitrary values. The values within an enumeration are instances of a
type (subclass of EnumValue), unique to that enumeration.

Values from an enumeration can be compared for equality (__eq__,
__ne__) with any other value, and will only return True for the same
value from the same enumeration, otherwise False. Other comparison
operations can be performed only with values from the same
enumeration, otherwise an exception (EnumValueCompareError, subclass
of ValueError) is raised.

Looks good to me.

This might result in the following simplistic implementation::

class EnumValue(object):
# ...

def __eq__(self, other):
result = False
if self is other:
result = True
return result

def __cmp__(self, other):
if isinstance(other, EnumValue):
if is_same_enumeration(self, other):
result = cmp(self.index, other.index)
else:
raise EnumValueCompareError(self, other)
return result

This gives meaning to the "equal value" comparisons, but ensures that
other comparisons are errors.

Comments so far?

Isn't there an else missing? I have the impression that if you compare
two enums from a different enumeration __cmp__ will raise UnboundLocalError.

 

[Ben Sizer]

Is it possible to make it have the following sort of behaviour? :

False

It works for comparing a boolean (True) vs. an integer (1), so it has
some sort of precedent. (Especially if you make the tenuous assumption
that True,False are language-supported 'enums' for 0 and 1.)

I'm against it. I don't like the following returning True:

ShirtSize.small in [ShirtSize.Medium, AppleSize.small]

I agree to an extent. I can see that being unwanted behaviour, although
not really a big one, and no worse than in C++. I think that when you
have a list of alternatives like that, they're either hard-coded by
selectively picking from the enumeration's initialisation list, or by
being generated according to some other criteria. Either way, it would
be hard to end up with the wrong type of value into that list, I think.

I also think it may cause problems with other comparisons.

Supose the following:

col = Enum('red', 'green', 'blue')
paint = Enum('violet' , 'blue', 'red')

Then we get the following situation:

col.red == paint.red and col.blue == paint.blue

but

col.red < col.blue and paint.blue < paint.red

I don't think that's a problem - does any other language make any
guarantees on ordering across multiple enumerations? Transitivity
within any single enumeration plus transivity of equivalence across
multiple enumerations, should be enough for most needs, no?

 

[skip]

Without downloading and installing your code, can you tell me what the
result of these comparisons would be?

col = Enum('red', 'green', 'blue')
day = Enum('mon', 'tue', 'wed', 'thu', 'fri', 'sat', 'sun')
col.blue == "blue"
day.tue == 23

If they return False I would expect

col.blue == day.tue

to return False as well, not raise an exception.

Skip

 

[skip]

Ben> What should be the result of this comparison::

False. They are values from different objects. Just make __eq__ map to
"is". I think you'll be fine.

Ben> Or is it an error to even try comparing them?

As someone else pointed out containment tests become difficult with your
current formulation.

Ben> No to both; I believe this may be a defining property of
Ben> enumerations. Am I wrong?

I think so. <0.5 wink>. I think you should be able to compare any two
objects. I think the complex types give you a little wiggle room on the
size comparisons (<, <=, >, >=), but I think == and != really ought to work.

Skip

 

[Mike Meyer]

That is, [perhaps] trying to compare enumerations that should not
be compared *is* an error (raising an exception) *because* the
whole point of enumerations is to avoid errors in such cases.

Except it might not be an error. For instance, if I've got a list of
enum objects taken from various types (say I've got one set of enums
for days of the week, another for months of the year, and so on, and
which I use depends on whether the user wants to select days of the
week, months of the year, etc), it makes perfect sense to want to know
if a specific enum value is in the list, and the obvious way to check
it is with "my_value in enum_list". That won't work if you raise an
exception - it takes a relatively convoluted bit of code to make this
test.

What's being discussed here is what happens when comparing the *values*
from the enumeration.

That's what I thought I was discussing, but apparently I wasn't clear
enough. Let me try again.

I think it's perfectly reasonable to store enum values from different
enums in a list, and check for a specific value being in that list. If
comparing two enum values can raise an exception, then doinng this
become problematic, as you may get an exception. According to what you
say below, this isn't true for any builtin type or any type in the
standard library.

The problem with "is the same value" as an explanation for '==' is
that it doesn't help in cases such as::

Actually, it provides a very explicit guideline for these cases. The
guidelines is by no means obvious, as witness languages like LISP,
which have multiple equality operators.

What should be the result of this comparison::
Are they "the same value"? They're both "small" (and they both coerce
to the same string value, and in this case the same integer value).

That depends on the semantics you want to place on the values of
enums. Reasonable arguments can be made for anything from "they're
exactly like small integers" to "the values are an implementation
detail, and you shouldn't worry about them." That they coerce to the
same string is irreleevant, at least in python, which doesn't coerce
strings to

No to both; I believe this may be a defining property of
enumerations. Am I wrong?

There are enum implementations that don't treat comparing enum values
from different enums as an error, so I'd say you're wrong. In Python's
case, comparing two objects for identity never raises an exception for
any type that comes with the language, so I'd say that that was a
defining property of python.

<mike

 

[Steven D'Aprano]

This does seem a point that could be confusing. Would it be better if
every Enum instance had its own unique subclass of EnumValue, that was
used to instantiate values for that enumeration?

That seems to be creating complication for its own sake.

I also think it would be more usefull if enums from different
enumerations just tested unequal.

My rationale for this is: The only purpose of an enumeration is to
have a set of arbitrary unique values within that enumeration. The
values make no sense in the context of a different enumeration, so I
see three different comparison results:
[...]
enum.EnumValueCompareError: Not values from the same enumeration

However, I'm aware that this is not how other Python types behave:
False

Is there a semantic difference between these cases? Is that difference
enough to want different behaviour?

I don't believe so.

It seems reasonable to me for other comparisons such as __lt__ etc.
to fail unless the objects belong to the same enumeration. But equality
and inequality are special. Unordered objects support equality testing.

The question whether Mohammad is greater than or less than the mountain
has no answer, because we don't know how to order the two (by height? by
mass? by religious belief? by economic value? by importance to human
history?). But we can certainly answer the question is Mohammad equal to
the mountain -- the answer is certainly no, no matter how we order them.

These are valid concerns. I can't see how to reconcile these against
the desire for values from different enums to fail comparison.

Perhaps you are just being over-strict -- soon you'll be writing in Pascal


Failing order comparisons is reasonable, but why refuse to test for
equality? If you ask "is Monday equal to Blue?" the intuitive answer is
"no", not "my brain just crashed trying to evaluate the answer!".

 

[Steven D'Aprano]

While I'm largely a disinterested bystander, it occurs to me that if you
look at the reasons behind wanting enumerations in the first place, you
might find a good reason to defend this view.

For example, if enumerations are intended to reduce the likelihood of
certain types of errors (where the use of typical XXXX=3 "constants"
might be more prone to errors), then perhaps this suggests that passing
errors silently is bad. That is, trying to compare enumerations that
should not be compared *is* an error (raising an exception) *because*
the whole point of enumerations is to avoid errors in such cases.

In my opinion, rather than make all enums fail equality testing, two
better solutions would be:

(1) sub-class Enum to have StrictEnum. Enum allows equality testing for
normal Pythonic behaviour. StrictEnum fails equality testing for extra B&D
behaviour.

(2) Expect the user to make an explicit test when they want the stricter
testing. At the moment two different enumerations have the same class, but
Ben could (perhaps) hash the enumeration values to get a unique ID that
distinguishes one enumeration from another. Or some other strategy that
would allow something like:

if value.from_same_enumeration(other) and value == other

for value and other both enums.

Of the two, I suspect that (1) is far less work for both Ben and the
people using his class, although for introspection purposes it might be
nice to have some sort of unique enumeration ID. That could even be
something as simple as a tuple of the enumeration values.

 

[Steven D'Aprano]

The problem with "is the same value" as an explanation for '==' is
that it doesn't help in cases such as::


What should be the result of this comparison::


Are they "the same value"? They're both "small" (and they both coerce
to the same string value, and in this case the same integer value).

I don't think anyone seriously intends to argue that ShirtSize.small
should test equal to AppleSize.small.

If not, is 'False' the right way to indicate that?

Why not? The intuitive answer is that they are "not equal".

In the case of apples and shirts, the intuitive answer can be misleading
because if you had a small enough shirt (say, for a Barbi doll) one might
be tempted to say "Yes, they are the same size".

A better example might be StringLength.long == BookSize.long. Both coerce
to the same string, but one measures physical length and the other
measures number of words. They are incompatible measures, and can't be
ordered. A strict physicist might argue that asking "is five kilograms
equal to three metres?" is a nonsensical question, but that's not the
Python way.

(And an even better physicist will answer that kilograms and metres can
not only be tested for equality, but can be ordered -- there are
mathematical transformations, common in relativistic physics, where all
physical quantities including time and mass, are transformed into lengths.
These physicists calculated *everything* in terms of dimensionless values
and multiples of length, length squared, length cubed, etc.)

Or is it an error to even try comparing them?

Perhaps there are unusual cases where comparing strings and ints for
equality should be an error. There may even be unusual cases where
comparing ints and floats should be an error. But I don't believe that
this should be the default behaviour, and it isn't the Python way.

Likewise for enums from different enumerations. By default they should
evaluate unequal, not raise an exemption.

 

[Steven D'Aprano]

Is it possible to make it have the following sort of behaviour? :

True

Okay, so I was wrong to say that nobody was seriously suggesting that sort
of behaviour.

It works for comparing a boolean (True) vs. an integer (1), so it has
some sort of precedent. (Especially if you make the tenuous assumption
that True,False are language-supported 'enums' for 0 and 1.)

Enums are not conceptually subclasses of integers. Integers just happen to
be a useful method to implement enumerations.

 

[Ben Finney]

All in all, comparing by object identity doesn't sound too good.
Maybe you want to have the enum contain its own name internally, and
do a string comparison.

The "__eq__ compares identity" was a glib pseudo-implementation; I
didn't think it through. The existing 'enum' implementation doesn't
use identity, and future ones won't either.

The intended meaning was more like:

class EnumValue(object):
# ...
def __eq__(self, other):
result = compare_attribute_values(self, other)
return result

In other words, identity doesn't matter, but every value from every
enumeration will compare inequal to each other. If an EnumValue
instance is copied, the copies will compare equal.