E
Eric Snow
This is a long post, so to sum up:
Is it bad to set __abstractmethod__ on non-functions in order to
trigger the ABC abstractness checks? If not, are __isabstractmethod__
on objects and __abstractmethods__ on classes misleading names?
I don't mean to imply that Python has it wrong. On the contrary, I
typically start by assuming that Python has it right and then try to
figure out what I am missing [1]. Without further ado, here's a
little more meat so that my questions have a better context.
Background
Last week I jumped into an tracker issue that has been developing for
some time [2]. The issue involves the interplay between abstract base
classes and descriptors (like property). Keep in mind that the issue
is only the catalyst for my question here, and my question is a
tangent. If possible I would rather any discussion here center on
that, and anything meaningful to the issue be put in the tracker.
Perhaps I did not think it through, but I made a point that the term
abstractmethod is potentially misleading. I'm hoping to get some
feedback on that from you. The reason I bring it up here is because
of the brief response I got in the ticket. I did not want to further
side-track the discussion going on there, so I'm bringing it here
instead.
In the issue Nick explained the following:
Regarding the naming, @abstractmethod and __isabstractmethod__ are
definitely about methods, or method-based descriptors (such as
property).
There *could* be a case to be made to allow for abstract data
attributes,
but really, using an abstract property should cover that use case well
enough.
Nick is a super smart guy, has probably 100x the experience I do, and,
from my experience with him, simply <gets it> a ton better than I do,
so I trust what he says. Also, he was involved with all this when
abstract base classes were added to the language [3]. From what Nick
said and from the name used, I'm guessing that the motivation was to
have the abstractness of ABCs focused explicitly on methods.
Motivation
The reason I brought it up in the tracker issue is that I've taken to
using __isabstractmethod__ on non-methods. The code in object_new
(typeobject.c) does not care about the types of a class's attributes
[4]. Neither does the code in ABCMeta (abc.py) [5]. All that matters
is that an attribute has __abstractmethod__ set to True to indicate
that the class is abstract. So with that in mind, I use it on
non-function attributes that I want to be abstract [6].
Nick pointed out that you can use an abstract property to accomplish
this same thing. That's what I had been doing for quite a while.
However, I was uncomfortable using an abstract property to describe
something that I did not expect to be implemented as a property. But
that may be just me.
Keep in mind that ABCMeta and typeobject.c do not do any "signature
checking" of objects, so the implementation of an abstract attribute
in a subclass can be any object. All that matters is that the
matching name be bound on the namespace of the subclass. Thus, an
abstract method does not have to be implemented with a method, nor an
abstract property with a property. However, the key point is that the
abstracting decorators provide an indicator of the expectation of the
implementation [7].
Questions
So this all comes back to those two questions. Before jumping into
tracker issue, I had not considered that my use of
__isabstractmethod__ may be contrary to the expected use. Is it? If
not, then perhaps __isabstractmethod__ (on objects) would be better
called __isabstractattribute__; and __abstractmethods__ (on classes)
would be better called __abstractattributes__ [8].
I'm certainly not saying that we should jump up and change them if
they are a potentially incomplete description of their usage, which
they may not be. The fact is, Python lets you do things that many
other languages would not allow, with the stipulation that "we're all
consenting adults"--so if you step outside the lines of expected use,
you're on your own (that's one awesome aspect of Python). Thus, even
if my use case is valid, it may be of such small impact relative to
abstract methods that it makes more sense to use "method" in the
names.
So again, do you think my use of __abstractmethod__ with non-functions
is appropriate? If so, is it a valid enough use case of abstractness
that using "method" in those two attribute names should change? My
gut says yes to the first and no to the second, but I want to hear
what you think!
-eric
p.s. Sorry for the length of this message. Not only do I get
long-winded every once in a while <wink> but I wanted to give this a
thorough treatment.
[1] One thing that I love about Python is that it is driven by really
smart yet practical people that want an easy-to-use, powerful
language. The design is not driven by pure language theory. But it's
not driven by "let's just get it done". It is a powerful mix.
Features are not added unless they satisfy a real problem that can't
be met using existing features without a Python-feature-threshold
amount of work. Usually the problem must be widely (enough) spread in
the Python community, as demonstrated by a large number of uses in the
wild or a widely used third-party package that fills the gap. Even
then, enough core developers have to agree that the problem is big
enough to warrant changing the language. Finally, someone has to do
the work. So the more I saw what went into Python, the less I
questioned it.
[2] http://bugs.python.org/issue11610#msg138189
[3] the abc module, collections module, and some pieces added to
typeobject.c centered on the __abstractmethods__ attribute.
[4] http://hg.python.org/cpython/file/fd6446a88fe3/Objects/typeobject.c#l2882
[5] http://hg.python.org/cpython/file/fd6446a88fe3/Lib/abc.py#l116
[6] I even made a "decorator" that I use for this (abstractattribute).
I can use that decorator on any object I like. The decorator
argument isn't expected to be only a function. However, it is
essentially a clone of abstractmethod. I also have an
AbstractAttribute class that I use instead of using abstract
properties, for attributes that I do not expect to implemented as
properties. I do like how functions provide a docstring that shows up
in help(), so abstract properties are nice in that way. However, you
can set the docstring on any object (almost) for the same effect (said
wishfully), which is what I do with my AbstractAttribute class.
[7] Of the four decorators in abc.py, only abstractmethod simply tags
the passed object with __isabstractmethod__. The other three (for
property, staticmethod, and classmethod) replace the passed object
with themselves. However, this has no bearing on the implementation
in subclasses.
[8] The abstracting decorators in abc.py are still fine. The are
great for saying what kind of object you expect in the implementation
of subclasses.
Is it bad to set __abstractmethod__ on non-functions in order to
trigger the ABC abstractness checks? If not, are __isabstractmethod__
on objects and __abstractmethods__ on classes misleading names?
I don't mean to imply that Python has it wrong. On the contrary, I
typically start by assuming that Python has it right and then try to
figure out what I am missing [1]. Without further ado, here's a
little more meat so that my questions have a better context.
Background
Last week I jumped into an tracker issue that has been developing for
some time [2]. The issue involves the interplay between abstract base
classes and descriptors (like property). Keep in mind that the issue
is only the catalyst for my question here, and my question is a
tangent. If possible I would rather any discussion here center on
that, and anything meaningful to the issue be put in the tracker.
Perhaps I did not think it through, but I made a point that the term
abstractmethod is potentially misleading. I'm hoping to get some
feedback on that from you. The reason I bring it up here is because
of the brief response I got in the ticket. I did not want to further
side-track the discussion going on there, so I'm bringing it here
instead.
In the issue Nick explained the following:
Regarding the naming, @abstractmethod and __isabstractmethod__ are
definitely about methods, or method-based descriptors (such as
property).
There *could* be a case to be made to allow for abstract data
attributes,
but really, using an abstract property should cover that use case well
enough.
Nick is a super smart guy, has probably 100x the experience I do, and,
from my experience with him, simply <gets it> a ton better than I do,
so I trust what he says. Also, he was involved with all this when
abstract base classes were added to the language [3]. From what Nick
said and from the name used, I'm guessing that the motivation was to
have the abstractness of ABCs focused explicitly on methods.
Motivation
The reason I brought it up in the tracker issue is that I've taken to
using __isabstractmethod__ on non-methods. The code in object_new
(typeobject.c) does not care about the types of a class's attributes
[4]. Neither does the code in ABCMeta (abc.py) [5]. All that matters
is that an attribute has __abstractmethod__ set to True to indicate
that the class is abstract. So with that in mind, I use it on
non-function attributes that I want to be abstract [6].
Nick pointed out that you can use an abstract property to accomplish
this same thing. That's what I had been doing for quite a while.
However, I was uncomfortable using an abstract property to describe
something that I did not expect to be implemented as a property. But
that may be just me.
Keep in mind that ABCMeta and typeobject.c do not do any "signature
checking" of objects, so the implementation of an abstract attribute
in a subclass can be any object. All that matters is that the
matching name be bound on the namespace of the subclass. Thus, an
abstract method does not have to be implemented with a method, nor an
abstract property with a property. However, the key point is that the
abstracting decorators provide an indicator of the expectation of the
implementation [7].
Questions
So this all comes back to those two questions. Before jumping into
tracker issue, I had not considered that my use of
__isabstractmethod__ may be contrary to the expected use. Is it? If
not, then perhaps __isabstractmethod__ (on objects) would be better
called __isabstractattribute__; and __abstractmethods__ (on classes)
would be better called __abstractattributes__ [8].
I'm certainly not saying that we should jump up and change them if
they are a potentially incomplete description of their usage, which
they may not be. The fact is, Python lets you do things that many
other languages would not allow, with the stipulation that "we're all
consenting adults"--so if you step outside the lines of expected use,
you're on your own (that's one awesome aspect of Python). Thus, even
if my use case is valid, it may be of such small impact relative to
abstract methods that it makes more sense to use "method" in the
names.
So again, do you think my use of __abstractmethod__ with non-functions
is appropriate? If so, is it a valid enough use case of abstractness
that using "method" in those two attribute names should change? My
gut says yes to the first and no to the second, but I want to hear
what you think!
-eric
p.s. Sorry for the length of this message. Not only do I get
long-winded every once in a while <wink> but I wanted to give this a
thorough treatment.
[1] One thing that I love about Python is that it is driven by really
smart yet practical people that want an easy-to-use, powerful
language. The design is not driven by pure language theory. But it's
not driven by "let's just get it done". It is a powerful mix.
Features are not added unless they satisfy a real problem that can't
be met using existing features without a Python-feature-threshold
amount of work. Usually the problem must be widely (enough) spread in
the Python community, as demonstrated by a large number of uses in the
wild or a widely used third-party package that fills the gap. Even
then, enough core developers have to agree that the problem is big
enough to warrant changing the language. Finally, someone has to do
the work. So the more I saw what went into Python, the less I
questioned it.
[2] http://bugs.python.org/issue11610#msg138189
[3] the abc module, collections module, and some pieces added to
typeobject.c centered on the __abstractmethods__ attribute.
[4] http://hg.python.org/cpython/file/fd6446a88fe3/Objects/typeobject.c#l2882
[5] http://hg.python.org/cpython/file/fd6446a88fe3/Lib/abc.py#l116
[6] I even made a "decorator" that I use for this (abstractattribute).
I can use that decorator on any object I like. The decorator
argument isn't expected to be only a function. However, it is
essentially a clone of abstractmethod. I also have an
AbstractAttribute class that I use instead of using abstract
properties, for attributes that I do not expect to implemented as
properties. I do like how functions provide a docstring that shows up
in help(), so abstract properties are nice in that way. However, you
can set the docstring on any object (almost) for the same effect (said
wishfully), which is what I do with my AbstractAttribute class.
[7] Of the four decorators in abc.py, only abstractmethod simply tags
the passed object with __isabstractmethod__. The other three (for
property, staticmethod, and classmethod) replace the passed object
with themselves. However, this has no bearing on the implementation
in subclasses.
[8] The abstracting decorators in abc.py are still fine. The are
great for saying what kind of object you expect in the implementation
of subclasses.