How about "pure virtual methods"?

[Doug Holton]

well, since I'm not in the ego-stroking business, what if I promise never to
reply to posts by you, robert, and alex?

That's not fair to the rest of us though

 

[Simon Wittber]

I do understand the feeling though; Fredrik Lundh jumped at me only a

few days ago when I said that I personally found list comprehensions
more readable than map. I certainly don't mind being advised or
corrected if I've written incorrect or even just suboptimal code, but
attacking a person's mental processes seems kind of needlessly
aggressive to me.

We've just had a Python 2.4 release, and poor Mr Lundh has likely had
hundreds of user's clamouring for a binary, 2.4 compatible PIL
release.

That's enough to make any programmer grumpy.

Sw.

 

[Fredrik Lundh]

We've just had a Python 2.4 release, and poor Mr Lundh has likely had
hundreds of user's clamouring for a binary, 2.4 compatible PIL
release.

many hundreds, indeed. but that's not a problem at all; I have a webserver
that helps me with things like that.

</F>

 

[Doug Holton]

I don't really have a good solution; despite the unnecessarily vicious
comments, I don't feel like I can set my newsreader to ignore messages
from, for example, Fredrik, because his answers, when not attacks, are
often very insightful. If you find a good solution to this problem,
please let me know.

Thankyou for bringing it up. It looks like it may have a positive effect.

 

[Steven Bethard]

That's not fair to the rest of us though

That's not even fair to the non-rest of us. =) As I noted, "his answers
.... are often very insightful" -- it would be a pity to lose them.

Steve

 

[Doug Holton]

That's not even fair to the non-rest of us. =) As I noted, "his answers
... are often very insightful" -- it would be a pity to lose them.

He was only acknowledging the problem to those 3 people who complained
about it. I was making the point that others do not like being trolled
either.

 

[Steven Bethard]

He was only acknowledging the problem to those 3 people who complained
about it. I was making the point that others do not like being trolled
either.

Ahh, gotcha. Read your comment with the right intonation this time. =)

Steve

 

[Carl Banks]

(I think you could create some kind of drinking game based on the number of
non-quoted lines between the "From"-line and the first line containing the word
"boo" in Doug's posts on comp.lang.python. A little like the game based on the
number of minutes between someone mentioning ElementTree or Anobind or
some other Python/XML toolkit in a blog post, and David Mertz adding a
comment about how his own tool can do the same thing faster and

better...)

Or until Peter Hansen mentions unit testing, or Fernando Perez mentions
IPython, or Michele Simionato (sp?) proposes a metaclass solution to a
problem, or Paul Rubin decries any aspect of Python that differs from
Lisp,
or Aahz asserts that the GIL isn't a significant problem in practice,
or....

all-in-fun-of-course-ly yr's,

 

[Doug Holton]

I think there are some solutions like PyProtocols, see section 2.2 on
this page: http://www.python.org/cgi-bin/moinmoin/MetaClasses

Let me rephrase the rest of my answer.

This is something we could use in Python. You should add a feature
request to have this in Python 3.0, a.k.a. Python 3000:
http://www.python.org/cgi-bin/moinmoin/Python3.0

You might have simple code such as:

inteface ICallable:
def Call(args)

However, Python 3.0 is likely years away. If you want to know how to
run code like this and have real interfaces, abstract classes and
virtual methods today, consult Fredrik Lundh.

 

[Doug Holton]

Thankyou for bringing it up. It looks like it may have a positive effect.

Unfortunately, I may have jumped the gun on that one. He does not even
acknowledge his behavior outside of the three instances he referred to.

 

[Aahz]

Unfortunately, I may have jumped the gun on that one. He does not even
acknowledge his behavior outside of the three instances he referred to.

From my POV, your use of a munged e-mail address makes you far more
antisocial than Fredrik.

 

[Noam Raphael]

Thank you all, especially Alex for your enlightening discussion, and
Scott for your implementation. I'm sorry that I can't be involved in a
daily manner - but I did read all of the posts in this thread. They
helped me understand the situation better, and convinced me that indeed
this feature is needed. Let's see if I can convince you too.

First, the actual situation in which I stood, which made me think, "I
would like to declare a method as not implemented, so that subclasses
would have to implement it."

I wrote a system in which objects had to interact between themselves. In
my design, all those objects had to implement a few methods for the
interaction to work. So I wrote a base class for all those objects, with
a few methods which the subclasses had to implement. I think it's good,
for *me*, to have an explicit list of what should be implemented, so
that when (in a function) I expect to get an object of this kind I know
what I may and may not do with it.

Then, I wrote the classes themselves. And I wrote unit tests for them.
(Ok, I lie. I didn't. But I should have!) Afterwards, I decided that I
needed all my objects of that kind to supply another method. So I added
another "raise NotImplementedError" method to the base class. But what
about the unit tests? They would have still reported a success - where
of course they shouldn't have; my classes, in this stage, didn't do what
they were expected to do. This problem might arise even when not
changing the interface at all - it's quite easy to write a class which,
by mistake, doesn't implement all the interface. Its successful unit
tests may check every single line of code of that class, but a complete
method was simply forgotten, and you wouldn't notice it until you try
the class in the larger framework (and, as I understand, the point of
unit testing is to test the class on its own, before integrating it).

Ok. This was the practical reason why this is needed. Please note that I
didn't use "isinstance" even once - all my functions used the
*interface* of the objects they got. I needed the extra checking for
myself - if someone wanted to implement a class that wouldn't inherit
from my base class, but would nevertheless implement the required
interface, he was free to do it, and it would have worked fine with the
framework I wrote.

Now for the "theoretical" reason why this is needed. My reasoning is
based on the existence of "isinstance" in Python. Well, what is the
purpose of isinstance? I claim that it doesn't test if an object *is* of
a given type. If that would have been its purpose, it would have checked
whether type(obj) == something. Rather, it checks whether an object is a
subclass of a given type. Why should we want such a function? A subclass
may do a completely different thing from what the original class did!
The answer is that a subclass is guaranteed to have the same *interface*
as the base class. And that's what matters.

So I conclude that a subclass, in Python, must implement the interface
of its parent class. Usually, this is obvious - there's no way for a
subclass not to implement the interface of its parent class, simply
because it can only override methods, but can't remove methods. But what
shall we do if the some methods in the base class consist *only* of an
interface? Can we implement only a part of the interface, and claim that
instances of that class are instances of the original class, in the
"isinstance" fashion? My answer is no. The whole point of "isinstance"
is to check whether an instance implements an interface. If it doesn't -
what is the meaning of the True that isinstance returns? So we should
simply not allow instances of such classes.

You might say that abstract classes at the base of the hierarchy are
"not Pythonic". But they are in Python already - the class basestring is
exactly that. It is an uninstantiable class, which is there only so that
you would be able to do isinstance(x, basestring). Classes with
"notimplemented" methods would behave in exactly the same way - you
wouldn't be able to instantiate them, just to subclass them (and to
check, using isinstance, whether they implement the required protocol,
which I agree that wouldn't be Pythonic, probably).

Ok. This is why I think this feature fits Python like a glove to a hand.
Please post your comments on this! I apologize now - I may not be able
to reply in the next few days. But I will read them at the end, and I
will try to answer.

Have a good day,
Noam

 

[Noam Raphael]

Even if you can do it, how would you then implement a class hierarchy
where the ultimate base class had virtual methods, and you wanted to
derive from that class another class, to be used as a base class for
usable classes, which implemented only a subset of the virtual methods,
leaving the others to be implemented by the ultimate subclasses?

What I suggest is that only *instantiation* would be forbidden. You are
free to make a subclass which defines only some of the abstract methods,
and to subclass the subclass and define the rest. You would only be able
to make instances of the subclass of the subclass, but that's ok.

See Scott's implementation - the test at the end does exactly this.

I hope this helped,
Noam

 

[Noam Raphael]

My long post gives all the philosophy, but I'll give here the short answers.

+0

Python doesn't use classes for typing. As Alex Martelli puts it,
Python uses protocols. So the client expecting a concrete subclass of
your abstract class may get an instantiation of a class that doesn't
inherit from the abstract class at all.

That's right - this mechanism is useful mostly for he who implements
that class, to make sure that he implemented all that is needed to be
assigned the title "a subclass of that class".

Or maybe the subclass is only going to use a subset of the features of
the abstract class, and the author knows that sum deferred methods
won't be invoked. The correct behavior in this case would be to allow
the subclass to be instantiated, and then get a runtime error if one
of the features the author thought he could skip was actually called.

I disagree - my reasoning is that a subclass must implement the complete
interface of its base class (see my long post). The author may implement
a class which defines only a part of the interface, and give it to the
function, and it may work and be great. But it must not be called "an
instance of the abstract class".

Finally, in a sufficiently complex class hierarchy, this still leaves
you wondering through the hierarchy trying to find the appropriate
parent class that tagged this method as unimplemented, and then
figuring out which class should have implemented it - as possibly a
parent of the class whose instantiation failed is the subclass that
should have made this method concrete.

You are right - but I needed this for a class hierarchy of only two
levels (the base abstract class and the concrete subclasses), so there
were not many classes to blame for a missing method.

<mike

I hope this seems reasonable,
Noam

 

[Jeff Shannon]

But what about the unit tests? They would have still reported a
success - where of course they shouldn't have; my classes, in this
stage, didn't do what they were expected to do. This problem might
arise even when not changing the interface at all - it's quite easy to
write a class which, by mistake, doesn't implement all the interface.
Its successful unit tests may check every single line of code of that
class, but a complete method was simply forgotten, and you wouldn't
notice it until you try the class in the larger framework (and, as I
understand, the point of unit testing is to test the class on its own,
before integrating it).

Except that unit tests should be written to the *specification*, not the
implementation. In other words, forgetting a complete method would
require that you forget to write the method, *and* that you failed to
translate the specification into unit tests *for that same method*.

In the context of changing an existing interface, a unit-testing
scenario would mean that, instead of installing a "pure virtual" method
on a base class, you'd change the unit-tests to follow the new
specification, and then write code that would pass the unit tests. If
you are subclassing from a common base, then you'd only need to change
the unit test for that common base class (presuming that all derived
classes would run those unit tests as well).

Jeff Shannon
Technician/Programmer
Credit International

 

[Noam Raphael]

Except that unit tests should be written to the *specification*, not the
implementation. In other words, forgetting a complete method would
require that you forget to write the method, *and* that you failed to
translate the specification into unit tests *for that same method*.

You are absolutely right - but when you are not that tidy, and don't
have a written specification apart from the code, it would be natural to
go over each method in the class definition, and write a test to check
if it does what it should. I'm not saying that it's the ideal way, but
it is not that bad, usually.

In the context of changing an existing interface, a unit-testing
scenario would mean that, instead of installing a "pure virtual" method
on a base class, you'd change the unit-tests to follow the new
specification, and then write code that would pass the unit tests. If
you are subclassing from a common base, then you'd only need to change
the unit test for that common base class (presuming that all derived
classes would run those unit tests as well).

The problem is that I couldn't write a general unit test, since the base
class wasn't instantiable - there wasn't even any point in making it
instantiable, since every subclass was constructed with different
argument definition. They were just required to provide some methods
(check whether they contain an object, for example) - I don't know how
to write a unit test for such a base class, or what does it mean. (Well,
it may mean: check whether all the required methods exist, but come on -
that's exactly the idea of my suggestion. There's no point in having to
write the list of required methods another time).

Jeff Shannon
Technician/Programmer
Credit International

Thanks for your comment. You're welcomed to reply if you don't agree.
Noam

 

[Noam Raphael]

class Abstract(object):
'''A class to stick anywhere in an inheritance chain'''
__metaclass__ = MustImplement


def notimplemented(method):
'''A decorator for those who prefer the parameters declared.'''
return NotImplemented

I just wanted to say that I thought of notimplemented as a class, that
would save a reference to the functions it got in the constructor. In
that way pydoc and his friends would be able to find the arguments the
method was expected to get, and its documentation string.

But it's a great implementation.

Noam

Oh, and another thing - maybe "abstract" is a better name than
"notimplemented"? notimplemented might suggest a method which doesn't
have to be implemented - and raises NotImplementedError when it is
called. What do you think?

 

[Mike Meyer]

The answer is that a subclass is guaranteed to have the same
*interface* as the base class. And that's what matters.

This is false. For instance:

class A(object):
def method(self, a):
print a

class B(A):
def method(self, a, b):
print a, b

B implements a different interface than A. Statically typed OO
languages either use multi-methods or disallow changing the signature
of an overridden method.

A tool to detect such cases would probably be almost as useful as the
tool you've proposed.

<mike

 

[Fredrik Lundh]

Oh, and another thing - maybe "abstract" is a better name than "notimplemented"? notimplemented
might suggest a method which doesn't have to be implemented - and raises NotImplementedError when
it is called. What do you think?

what's the difference? no, really?

</F>

 

[Jeff Shannon]

Noam Raphael wrote:




what's the difference? no, really?

The difference would be that a derived class *must* override an
"abstract" method, but has the choice of leaving a "notimplemented"
method unimplemented -- one is obligate, the other optional.

Personally, all of this seems like more bother than it's worth to me.
I'm perfectly willing to "struggle" along without abstract base classes
enforcing an interface, and merely make do with unenforced, informal
protocols... but then, I'm hardly an expert in such matters. (Or any
other sort of matters, either.) ::shrug::

Jeff Shannon
Technician/Programmer
Credit International