How about "pure virtual methods"?

Discussion in 'Python' started by Noam Raphael, Dec 18, 2004.

  1. Noam Raphael

    Noam Raphael Guest

    Hello,

    I thought about a new Python feature. Please tell me what you think
    about it.

    Say you want to write a base class with some unimplemented methods, that
    subclasses must implement (or maybe even just declare an interface, with
    no methods implemented). Right now, you don't really have a way to do
    it. You can leave the methods with a "pass", or raise a
    NotImplementedError, but even in the best solution that I know of,
    there's now way to check if a subclass has implemented all the required
    methods without running it and testing if it works. Another problem with
    the existing solutions is that raising NotImplementedError usually means
    "This method might be implemented some time", and not "you must
    implement this method when you subclass me".

    What I suggest is a new class, called notimplemented (you may suggest a
    better name). It would get a function in its constructor, and would just
    save a reference to it. The trick is that when a new type (a subclass of
    the default type object) is created, It will go over all its members and
    check to see if any of them is a notimplemented instance. If that is the
    case, it would not allow an instantiation of itself.

    What I want is that if I have this module:

    ======================

    class BaseClass(object):
    def __init__(self):
    ...

    @notimplemented
    def save_data(self, filename):
    """This method should save the internal state of the class to
    a file named filename.
    """
    pass

    class RealClass(BaseClass):
    def save_data(self, filename):
    open(filename).write(self.data)

    ======================

    then if I try to instantiate BaseClass I would get an exception, but
    instantiating RealClass will be ok.


    Well, what do you say?

    Noam Raphael
     
    Noam Raphael, Dec 18, 2004
    #1
    1. Advertising

  2. Noam Raphael wrote:

    > Well, what do you say?


    Raising NotImplementedError in the methods that need to be overridden is
    much more customary, more straightforward, and already works fine.

    --
    Erik Max Francis && && http://www.alcyone.com/max/
    San Jose, CA, USA && 37 20 N 121 53 W && AIM erikmaxfrancis
    We learn from history that we do not learn from history.
    -- Georg Friedrich Wilhelm Hegel
     
    Erik Max Francis, Dec 19, 2004
    #2
    1. Advertising

  3. Noam Raphael <> wrote:

    > What I want is that if I have this module:
    >
    > ======================
    >
    > class BaseClass(object):
    > def __init__(self):
    > ...
    >
    > @notimplemented
    > def save_data(self, filename):
    > """This method should save the internal state of the class to
    > a file named filename.
    > """
    > pass
    >
    > class RealClass(BaseClass):
    > def save_data(self, filename):
    > open(filename).write(self.data)
    >
    > ======================
    >
    > then if I try to instantiate BaseClass I would get an exception, but
    > instantiating RealClass will be ok.
    >
    >
    > Well, what do you say?


    It's already easy to do as long as you don't insist that BaseClass must
    subclass object (or allow a __metaclass__ assignment there). Basically,
    what you describe is the job of a custom metaclass, because that's where
    the __call__ method on classes is defined (classes being instances of
    the metaclass)... much cleaner than tampering with the __new__ or
    __init__ at class level.

    At sketch level, you'd have, say:

    def notimplemented(f):
    # save some data about f somewhere if you wish, then, e.g.:
    return notimplemented

    import inspect

    class Meta_for_NR(type):
    def __init__(cls, cn, cb, cd):
    super(Meta_for_NR, cls).__init__(cn, cb, cd)
    abstract_methods = []
    for n, v in inspect.getmembers(cls, inspect.ismethod):
    if v is notimplemented: abstract_methods.append(n)
    cls._abstract_methods = abstract_methods
    def __call__(cls, *a, **k):
    if cls._abstract_methods:
    raise TypeError, ("Cannot instantiate abstract class %s."
    " Abstract methods: %s." % (cls.__name__,
    ', '.join(cls._abstract_methods)))
    return super(Meta_for_NR, cls).__call__(*a, **k)

    class Base_for_NR: __metaclass__ = Meta_for_NR


    Inheriting from Base_for_NR instead of inheriting from object should
    give you just the behavior you desire (net of the fact that this code of
    mine is untested, just a sketch -- but the key ideas shd be ok).


    If you can implement this and get it into wide usage, it may well be
    adopted in the standard library. Unless there is proof by experience
    that a lot of people like this approach better than the de facto
    standard, such adoption is very unlikely. But the great thing is that
    it's EASY to implement your ideas and try to garner a following for
    them, so you stand a chance to get them into the standard library if you
    can prove they really work well and are well-liked!


    Alex
     
    Alex Martelli, Dec 19, 2004
    #3
  4. Noam Raphael

    John Machin Guest

    Alex Martelli wrote:

    > class Meta_for_NR(type):
    > def __init__(cls, cn, cb, cd):
    > super(Meta_for_NR, cls).__init__(cn, cb, cd)
    > abstract_methods = []
    > for n, v in inspect.getmembers(cls, inspect.ismethod):
    > if v is notimplemented: abstract_methods.append(n)

    etc etc etc

    I'm missing the point of the question and Alex's answer. Let's step
    through this:

    1. Programmer A designs a base class some (or all) of whose methods are
    abstract i.e. not implemented in a do-something-useful fashion; they
    may be only stubs which document their arguments.

    2. Programmer B designs a sub-class, which *should* implement methods
    which override those of the base class.

    3. Programmer C writes an application which *should* create instances
    of the sub-class.

    4. Things that can go wrong are (a) C instantiates the base class (b) B
    has not provided all the methods (c) B has provided methods but they
    don't work properly (d) C's app doesn't invoke B's methods in the
    approved manner or sequence or with meaningful arguments or ...

    5. The standard approach in Python is to have the virtual methods raise
    an exception if invoked. This of course doesn't help with problems c
    and d. In an attempt to root out instances of problems c and d,
    programmer C will unit-test the @#$% out of the app. This will quickly
    discover any instances of problems a and b. One would hope that as well
    B has unit-tested the @#$% out of the sub-class.

    6. Alex's magnum opus appears to operate on problem a, and maybe on b
    (it's over my head). It involves a fair chunk of mucking about -- for
    what? Early warning, a few microseconds ahead of the invocation of a
    method which will cause the stub in the base class to raise an
    exception?
     
    John Machin, Dec 19, 2004
    #4
  5. Noam Raphael

    Nick Coghlan Guest

    John Machin wrote:
    > 6. Alex's magnum opus appears to operate on problem a, and maybe on b
    > (it's over my head). It involves a fair chunk of mucking about -- for
    > what? Early warning, a few microseconds ahead of the invocation of a
    > method which will cause the stub in the base class to raise an
    > exception?


    Alex eats metaclasses for breakfast though, so while it may look like a magnum
    opus to mere mortals, it is in reality nothing more than an idle twiddle :)

    Anyway, I see two major problems with the OP's idea:

    1. It encourages laziness when subclassing
    Subclassing necessarily involves intimate coupling of code. If you don't
    understand the base class well enough to know which methods you are required to
    implement, and haven't even included those methods in your unit tests. . . then
    I'm willing to bet those missing methods are the least of your worries when it
    comes to correct operation of your class.

    2. It gives up some of the dynamism of Python
    The OP's idea denies the inheritor the ability to partially implement the
    defined interface. Sometimes, particularly for objects used for a very specific
    purpose, it makes sense to only implement the portion of an interface that you
    want to use. The standard method makes it easy to flag a portion of the
    interface as off limits - just leave it raising a Not Implemented exception.
    With the OP's idea in place, the inheritor is forced to implement the entirety
    of the interface, even if they only need a fraction of it.

    Cheers,
    Nick.

    --
    Nick Coghlan | | Brisbane, Australia
    ---------------------------------------------------------------
    http://boredomandlaziness.skystorm.net
     
    Nick Coghlan, Dec 19, 2004
    #5
  6. John Machin <> wrote:
    ...
    > 6. Alex's magnum opus appears to operate on problem a, and maybe on b
    > (it's over my head). It involves a fair chunk of mucking about -- for


    A dozen lines of code to write *ONCE* and put in your utilities module
    is a "magnum opus" and "a fair chunk of mucking about"?! Oh my -- I
    wonder what would count for you as parvus and incredibly tiny -- SIX
    lines? THREE? ONE AND A HALF, for Pete's sake?!?!?!

    If your problem with my code is simply that it uses parts of Python that
    you're not familiar about, then I count this as a positive aspect of my
    code, not a negative one: by providing motivation to study metaclasses
    and inspect (by givin some small usefulness in very small space) it may
    help you indirectly far more than directly.

    > what? Early warning, a few microseconds ahead of the invocation of a
    > method which will cause the stub in the base class to raise an
    > exception?


    Exactly. Microseconds don't count, but stack levels do -- getting the
    traceback end as close as possible to the real CAUSE of the problem,
    rather than a few levels further down where the SYMPTOM appears, can
    easily shave whole minutes off debugging time. Since the whole of what
    you call "magnum opus" took about 4 minutes to write, if it acts this
    way twice I'm gaining.

    Indeed, this issue is important enough that it might be worth one's
    while to ensure the exception comes at class definition rather than
    instantiation time -- e.g. by saying that a class must explicitly expose
    (not just inherit) a class attribute '__abstract__' that's True, or else
    having abstract methods still present is an error. This obviously takes
    a trivial mod to the (ha!) "magnum" opus, and makes errors even easier
    to diagnose. Still, it wasn't among the OP's desiderata, so I didn't
    change the semantics in this way.


    Note that either or both changes work particularly well wrt UNIT tests,
    tests of some component which sees the abstract base class as 'external'
    and thus not subject to its unit tests. Say that 'external' component X
    exposes ABC, the abstract base class, and many functions f, g, h, ...,
    which take an ABC instance and do things with it. My own component Y
    subclasses ABC with WCC, the wannabe concrete class, and passes an
    instance of WCC to *SOME* of X.f, X.g, X.h -- SOME, not necessarily ALL,
    because UNIT tests don't have to fully probe interfaces BETWEEN
    components!!! (that's why they're UNIT tests, not INTEGRATION ones;-).

    With Python's standard approach, no guarantee that WCC implements all
    the methods it should. Maybe it did implement all methods needed at the
    time it was written, then X release 1.1 added a couple abstract methods
    to ABC and a couple new functions j and k using them, and Y knows
    nothing about this upgrade. Y's unit tests are still gonna pass. And
    somebody ELSE who's instantiating a Y.WCC and passing the instance to
    X.k is gonna feel the pain -- not in their unit tests, mind you, this
    has VERY MUCH become a problem of inter-component orchestration.

    With the cost of 12 lines, ONCE, in a utility module, you ensure halfway
    decent unit tests WILL expose all 'failed to implement needed methods'
    errors, even as X's specs keep evolving. If you make any substantial
    use of abstract classes and methods, the tradeoff is VERY favourable.


    So, do _I_ use this? Nah: I think abstract classes with
    explicitly-abstract "somebody's gotta implement this!" methods are
    rarely a good idea and tend to lead to fat base classes and overuse of
    inheritance. But if one disagrees, then I think that Python's suppport
    for them with NotImplementedError is suboptimal, and some variation and
    evolution along the lines of the tiny sketch I gave would be better.


    Alex
     
    Alex Martelli, Dec 19, 2004
    #6
  7. Alex Martelli:

    >> what? Early warning, a few microseconds ahead of the invocation of a
    >> method which will cause the stub in the base class to raise an
    >> exception?

    >
    > Exactly. Microseconds don't count, but stack levels do -- getting the
    > traceback end as close as possible to the real CAUSE of the problem,
    > rather than a few levels further down where the SYMPTOM appears, can
    > easily shave whole minutes off debugging time.


    minutes?

    Traceback (most recent call last):
    File "mymodule.py", line 9, in somefunction
    someobj.bar()
    ... zero or more lines ...
    File "somelibrary.py", line 3, in bar
    raise NotImplementedError("must implement abstract method bar")
    NotImplementedError: must implement abstract method bar

    you have to look at the last line of a traceback to find the error message, and
    the offending module/method is two lines above that.

    > So, do _I_ use this? Nah: I think abstract classes with
    > explicitly-abstract "somebody's gotta implement this!" methods are
    > rarely a good idea


    yet you claim to be an expert on issues related to debugging NotImplemented-
    Error exceptions?

    </F>
     
    Fredrik Lundh, Dec 19, 2004
    #7
  8. Fredrik Lundh <> wrote:

    > Alex Martelli:
    >
    > >> what? Early warning, a few microseconds ahead of the invocation of a
    > >> method which will cause the stub in the base class to raise an
    > >> exception?

    > >
    > > Exactly. Microseconds don't count, but stack levels do -- getting the
    > > traceback end as close as possible to the real CAUSE of the problem,
    > > rather than a few levels further down where the SYMPTOM appears, can
    > > easily shave whole minutes off debugging time.

    >
    > minutes?
    >
    > Traceback (most recent call last):
    > File "mymodule.py", line 9, in somefunction
    > someobj.bar()
    > ... zero or more lines ...
    > File "somelibrary.py", line 3, in bar
    > raise NotImplementedError("must implement abstract method bar")
    > NotImplementedError: must implement abstract method bar
    >
    > you have to look at the last line of a traceback to find the error
    > message, and the offending module/method is two lines above that.


    The offending module is the one which instantiates a class it shouldn't
    instantiate -- a class which isn't made concrete by overriding all
    methods which need to be. What is that module? What is that class?
    The error is generally not the calling of 'bar' on someobj: someobj
    arrives as a argument to somefunction from somewhere -- the caller of
    somefunction is quite often innocent, in turn -- who BUILT someobj, by
    instantiating which class? I think `minutes' is a very fair (not at all
    pessimistic) assessment of the time it will generally take to find out.

    With an exception at the time of instantiation, you shave those minutes
    off. As I already explained: if it was acceptable to raise at the time
    the class statement is executed, which doesn't make the class fully
    concrete yet fails to explicitly assert __abstract__=True in classbody,
    the distance between where the error is in the code, and where it's
    diagnosed, could be further reduced in many cases; sometimes the error
    is to mistakenly instantiate a class which is not yet meant to be fully
    concrete, but equally well it could be to fail to override all needed
    methods in a class which IS meant to be fully concrete. However, this
    further refinement turns out to be generally of lesser impact wrt the
    primary one: as long as the error msg at instantiation time fully
    identifies what class is involved and why (because of what still
    abstract methods) it's an error to instantiate it, then the productivity
    plus of getting the error at class-statement time vs instantiation time
    is generally no big deal -- the big deal is not having to wait until,
    somewhere, somehow, a still-abstract method finally gets _called_, and
    then having to trace back to how and where that class was instantiated.


    > > So, do _I_ use this? Nah: I think abstract classes with
    > > explicitly-abstract "somebody's gotta implement this!" methods are
    > > rarely a good idea

    >
    > yet you claim to be an expert on issues related to debugging NotImplemented-
    > Error exceptions?


    Yes, the body of experience which essentially convinced me to avoid the
    above idiom (because, as I say in the snippet I quote here, it's rarely
    a good idea -- you snipped my summary of reasons why) essentially
    overlaps with the body of experience which suggests to me that IF
    somebody's keen to use the idiom widely anyway, they're going to be far
    better off with the errors it's meant to catch causing errors to be
    raised ASAP, as the custom metaclass so easily allows, rather than as
    late as possible, as using 'raise NotImplementedError' leads to.

    [[ Protocols (also unfortunately known as interfaces) and adaptation
    work much better, essentially by not depending on inheritance. ]]

    I really don't understand what's so difficult here, for you, as to make
    it apparently so very hard for you to follow. The idiom (meant to catch
    some wrong uses of inheritance) of 'abstract classes with certain
    methods that need to get overridden' isn't all that helpful in Python
    when all that happens is a NotImplementedError exception. If the base
    class had simply omitted defining the method, as used to be the idiom,
    you'd generally get an AttributeError instead -- slightly more generic
    but happening basically at the same spot in the code, time difference in
    finding the errors usually a few seconds. With a custom metaclass (and
    optionally a decorator, though there are other ways, which we can
    discuss in detail if you think such distinctions are oh so crucial) you
    can easily make the idiom more helpful for the purpose of diagnosing
    errors in inheritance &c -- i.e., the idiom's sole purpose.

    Nevertheless, even when one has made the idiom as good as it can be, I
    think it's still rarely a good idea. Protocols and adaptation take a
    fresh approach, not as heavily influenced by C++ and Java limitations as
    the idiom we're discussing here, and it's my contention that said
    approach does a better job. One can perfectly well still use
    inheritance for what inheritance does best (easy, simple, fast sharing
    of some implementation) without depending on it for those protocol
    compliance issues which were never its forte.


    Alex
     
    Alex Martelli, Dec 19, 2004
    #8
  9. Alex Martelli wrote:

    >> Traceback (most recent call last):
    >> File "mymodule.py", line 9, in somefunction
    >> someobj.bar()
    >> ... zero or more lines ...
    >> File "somelibrary.py", line 3, in bar
    >> raise NotImplementedError("must implement abstract method bar")
    >> NotImplementedError: must implement abstract method bar
    >>
    >> you have to look at the last line of a traceback to find the error
    >> message, and the offending module/method is two lines above that.

    >
    > The offending module is the one which instantiates a class it shouldn't
    > instantiate -- a class which isn't made concrete by overriding all
    > methods which need to be. What is that module?


    in real life, it's mymodule.py or mylib.py, in code that you wrote very recently.

    > The error is generally not the calling of 'bar' on someobj: someobj
    > arrives as a argument to somefunction from somewhere -- the caller of
    > somefunction is quite often innocent, in turn -- who BUILT someobj, by
    > instantiating which class? I think `minutes' is a very fair (not at all
    > pessimistic) assessment of the time it will generally take to find out.


    you can blather for years if you want to, but your arguments are just another variant
    of the "you cannot use whitespace for indentation" stuff we've all seen from time to
    time; it's trivial to come up with a large number of theoretical problems with any given
    approach, but in real life, for real-life programmers, using real-life libraries designed
    by real-life developers, it's simply not the problem you want it to be.

    but not that I'm going to convince you, now that you've spent 15,000 characters on
    this topic...

    </F>
     
    Fredrik Lundh, Dec 19, 2004
    #9
  10. Noam Raphael

    Steve Holden Guest

    Noam Raphael wrote:
    > Hello,
    >
    > I thought about a new Python feature. Please tell me what you think
    > about it.
    >
    > Say you want to write a base class with some unimplemented methods, that
    > subclasses must implement (or maybe even just declare an interface, with
    > no methods implemented). Right now, you don't really have a way to do
    > it. You can leave the methods with a "pass", or raise a
    > NotImplementedError, but even in the best solution that I know of,
    > there's now way to check if a subclass has implemented all the required
    > methods without running it and testing if it works. Another problem with
    > the existing solutions is that raising NotImplementedError usually means
    > "This method might be implemented some time", and not "you must
    > implement this method when you subclass me".
    >
    > What I suggest is a new class, called notimplemented (you may suggest a
    > better name). It would get a function in its constructor, and would just
    > save a reference to it. The trick is that when a new type (a subclass of
    > the default type object) is created, It will go over all its members and
    > check to see if any of them is a notimplemented instance. If that is the
    > case, it would not allow an instantiation of itself.
    >
    > What I want is that if I have this module:
    >
    > ======================
    >
    > class BaseClass(object):
    > def __init__(self):
    > ...
    >
    > @notimplemented
    > def save_data(self, filename):
    > """This method should save the internal state of the class to
    > a file named filename.
    > """
    > pass
    >
    > class RealClass(BaseClass):
    > def save_data(self, filename):
    > open(filename).write(self.data)
    >
    > ======================
    >
    > then if I try to instantiate BaseClass I would get an exception, but
    > instantiating RealClass will be ok.
    >
    >
    > Well, what do you say?
    >
    > Noam Raphael


    OK, you've had many replies telling you why your suggestion isn't
    particularly worthwhile, and a few telling you how you can already do
    this in present-day Python.

    Even if you can do it, how would you then implement a class hierarchy
    where the ultimate base class had virtual methods, and you wanted to
    derive from that class another class, to be used as a base class for
    usable classes, which implemented only a subset of the virtual methods,
    leaving the others to be implemented by the ultimate subclasses?

    This use case makes your suggestion look impractical, methinks, since it
    requires that the programmer do what you would apparently like to
    specifically forbid.

    regards
    Steve
    --
    Steve Holden http://www.holdenweb.com/
    Python Web Programming http://pydish.holdenweb.com/
    Holden Web LLC +1 703 861 4237 +1 800 494 3119
     
    Steve Holden, Dec 19, 2004
    #10
  11. Noam Raphael

    Mike Meyer Guest

    Noam Raphael <> writes:

    > What I suggest is a new class, called notimplemented (you may suggest
    > a better name). It would get a function in its constructor, and would
    > just save a reference to it. The trick is that when a new type (a
    > subclass of the default type object) is created, It will go over all
    > its members and check to see if any of them is a notimplemented
    > instance. If that is the case, it would not allow an instantiation of
    > itself.


    +0

    Python doesn't use classes for typing. As Alex Martelli puts it,
    Python uses protocols. So the client expecting a concrete subclass of
    your abstract class may get an instantiation of a class that doesn't
    inherit from the abstract class at all.

    Or maybe the subclass is only going to use a subset of the features of
    the abstract class, and the author knows that sum deferred methods
    won't be invoked. The correct behavior in this case would be to allow
    the subclass to be instantiated, and then get a runtime error if one
    of the features the author thought he could skip was actually called.

    Finally, in a sufficiently complex class hierarchy, this still leaves
    you wondering through the hierarchy trying to find the appropriate
    parent class that tagged this method as unimplemented, and then
    figuring out which class should have implemented it - as possibly a
    parent of the class whose instantiation failed is the subclass that
    should have made this method concrete.

    <mike
    --
    Mike Meyer <> http://www.mired.org/home/mwm/
    Independent WWW/Perforce/FreeBSD/Unix consultant, email for more information.
     
    Mike Meyer, Dec 19, 2004
    #11
  12. Noam Raphael

    Doug Holton Guest


    > Noam Raphael wrote:
    > even in the best solution that I know of,
    >> there's now way to check if a subclass has implemented all the
    >> required methods without running it and testing if it works.


    I think there are some solutions like PyProtocols, see section 2.2 on
    this page: http://www.python.org/cgi-bin/moinmoin/MetaClasses

    But what you are wanting is used more in statically typed languages.
    If you want interfaces, abstract classes and virtual methods in a
    python-like language see boo: http://boo.codehaus.org/
    It will catch errors like this at compile-time.
     
    Doug Holton, Dec 19, 2004
    #12
  13. .... From: Doug ...

    (I think you could create some kind of drinking game based on the number of
    non-quoted lines between the "From"-line and the first line containing the word
    "boo" in Doug's posts on comp.lang.python. A little like the game based on the
    number of minutes between someone mentioning ElementTree or Anobind or
    some other Python/XML toolkit in a blog post, and David Mertz adding a
    comment about how his own tool can do the same thing faster and better...)

    .... boo ...

    (</F>)
     
    Fredrik Lundh, Dec 19, 2004
    #13
  14. Noam Raphael

    John Machin Guest

    Alex Martelli wrote:
    > John Machin <> wrote:
    > ...
    > > 6. Alex's magnum opus appears to operate on problem a, and maybe on

    b
    > > (it's over my head). It involves a fair chunk of mucking about --

    for
    >
    > A dozen lines of code to write *ONCE* and put in your utilities

    module
    > is a "magnum opus" and "a fair chunk of mucking about"?! Oh my -- I
    > wonder what would count for you as parvus and incredibly tiny -- SIX
    > lines? THREE? ONE AND A HALF, for Pete's sake?!?!?!


    "Magnum opus" was exaggerated politeness.

    >
    > If your problem with my code is simply that it uses parts of Python

    that
    > you're not familiar about, then I count this as a positive aspect of

    my
    > code, not a negative one: by providing motivation to study

    metaclasses
    > and inspect (by givin some small usefulness in very small space) it

    may
    > help you indirectly far more than directly.


    Thank you for the motivation. I now understand what it is attempting to
    do. Unfortunately it doesn't succeed. Instead of:

    if v is notimplemented: abstract_methods.append(n)

    you need:
    if v.im_func is notimplemented: abstract_methods.append(n)
     
    John Machin, Dec 19, 2004
    #14
  15. Fredrik Lundh <> wrote:
    ...
    > Alex Martelli wrote:
    >
    > >> Traceback (most recent call last):
    > >> File "mymodule.py", line 9, in somefunction
    > >> someobj.bar()
    > >> ... zero or more lines ...
    > >> File "somelibrary.py", line 3, in bar
    > >> raise NotImplementedError("must implement abstract method bar")
    > >> NotImplementedError: must implement abstract method bar
    > >>
    > >> you have to look at the last line of a traceback to find the error
    > >> message, and the offending module/method is two lines above that.

    > >
    > > The offending module is the one which instantiates a class it shouldn't
    > > instantiate -- a class which isn't made concrete by overriding all
    > > methods which need to be. What is that module?

    >
    > in real life, it's mymodule.py or mylib.py, in code that you wrote very
    > recently.


    In my experience, it ain't necessarily so. Maybe I happen to work more
    than you do with, for example, people who've had indoctrination into
    "the proper ways to do OOP" (meaning C++ or Java) -- indeed I've found
    that wanting to make abstract classes with methods which raise
    exceptions, rather than just omitting the methods (and thus getting an
    AttributeError if anybody tries calling them on instances of subclasses
    which lack them) has a good correlation with people with the mindset
    typically coming from such doctrine.

    Such people write modules which define abstract classes and also the
    functions that receive instances of such abstract classes (sometimes
    they even check for that with isinstance!) and call methods on them --
    never, in practice, will the same 'mymodule.py' that calls someobj.bar()
    be the one which created 'someobj'; rather 'someobj' will be an argument
    to 'somefunction' -- so you need to continue the study up the stack of
    calls. I'm not saying "it can't be done"! I'm saying it's quite likely
    to take EXTRA MINUTES (that's ALL I'm claiming, remember: so, your
    counterclaim is that such a search *hardly ever takes more than an extra
    119 seconds*!!!) to find out where the should-be-concrete class is
    defined and instantiated, starting from the first call to one method it
    has wrongfully left un-overridden, compared to the case where the class
    statement itself, or the instantiation, raise the exception.

    As for the "code you've written recently": maybe, IF that abstract class
    and supporting infrastructure never changes. If your application-level
    code uses frameworks which change (never a pleasant situation, but it
    does often happen), then it's quite likely the *framework* has changed
    recently, adding a deuced new 'bar' method that now you're supposed to
    override, too. Again, you must find out where you defined the subclass
    which happens to be the class of 'someobj'. Again, I would NOT be
    surprised if it took 121, even 122 seconds... or (gasp!) occasionally
    even more!


    > > The error is generally not the calling of 'bar' on someobj: someobj
    > > arrives as a argument to somefunction from somewhere -- the caller of
    > > somefunction is quite often innocent, in turn -- who BUILT someobj, by
    > > instantiating which class? I think `minutes' is a very fair (not at all
    > > pessimistic) assessment of the time it will generally take to find out.

    >
    > you can blather for years if you want to, but your arguments are just
    > another variant of the "you cannot use whitespace for indentation" stuff
    > we've all seen from time to


    I'm not saying you *cannot* do anything: I'm claiming that, if you do
    things in a slightly different way, it will often save you 120 or more
    seconds of time in finding out where you made a mistake.

    If you think it's worth the bother of putting in 'empty' abstract
    methods which raise NotImplementedException (rather than saving the time
    in the first place by just omitting them), you must be convinced they
    help productivity -- e.g. make it faster to find errors, compared to the
    easier-to-obtain effect of getting an AttributeError. How? And how
    comes a more precise indication of where the override was not done
    (ideally by raising at the class statement, but instantiation time isn't
    all that much worse) appears in your opinion to give NO further added
    value, not even two minutes' worth?

    > time; it's trivial to come up with a large number of theoretical problems
    > with any given approach, but in real life, for real-life programmers,
    > using real-life libraries designed by real-life developers, it's simply
    > not the problem you want it to be.


    I don't "want" anything in particular. I _have_ observed people
    spending over 120 seconds finding out where the errant class was defined
    and what methods it was missing -- with real-life everythings; just the
    same amount of time they'd have spent without the silly
    NotImplementedError to "help", just by reasoning from AttributeError.


    > but not that I'm going to convince you, now that you've spent 15,000
    > characters on this topic...


    Characters are cheap. On the other hand, I can't see how you can
    convince me that what I've observed (and common sense confirms) is an
    illusion, that one can't waste two minutes finding exactly where the
    errant class was defined. Since it takes about 4 minutes to code that
    metaclass, if twice in a project's lifetime, 2 minutes each time are
    saved, one breaks even; if three times, one's gaining. This doesn't
    take into account the large amount of times wasted on this incredibly
    silly thread, of course, but hey, I disclaim any responsibility for it:
    I posted a sketch of a dozen lines of code to solve a real-life problem
    somebody seemed to be having, and found myself unexpectedly under attack
    for 'posting a magnum opus', 'blathering for years', and now even
    arguing against using whitespace for indentation.

    Definitely not the c.l.py I recalled; if that's what one can expect,
    these days, for posting a dozen lines of HTH code -- amount of thanks or
    any expression of gratitude a nice round zero, amount of flames and
    direct attacks as high as I've gotten so far -- I guess I'm not going to
    stay around all that long this time.


    Alex
     
    Alex Martelli, Dec 19, 2004
    #15
  16. John Machin <> wrote:

    > Thank you for the motivation. I now understand what it is attempting to
    > do. Unfortunately it doesn't succeed. Instead of:
    >
    > if v is notimplemented: abstract_methods.append(n)
    >
    > you need:
    > if v.im_func is notimplemented: abstract_methods.append(n)


    Right, thanks -- good thing I clearly posted it was a sketch and
    untested;-). In practice it's no doubt better to mark the function as
    being abstract (e.g. with a function attribute) and otherwise leave it
    alone; people who are keen about abstract classes and methods often want
    to be able to call such abstract methods explicitly (for dubious
    reasons, but it _is_ a common practice in C++, for example -- make a
    method pure virtual, and yet give it a body anyway, for the benefit of
    subclasses' overriding-methods).


    Alex
     
    Alex Martelli, Dec 19, 2004
    #16
  17. John Machin wrote (of Alex Martelli's sketch):
    > Thank you for the motivation. I now understand what it is attempting to
    > do. Unfortunately it doesn't succeed. Instead of:
    > if v is notimplemented: abstract_methods.append(n)
    > you need:
    > if v.im_func is notimplemented: abstract_methods.append(n)


    Here is an implementation from that sketch:

    from inspect import getmembers

    class MustImplement(type):
    '''A metaclass to detect instantiation of abstract classes'''

    def __init__(class_, name, bases, defined):
    super(MustImplement, class_).__init__(name, bases, defined)
    class_._abstract_methods = sorted(name
    for name, value in getmembers(class_)
    if value is NotImplemented)

    def __call__(class_, *args, **kwargs):
    if class_._abstract_methods:
    raise NotImplementedError, 'Abstract class %s needs %s.' % (
    class_.__name__, ', '.join(class_._abstract_methods))
    return super(MustImplement, class_).__call__(*args, **kwargs)


    class Abstract(object):
    '''A class to stick anywhere in an inheritance chain'''
    __metaclass__ = MustImplement


    def notimplemented(method):
    '''A decorator for those who prefer the parameters declared.'''
    return NotImplemented


    if __name__ == '__main__':
    class SomeBase(dict):
    __metaclass__ = MustImplement
    needed = also_needed = NotImplemented
    class Derived(SomeBase):
    def needed(x): return 3
    class Further(Derived):
    def also_needed(x): return 1

    class SomethingMissing(object):
    @notimplemented
    def function(): return 23

    class UsesMissing(SomethingMissing, Abstract): pass

    f = Further() # works
    try:
    d = Derived()
    except NotImplementedError, e:
    print 'Got expected exception:', e
    else:
    raise ValueError, 'Did not get expected exception'

    s = SomethingMissing() # works
    try:
    u = UsesMissing()
    except NotImplementedError, e:
    print 'Got expected exception:', e
    else:
    raise ValueError, 'Did not get expected exception'


    --Scott David Daniels
     
    Scott David Daniels, Dec 19, 2004
    #17
  18. atmosphere on c.l.py (WAS: How about "pure virtual methods"?)

    Alex Martelli wrote:
    > Definitely not the c.l.py I recalled

    [snip]
    > I guess I'm not going to stay around all that long this time.


    That would be a sad loss for all of us out here who very much appreciate
    your very deep knowledge of Python and your willingness to share it.

    I do understand the feeling though; Fredrik Lundh jumped at me only a
    few days ago when I said that I personally found list comprehensions
    more readable than map. I certainly don't mind being advised or
    corrected if I've written incorrect or even just suboptimal code, but
    attacking a person's mental processes seems kind of needlessly
    aggressive to me.

    I don't really have a good solution; despite the unnecessarily vicious
    comments, I don't feel like I can set my newsreader to ignore messages
    from, for example, Fredrik, because his answers, when not attacks, are
    often very insightful. If you find a good solution to this problem,
    please let me know.

    Steve
     
    Steven Bethard, Dec 19, 2004
    #18
  19. Re: atmosphere on c.l.py (WAS: How about "pure virtual methods"?)

    Steven Bethard wrote:

    > I do understand the feeling though; Fredrik Lundh jumped at me only a few days ago when I said
    > that I personally found list comprehensions more readable than map.


    if you think that's what you said, you're clearly didn't read the post you
    replied to very carefully. I read both posts before I made that comment.

    > If you find a good solution to this problem, please let me know.


    well, since I'm not in the ego-stroking business, what if I promise never to
    reply to posts by you, robert, and alex?

    </F>
     
    Fredrik Lundh, Dec 19, 2004
    #19
  20. Noam Raphael

    Doug Holton Guest

    Fredrik Lundh trolled:
    > (I think you could create some kind of drinking game based on the number of


    ....times the nasty trolls pounce on this list?

    No, I think the idea is to actually address the content of someone's
    question, politely and in the *holiday spirit*, not spirits.
     
    Doug Holton, Dec 19, 2004
    #20
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Victor Bazarov

    Re: Pure virtual methods

    Victor Bazarov, Jul 12, 2003, in forum: C++
    Replies:
    8
    Views:
    406
    Ron Natalie
    Jul 14, 2003
  2. qazmlp
    Replies:
    4
    Views:
    432
    David Harmon
    Feb 15, 2004
  3. cppaddict
    Replies:
    12
    Views:
    2,225
    Andy Venikov
    May 26, 2004
  4. Mitch Mooney
    Replies:
    2
    Views:
    374
    Bob Hairgrove
    Jun 18, 2004
  5. John
    Replies:
    6
    Views:
    499
    Alf P. Steinbach
    Jul 5, 2004
Loading...

Share This Page