overriding methods - two questions

[Donn Ingle]

Hi,
Here's a framework for the questions:

--- In a module, part of an API ---
class Basis ( object ):
def foo ( self, arg ):
pass

--- In user's own code ---
class Child ( Basis ):
def foo ( self, not, sure ):
...


Question 1:

Given that the user of the API can choose to override foo() or not, how can
I control the signature that they use? In the example the user has chosen
bad arguments and Python will complain, but it's describing the sig of the
*overridden* method and not the one in the parent class.

Is there some way I can control the error message to make it clear to the
user that they are using the signature of foo() incorrectly?

Question 2:

Say I am in class Basis, doing a loop and I have a list of Child objects. I
want to run the foo() method for each one that *has* a foo() method. i.e.
user has done this:

class Sam ( Child ):
...
*Sam does not define foo()

class Judy ( Child ):
def foo ( self, arg ):
...
* Judy does define foo()

Instances of Sam and Judy have been put into the list (within the instance)
of Basis. I want Basis to detect that Judy has foo() and run it.

I can handle question 2 by using a flag that must be set by the user.
Something like:
class Judy ( child ):
def __init__( self ):
self.pleaseCallFoo = true

And now, Basis can check for that var and only then call foo(), but this is
ugly and means more for the user to learn API-wise.

Any ideas?
/d

 

[Paul McGuire]

Hi,
Here's a framework for the questions:

--- In a module, part of an API ---
class Basis ( object ):
def foo ( self, arg ):
pass

--- In user's own code ---
class Child ( Basis ):
def foo ( self, not, sure ):
...

Question 1:

Given that the user of the API can choose to override foo() or not, how can
I control the signature that they use? In the example the user has chosen
bad arguments and Python will complain, but it's describing the sig of the
*overridden* method and not the one in the parent class.

Actually, Python is complaining about your user's poor choice of
argument names. 'not' is a reserved keyword. Change it to 'naught' or
'knot' or 'not_' and Python will accept this just fine.

Whether this is a good idea or not is a separate question. But given
Python's philosophy of "you are the human, so you must know what you
are doing" (which is both an assumption and a directive), I don't
think you will find much language machinery to prevent it.

-- Paul


-- Paul

 

[bearophileHUGS]

Donn Ingle:

Say I am in class Basis, doing a loop and I have a list of Child objects. I
want to run the foo() method for each one that *has* a foo() method.

This may help (on an old Python version):
.... def foo(self): pass
....

children = [Sam(), Judy(), Sam()]
for child in children: hasattr(child, "foo")

....
False
True
False

Bye,
bearophile

 

[Donn Ingle]

Actually, Python is complaining about your user's poor choice of

argument names. 'not' is a reserved keyword.

My example was poor, but my actual test code did't use 'not'. Python simply
checks the use of foo() to the local sig of foo() and does not go up the
chain. This is understandable and your next answer is more-or-less what I
was expecting.

Python's philosophy of "you are the human, so you must know what you
are doing" (which is both an assumption and a directive), I don't
think you will find much language machinery to prevent it.

Yeah. I guess I was hoping there'd be some clever trick to do it.

/d

 

[Bruno Desthuilliers]

Donn Ingle a écrit :

Hi,
Here's a framework for the questions:

--- In a module, part of an API ---
class Basis ( object ):
def foo ( self, arg ):
pass

--- In user's own code ---
class Child ( Basis ):
def foo ( self, not, sure ):
...


Question 1:

Given that the user of the API can choose to override foo() or not, how can
I control the signature that they use?

While technically possible (using inspect.getargspec), trying to make
your code idiot-proof is a lost fight and a pure waste of time.

Question 2:

Say I am in class Basis, doing a loop and I have a list of Child objects. I
want to run the foo() method for each one that *has* a foo() method. i.e.
user has done this:

class Sam ( Child ):
...
*Sam does not define foo()

class Judy ( Child ):
def foo ( self, arg ):
...
* Judy does define foo()

Instances of Sam and Judy have been put into the list (within the instance)
of Basis. I want Basis to detect that Judy has foo() and run it.

I can handle question 2 by using a flag that must be set by the user.
Something like:
class Judy ( child ):
def __init__( self ):
self.pleaseCallFoo = true

And now, Basis can check for that var and only then call foo(), but this is
ugly and means more for the user to learn API-wise.
Indeed.

Any ideas?

Quite a few, but I don't have enough context to tell which one would be
the best - nor why you want to do such a thing. Anyway, the simplest is
to just check :

for child in self.childrens:
if 'foo' in child.__class__.__dict__:
child.foo()

but this won't call foo for :

class Dude(Judy):
pass

Don't know if that's what you want. If not (ie, you want to call
child.foo if foo is not Basis.foo), then:

for child in self.childrens:
if child.foo.im_func is not self.foo.im_func:
child.foo()

HTH

 

[Donn Ingle]

This may help (on an old Python version):

... def foo(self): pass
...

children = [Sam(), Judy(), Sam()]
for child in children: hasattr(child, "foo")

...
False
True
False

That's not what my tests are showing. While Sam has no foo, it's coming from
(in my OP) Child (which is the parent class), so hasattr(Sam(),"foo") is
returning True.

/d

 

[Donn Ingle]

for child in self.childrens:

if 'foo' in child.__class__.__dict__:
child.foo()

Bruno, you're the man! I really must take the time to look into all those
under-under score things!

Thanks.

/d

 

[Bruno Desthuilliers]

Donn Ingle a écrit :

Bruno, you're the man! I really must take the time to look into all those
under-under score things!

Knowing Python's object model can help, indeed !-)

Now while this kind of stuff is ok in the low-level parts of a
framework, it shouldn't be seen too much in application code IMHO.

 

[davisn90210]

This may help (on an old Python version):

class Sam: pass
class Judy:

... def foo(self): pass
...

children = [Sam(), Judy(), Sam()]
for child in children: hasattr(child, "foo")

...
False
True
False

That's not what my tests are showing. While Sam has no foo, it's coming from
(in my OP) Child (which is the parent class), so hasattr(Sam(),"foo") is
returning True.

/d

But also in your OP: "I want to run the foo() method for each one that
*has* a foo() method ...." So hasattr(child, "foo") really does
answer the question as posed, even if it's not really what you want.
I am curious as to why you want to go through such contortions. What
do you gain. What happens, for example, if a subclass of Judy is
passed in that does not override foo? Should foo be called in that
case or not?

--Nathan Davis

 

[Steven D'Aprano]

While technically possible (using inspect.getargspec), trying to make
your code idiot-proof is a lost fight and a pure waste of time.

Worse: it's actually counter-productive!

The whole idea of being able to subclass a class means that the user
should be able to override foo() *including* the signature. Why do you
want to stop them? It's their subclass, not yours. You don't know what
arguments it needs.

Let me give a practical example: in mathematics there is a construct
known as a continued fraction. What it is isn't especially important, if
you're curious you can google for it. If you were defining a class for
continued fractions, you might do this:

class ContinuedFraction(object):
def __init__(self, list_of_numerators, list_of_denominators):
pass
# lots of other methods

cf = ContinuedFraction([3, 7, 2, 8, 9, 5], [2, 3, 1, 5, 3, 7])

If I wanted to subclass your ContinuedFraction class to provide regular
continued fractions, I could do this:

class RegularCF(ContinuedFraction):
def __init__(self, *denominators):
numerators = [1]*len(denominators)
super(RegularCF, self).__init__(numerators, denominators)
# all other methods are inherited from super-class without change

cf = RegularCF(4, 9, 1, 2, 6, 3)


But if you did what you're proposing to do, I couldn't do that. I'd need
to do something silly like this:

class RegularCF(ContinuedFraction):
def __init__(self, list_of_numerators, list_of_denominators):
numerators = [1]*len(list_of_denominators)
super(RegularCF, self).__init__(numerators, list_of_denominators)

cf = RegularCF(None, [4, 9, 1, 2, 6, 3])


just so that the signatures matched. What a waste of time.

And worse, what if my subclass needed *more* arguments than your
signature provided? The hoops I would have to jump through would not only
be flaming, they'd be spinning and flying through the air, with rotating
knives and trip-wires.

 

[George Sakkis]

Worse: it's actually counter-productive!

The whole idea of being able to subclass a class means that the user
should be able to override foo() *including* the signature. Why do you
want to stop them?

Primarily because of this: http://en.wikipedia.org/wiki/Liskov_substitution_principle.

Let me give a practical example:

(snip misleading example involving __init__)

Constructors are in general different from other regular methods
because you refer explicitly to the class name when creating an
instance; that is, the client writes "f = ContinuedFraction(...)" or
"f = RegularCF(...)" so it's not necessary to have the same signature
in __init__. For all other methods though, given that you have an
instance x so that isinstance(x, ContinuedFraction), the client should
be able to say x.foo(arg, kwd=v) without having to know whether
x.__class__ is ContinuedFraction. If not, you have a leaky abstraction
[1], i.e. in your example, a RegularCF is not really a
ContinuedFraction.

George


[1] http://www.joelonsoftware.com/articles/LeakyAbstractions.html

 

[Donn Ingle]

While technically possible (using inspect.getargspec), trying to make

Worse: it's actually counter-productive!
The whole idea of being able to subclass a class means that the user
should be able to override foo() *including* the signature.

Steven,
In this case, the def in question is named Draw. It's args are context and
framenumber. The body is 'pass'. I want users to use the exact signature
(but they can change the varnames to suit) because the draw() method is
*not* being called by the user but *by* my API (in a timeout loop).

So, my API is sending blah.Draw( cairo.context, currentFrame ) *to* the
user's own object.Draw.Thereafter, whatever Cairo drawing commands they use
in their override of Draw() is what gets drawn -- if that makes sense

I guess that's the long way of saying it's a call-back function

So, I'm trying to guarantee that they don't mess with it to make life easier
for them. As Bruno has mentioned, this is up to the human and so it'll be a
rule in the docs instead.

/d

 

[Donn Ingle]

I am curious as to why you want to go through such contortions.  What

do you gain.

for obj in list:
if obj has a foo() method:
a = something
b = figureitout ( )
object.foo ( a, b )

I am accepting objects of any class on a stack. Depending on their nature I
want to call certain methods within them. They can provide these methods or
not.

What happens, for example, if a subclass of Judy is
passed in that does not override foo?  Should foo be called in that
case or not?

No.

Bruno has given me a good solution:

for obj in list:
if 'foo' in obj.__class__.__dict__:
etc.

Although I am concerned that it's a loop ('in') and so may be slower than
some other way to detect foo().

So, that's the story.

/d

 

[Dennis Lee Bieber]

In this case, the def in question is named Draw. It's args are context and
framenumber. The body is 'pass'. I want users to use the exact signature
(but they can change the varnames to suit) because the draw() method is
*not* being called by the user but *by* my API (in a timeout loop).

If it is totally abstract (must be defined by the subclasses), I'd
suggest the body should not be "pass" but rather a "raise" (is
"NotImplemented" a common exception?)

So, my API is sending blah.Draw( cairo.context, currentFrame ) *to* the
user's own object.Draw.Thereafter, whatever Cairo drawing commands they use
in their override of Draw() is what gets drawn -- if that makes sense

This is strictly a documentation matter, in my mind. Python does not
offer any means to enforce the calling sequence of an "override method".

I guess that's the long way of saying it's a call-back function

As a "call-back", it is up the the writer of the call-back to follow
the expectations of the documented API. If they don't... <boom>

You might be able to wrap YOUR calling code with a try/except block
to trap errors if the callback doesn't like the "documented API"
arguments.
--
Wulfraed Dennis Lee Bieber KD6MOG
(e-mail address removed) (e-mail address removed)
HTTP://wlfraed.home.netcom.com/
(Bestiaria Support Staff: (e-mail address removed))
HTTP://www.bestiaria.com/

 

[Donn Ingle]

This is strictly a documentation matter, in my mind. Python does not

offer any means to enforce the calling sequence of an "override method".

Yes, I hear you.

You might be able to wrap YOUR calling code with a try/except block
to trap errors if the callback doesn't like the "documented API"
arguments.

Good plan. Thanks, dunno why I didn't think of that!

/d

 

[Gabriel Genellina]

for obj in list:
if 'foo' in obj.__class__.__dict__:
etc.

Although I am concerned that it's a loop ('in') and so may be slower than
some other way to detect foo().

'in' for dictionaries is fast and runs in constant time. You may be
thinking about 'in' for lists, that runs in time proportional to the
number of elements.

 

[Steven D'Aprano]

Steven,
In this case, the def in question is named Draw. It's args are context
and
framenumber. The body is 'pass'. I want users to use the exact signature
(but they can change the varnames to suit) because the draw() method is
*not* being called by the user but *by* my API (in a timeout loop).

You don't know that. How can you possibly guarantee that the user won't
find some other use for the draw() method as well as passing it to your
API? Maybe your application turns out to be a huge success, and people
start extending it in all sorts of ways you never ever imagined -- or at
least they would if you didn't needlessly lock down the classes they can
use.

Or, if you can't imagine that, imagine that in six months time *you*
decide to extend the application, and have another purpose for that same
draw() callback, but need a slightly different signature.

Which suggests that maybe your API should specify keyword arguments
rather than positional args. That way you can do something like this:

# (I'm guessing) in the main video editing loop:
blah.Draw(context=cario.context, frame=currentFrame)

# we want to display the third frame as we draw
blah.Draw(context=another_context, frame=3, window=preview)

# don't forget to flip the title frame
blah.Draw(flip=True, context=cairo.context, frame=title)


BTW, it is a convention for method names to be lower case, and classes to
be Title case. Seeing something like obj.Draw, most(?) Python developers
will expect that the Draw attribute of obj is itself a class:
.... class Draw(object):
.... pass
....<class '__main__.Draw'>

 

[Peter Otten]

for obj in list:
if obj has a foo() method:
a = something
b = figureitout ( )
object.foo ( a, b )

I am accepting objects of any class on a stack. Depending on their nature I
want to call certain methods within them. They can provide these methods or
not.

No.

Hmm, you are missing the point of inheritance. Behaviour not
meant to be shared by all subclasses should not be implemented in
the base class, i. e. either Sam should not inherit from Child or Child
should not implement foo().

However, here is another hack to hide an inherited method:
.... def foo(self): print "Child's foo"
.... def __str__(self): return self.__class__.__name__
.... .... @property
.... def foo(self): raise AttributeError
.... .... def foo(self): print "Judy's foo"
....

for item in [Sam(), Judy()]:

.... try:
.... foo = item.foo
.... except AttributeError:
.... print "no foo for", item
.... else:
.... foo()
....
no foo for Sam
Judy's foo

If you find try ... except too verbose -- it works with hasattr(), too.

Bruno has given me a good solution:

for obj in list:
if 'foo' in obj.__class__.__dict__:
etc.

Although I am concerned that it's a loop ('in') and so may be slower
than some other way to detect foo().

__dict__ is a dictionary (who would have thought), the "in" operator
therefore does not trigger a loop and is very fast.

Peter

 

[Donn Ingle]

*not* being called by the user but *by* my API (in a timeout loop).

You don't know that. How can you possibly guarantee that the user won't
find some other use for the draw() method

Well, as per your good examples, I would answer that as the parameters
passed to draw() grow in number, so the API is actually growing and so the
draw() call will be updated. Or, other calls can be introduced like
drawPreview() etc. (Better cos it won't break old code.)

The way it is now, in heavy alpha , is that the draw() is *only* called
outwards and does not return or call parentClass.draw( self, ...) back in

BTW, it is a convention for method names to be lower case, and classes to
be Title case. Seeing something like obj.Draw, most(?) Python developers
will expect that the Draw attribute of obj is itself a class:

Thanks, I'm pretty damn unorganized in that way. Is it convention to do:

class BigLetterAndCamels():
def smallLetterAndCamels()

or
def smallletterandnocamels()

?

/d

 

[Donn Ingle]

Thanks, good tips all-round. I have it working okay at the moment with all
the suggestions. It may break in future, but that's another day


/d