Code design for a sub-class of built-ins

[Steven D'Aprano]

I'm having problems with sub-classes of built-in types.

Here is a contrived example of my subclass. It isn't supposed
to be practical, useful code, but it illustrates my problem.

class MyStr(str):
"""Just like ordinary strings, except it exhibits special behaviour
for one particular value.
"""
magic = "surprise"
def __init__(self, value):
str.__init__(self, value)
def __len__(self):
if self == self.magic:
print "Nobody expects the Spanish Inquisition!"
return str.__len__(self)
def upper(self):
if self == self.magic:
print "Nobody expects the Spanish Inquisition!"
return str.upper(self)
# and so on for every last string method...


The obvious problem is, I have to create a custom method for every string
method -- and if strings gain any new methods in some future version of
Python, my subclass won't exhibit the correct behaviour.

Here's a slightly different example:

class MyInt(int):
"""Like an int but with special behaviour."""
def __init__(self, value, extra):
int.__init__(self, value=None)
self.extra = extra
def __add__(self, other):
if self.extra is None:
return int.__add__(self, other)
else:
return int.__add__(self, other) + self.extra
# and again with all the other methods

This one doesn't even work!
Traceback (most recent call last):
File "<stdin>", line 1, in ?
TypeError: int() can't convert non-string with explicit base

Looks like my __init__ isn't even being called here. Why not, and how do I
fix this?

Is there a way to subclass built-in types without needing to write the
same bit of code for each and every method?

Am I approaching this the wrong way? Is there a better design I could be
using?

Thanks,

 

[Bruno Desthuilliers]

I'm having problems with sub-classes of built-in types.

Here is a contrived example of my subclass. It isn't supposed
to be practical, useful code, but it illustrates my problem.

class MyStr(str):
"""Just like ordinary strings, except it exhibits special behaviour
for one particular value.
"""
magic = "surprise"

def __init__(self, value):
str.__init__(self, value)

You don't need to override __init__ here.

def __len__(self):
if self == self.magic:
print "Nobody expects the Spanish Inquisition!"
return str.__len__(self)
def upper(self):
if self == self.magic:
print "Nobody expects the Spanish Inquisition!"
return str.upper(self)
# and so on for every last string method...

My my my....

The obvious problem is, I have to create a custom method for every string
method --

which makes subclassing almost useless - except to pass isinstance() tests.

and if strings gain any new methods in some future version of
Python, my subclass won't exhibit the correct behaviour.

You could replace subclassing by composition/delegation using the
__getattr__ hook, but this would break tests on type/class

Or you could keep subclassing and use the __getattribute__ hook, but
this is more tricky and IIRC may have negative impact on lookup perfs.

Or you could use a metaclass to decorate (appropriate) str methods with
a decorator doing the additional stuff.

choose your poison !-)

Here's a slightly different example:

class MyInt(int):
"""Like an int but with special behaviour."""
def __init__(self, value, extra):
int.__init__(self, value=None)
self.extra = extra

Won't work. You need to override the __new__ staticmethod. Look for
appropriate doc here:
http://www.python.org/download/releases/2.2.3/descrintro/#__new__
(FWIW, read the whole page)

(snip)

Looks like my __init__ isn't even being called here. Why not, and how do I
fix this?

cf above

Is there a way to subclass built-in types without needing to write the
same bit of code for each and every method?

cf above

Am I approaching this the wrong way? Is there a better design I could be
using?

probably !-)

HTH

 

[Dennis Lee Bieber]

def __init__(self, value, extra):
int.__init__(self, value=None)

Pardon... You are trying, if I understand this, to create an
integer with an initial value of None -- and None is not an integer
value.

I suspect what you really want (I'm not going to open an interpreter
to test) is:

def __init__(self, value, extra=None):
int.__init__(self, value)
--
Wulfraed Dennis Lee Bieber KD6MOG
(e-mail address removed) (e-mail address removed)
HTTP://wlfraed.home.netcom.com/
(Bestiaria Support Staff: (e-mail address removed))
HTTP://www.bestiaria.com/

 

[Alex Martelli]

The obvious problem is, I have to create a custom method for every string
method -- and if strings gain any new methods in some future version of
Python, my subclass won't exhibit the correct behaviour.

As others already suggested, automating such decoration is pretty easy;
you can do it with either a custom metaclass or a simple post-processing
of your class in a loop. Untested details below, but the general idea
would be something like:

class SDAstr(str):
magic = 'whatever'

def _SDAdecorate(methodname, strmethod):
def decorated(self, *a, **k):
if self == self.magic: print "whatever!"
return strmethod(self, *a, **k)
setattr(SDAstr, methodname, decorated)

for methodname, strmethod in \
inspect.getmembers(str, inspect.ismethoddescriptor):
_SDAdecorate(methodname, strmethod)

and variants thereof (to provide more accurate metainformation with the
decorated methods -- name, docstring, signature, whatever; and/or to
encapsulate things in a custom metaclass; whatever).

class MyInt(int):
"""Like an int but with special behaviour."""
def __init__(self, value, extra): ...
Looks like my __init__ isn't even being called here. Why not, and how do I
fix this?

Override __new__, which is what invariably gets called (before __init__,
which only gets called if __new__ returns an instance of the class that
you're instantiating).


Alex

 

[Steven D'Aprano]

You don't need to override __init__ here.

Perhaps not, but in a more realistic example I might need to.

My my my....

Exactly. Don't think I don't know this is a horrible strategy -- that's
why I'm asking for ideas for a better way to do it *wink*

which makes subclassing almost useless - except to pass isinstance()
tests.
Yep.



You could replace subclassing by composition/delegation using the
__getattr__ hook, but this would break tests on type/class

Or you could keep subclassing and use the __getattribute__ hook, but
this is more tricky and IIRC may have negative impact on lookup perfs.

Or you could use a metaclass to decorate (appropriate) str methods with
a decorator doing the additional stuff.

choose your poison !-)

Interesting... That will give me something to experiment with.


Thanks,

 

[Steven D'Aprano]

I suspect what you really want (I'm not going to open an interpreter
to test) is:

def __init__(self, value, extra=None):
int.__init__(self, value)

Yes, that's exactly what I meant -- it was a copy-and-paste and I didn't
clean it up correctly.

Thanks,

 

[Bruno Desthuilliers]

Perhaps not, but in a more realistic example I might need to.

Perhaps, but I can only comment on the example you gave !-)
Also, *you* are aware of this, but please keep in mind that cl.py is
read by a lot of newbies.

(snip)

Interesting... That will give me something to experiment with.

AFAICT, the last solution is probably the better of the three (well, at
least it's the better I can think of actually).

For the record, care to give more informations about your real use case?

 

[Steven D'Aprano]

As others already suggested, automating such decoration is pretty easy;
you can do it with either a custom metaclass or a simple post-processing
of your class in a loop. Untested details below, but the general idea
would be something like:

That's great, thanks. You've given me an angle of attack to take and see
where it leads.

 

[Steven D'Aprano]

[snip]

For the record, care to give more informations about your real use case?

Equal parts a learning exercise and a simple example of a genetic
algorithm.

I felt that the natural way to approach this would be for an object to
mutate itself, rather than have an external function that operated on a
string and returned a new string. The obvious approach was to subclass str
and give it methods to modify itself in place, but of course strings are
immutable.

This got me wondering how hard it would be to create a mutable string
class, whether I should look at subclassing list (to get the mutability)
and then add string-like methods to it, or try something completely
different.

A little bit of experimentation soon had me realising that I didn't
understand Python's new-style class model well enough to do these things
properly, hence the learning exercise.


Thanks for everybody's help on this.