Changing base class of a big hierarchy

[John J. Lee]

I'm trying to change a base class of a big class hierarchy. The
hierarchy in question is 4DOM (from PyXML). 4DOM has an FtNode class
that defines __getattr__ and __setattr__ that I need to override.

Lots of classes derive from FtNode, often through further base
classes, eg.

HTMLDirectoryElement --> HTMLElement --> Element --> FtNode

There are many leaf classes like HTMLDirectoryElement. The problem
is, I need to get new classes identical to the leaf classes, but
deriving from BrowserFtNode:

class BrowserFtNode(FtNode):
def __init__(self, *args, **kwargs):
...
def __getattr__(self, name):
...
def __setattr__(self, name, value):
...


Obviously, I don't want to manually derive from every damned leaf
class. I want to say "give me a new class hierarchy just like this
one, but derived from BrowserFtNode instead of FtNode". I also don't
want to actually mutate the xml.dom module, of course because other
code may be using it.

What's the easiest way to do that?

Maybe deep-copying the module or something, then fiddling with
__bases__?? Or a metaclass?

While I'm on the subject, does anybody have code to get a list of all
classes in a package that derive from a particular class? Or
something I could use to do that easily?


John

 

[Michele Simionato]

I'm trying to change a base class of a big class hierarchy. <snip>
What's the easiest way to do that?

Maybe deep-copying the module or something, then fiddling with
__bases__?? Or a metaclass?

While I'm on the subject, does anybody have code to get a list of all
classes in a package that derive from a particular class? Or
something I could use to do that easily?


John

Here is a hack that does the job (notice the word 'hack').
Suppose you have a module called x.py:

#x.py module

class B(object):
a='B' # in principle this is a very large class

class C(B):
pass

You want to change the base class B (and therefore C) without touching
the original source code. You can go along the following lines:

import inspect
from types import ModuleType

def MRO(cls):
count=0; out=["MRO of %s:" % cls.__name__]
for c in cls.__mro__:
name=c.__name__
bases=','.join([b.__name__ for b in c.__bases__])
s=" %s - %s(%s)" % (count,name,bases)
if type(c) is not type: s+="[%s]" % type(c).__name__
out.append(s); count+=1
return '\n'.join(out)

def modulesub(s,r,module):
name=module.__name__
source=inspect.getsource(module).replace(s,r)
dic={name: module}; exec source in dic # exec the modified module
module2=ModuleType(name+'2') # creates an an empty module; requires 2.3
for k,v in dic.iteritems(): setattr(module2,k,v) # populates it with dic
return module2

class B(object): # redefinition of the original class B
a='NewB'

import x
Bsource=inspect.getsource(x.B)
Bnewsource=inspect.getsource(B)

x=modulesub(Bsource,Bnewsource,x) # redefine the module

print MRO(x.C)

print x.C.a # will print NewB

The MRO function is here only for convenience, it is quite useful
in large hierarchies. .__subclasses__() is useful too.

HTH,

Michele

 

[John J. Lee]

Here is a hack that does the job (notice the word 'hack').
Suppose you have a module called x.py: [...]
def modulesub(s,r,module):

[...code generation hack snipped...]

import x
Bsource=inspect.getsource(x.B)
Bnewsource=inspect.getsource(B)

x=modulesub(Bsource,Bnewsource,x) # redefine the module

[...]

I agree with you that it's a hack ;-).


John

 

[Bengt Richter]

I'm trying to change a base class of a big class hierarchy. The
hierarchy in question is 4DOM (from PyXML). 4DOM has an FtNode class
that defines __getattr__ and __setattr__ that I need to override.

If FtNode lived in a separate module that was imported, could you
import an impostor module which would override subsequent imports?

If it's all one big file, I guess that won't work, but maybe it would
be a way to factor the problem if you have to do surgery anyway.
Maybe base classes should always come from separate modules, and have
a convention for precomposing the mix before importing the main app
and running that. Just a casual idea, don't know what it'll look like
the morning ;-P

Regards,
Bengt Richter

 

[John J. Lee]

If FtNode lived in a separate module that was imported, could you
import an impostor module which would override subsequent imports?

That's a tempting idea, but is it possible (without a lot of work) to
do it *without* ending up with sys.modules containing my hacked
FtNode? I don't want my hacked class hierarchy to infect other
peoples code that innocently imports xml.dom.

How would you go about it?


John

 

[Alex Martelli]

That's a tempting idea, but is it possible (without a lot of work) to
do it *without* ending up with sys.modules containing my hacked
FtNode? I don't want my hacked class hierarchy to infect other
peoples code that innocently imports xml.dom.

How would you go about it?

Well, you could:
-- import the original FtNode (say "import FtNode")
-- save xxx = sys.modules['FtNode']
-- patch sys.modules['FtNode'] = yourhackedmodule
-- do the deed
-- restore sys.modules['FtNode'] = xxx

Now lots of OTHER modules (imported during "do the deed") might
be 'infected', but regarding sys.modules['FtNode'] specifically,
you're safe. If you can identify the whole set of modules that
need to be saved, temporarily removed, than restored in this way,
the generalization should be easy. This may break down if any
code you're using has 'import' statements in functions or methods,
since said import statements will now access the restored modules
when they execute after you've done the restoration, but most
code imports only at module top-level, so this might work.

I'd still prefer to work on the class hierarchies and keep the
modules alone, due to this kind of issues and fragilities, but
the module-based approach might also be workable.


Alex

 

[John J. Lee]

John J. Lee wrote: [...]

How would you go about it?

Well, you could:
-- import the original FtNode (say "import FtNode")
-- save xxx = sys.modules['FtNode']
-- patch sys.modules['FtNode'] = yourhackedmodule
-- do the deed
-- restore sys.modules['FtNode'] = xxx

[...]

Ick. I don't understand that, and I don't think I want to.

I'll stick with your class-mapping idea, I think.

Thanks all.


John

 

[John J. Lee]

I'll stick with your class-mapping idea, I think.

Thanks all.

I just noticed most of 4DOM's HTML DOM code is automatically
generated, so it's going to be much easier than I thought. Lucky,
because the changes I need are more complicated than I first thought,
and I'd given up on modifying the class hierarchy from outside.


John