Swapping superclass from a module

[Emanuele D'Arrigo]

Hi everybody,

let's assume I have a module with loads of classes inheriting from one
class, from the same module, i.e.:

## myFile.py
class SuperClass(object)
class SubClass1(SuperClass)
class SubClass2(SuperClass)
class SubClass3(SuperClass)

In a separate file I also have:

## myOtherFile.py
class NewSuperClass(object)

Now, let's also assume that myFile.py cannot be changed or it's
impractical to do so. Is there a way to replace the SuperClass at
runtime, so that when I instantiate one of the subclasses
NewSuperClass is used instead of the original SuperClass provided by
the first module module?

That was the generic case. Would the solution change much if
NewSuperClass was actually inheriting from SuperClass, effectively
wedging itself between the SuperClass and the SubClasses?

Manu

 

[Arnaud Delobelle]

Hi everybody,

let's assume I have a module with loads of classes inheriting from one
class, from the same module, i.e.:

## myFile.py
class SuperClass(object)
class SubClass1(SuperClass)
class SubClass2(SuperClass)
class SubClass3(SuperClass)

In a separate file I also have:

## myOtherFile.py
class NewSuperClass(object)

Now, let's also assume that myFile.py cannot be changed or it's
impractical to do so. Is there a way to replace the SuperClass at
runtime, so that when I instantiate one of the subclasses
NewSuperClass is used instead of the original SuperClass provided by
the first module module?

That was the generic case. Would the solution change much if
NewSuperClass was actually inheriting from SuperClass, effectively
wedging itself between the SuperClass and the SubClasses?

Here is an attempt to do this. Imagine there is a module 'modfoo.py'
like this:

--------------------------------------------------
class Base(object):
def bar(self):
print 'Base.bar'

class A(Base):
def bar(self):
Base.bar(self)
print 'A.bar'

class B(Base):
def bar(self):
super(B, self).bar()
print 'B.bar'
--------------------------------------------------

The following script will patch the module so that all subclasses of
Base in module modfoo have their new superclass changed. I called it
'patchfoo.py'.

--------------------------------------------------
import modfoo

# This is the new base
class Wedge(modfoo.Base):
def bar(self):
super(Wedge, self).bar()
print 'Wedge.bar'

# Insert Wedge into each subclass of modfoo.Base
for subclass in modfoo.Base.__subclasses__():
if subclass.__module__ != 'modfoo': continue
attrs = dict(item for item in subclass.__dict__.items()
if item[0][:2] != '__')
name = subclass.__name__
setattr(modfoo, name, type(name, (Wedge,), attrs))

# Replace modfoo.Base with Wedge
modfoo.Base = Wedge

# Tests
a = modfoo.A()
b = modfoo.B()

for obj in 'a' ,'b':
call = '%s.bar()' % (obj,)
print '-'*5, call
exec call
--------------------------------------------------

Now let's try it:

marigold:junk arno$ python patchfoo.py
----- a.bar()
Base.bar
Wedge.bar
A.bar
----- b.bar()
Base.bar
Wedge.bar
B.bar

Of course, there are plenty of ways this could break.

 

[Emanuele D'Arrigo]

# Insert Wedge into each subclass of modfoo.Base
for subclass in modfoo.Base.__subclasses__():
    if subclass.__module__ != 'modfoo': continue
    attrs = dict(item for item in subclass.__dict__.items()
                      if item[0][:2] != '__')
    name = subclass.__name__
    setattr(modfoo, name, type(name, (Wedge,), attrs))

# Replace modfoo.Base with Wedge
modfoo.Base = Wedge

That-is-neat! I'm impressed. It took me a while to understand it all:
I didn't know about __subclasses__, the way you fill the dictionary is
completely novel to me and the use of type(name, tuple, attributes)
was also completely novel to me. But eventually I got it and it's
quite neat. Thank you.

Of course, there are plenty of ways this could break.

Uh-oh. Ok, you got me all excited and now I have to read the fine
prints. So, how can it break?

Ciao!

Manu

 

[Arnaud Delobelle]

# Insert Wedge into each subclass of modfoo.Base
for subclass in modfoo.Base.__subclasses__():
    if subclass.__module__ != 'modfoo': continue
    attrs = dict(item for item in subclass.__dict__.items()
                      if item[0][:2] != '__')
    name = subclass.__name__
    setattr(modfoo, name, type(name, (Wedge,), attrs))

# Replace modfoo.Base with Wedge
modfoo.Base = Wedge

That-is-neat! I'm impressed. It took me a while to understand it all:
I didn't know about __subclasses__, the way you fill the dictionary is
completely novel to me and the use of type(name, tuple, attributes)
was also completely novel to me. But eventually I got it and it's
quite neat. Thank you.

Of course, there are plenty of ways this could break.

Uh-oh. Ok, you got me all excited and now I have to read the fine
prints. So, how can it break?

I can think of 3 easy ways (and there may be more):

1. For simplicity I chose to ignore attributes that start with '__' but of
course in real life some of them would need to be included (__init__ &
co...).

2. It wouldn't work with multiple inheritance.

3. If the original classes are not just defined, but also used when the
module is loaded then this will cause problems. E.g. imagine that the
file 'modfoo.py' in my example contains the additional line

AA = A

Then when A is patched, AA will remain bound to the original class A,
and there is not much that can be done about this.


1. and 2. are only problems because my code was proof-of-concept and can
be addressed without difficulty I think.

3. is a more serious limitation, but if your module doesn't use the
classes (i.e. it just defines them), then the method might work OK I
guess.

 

[Steven D'Aprano]

Hi everybody,

let's assume I have a module with loads of classes inheriting from one
class, from the same module, i.e.: [...]
Now, let's also assume that myFile.py cannot be changed or it's
impractical to do so. Is there a way to replace the SuperClass at
runtime, so that when I instantiate one of the subclasses NewSuperClass
is used instead of the original SuperClass provided by the first module
module?

That's called "monkey patching" or "duck punching".

http://en.wikipedia.org/wiki/Monkey_patch

http://wiki.zope.org/zope2/MonkeyPatch

http://everything2.com/title/monkey%20patch

 

[Terry Reedy]

Hi everybody,

let's assume I have a module with loads of classes inheriting from one
class, from the same module, i.e.: [...]
Now, let's also assume that myFile.py cannot be changed or it's
impractical to do so. Is there a way to replace the SuperClass at
runtime, so that when I instantiate one of the subclasses NewSuperClass
is used instead of the original SuperClass provided by the first module
module?

That's called "monkey patching" or "duck punching".

http://en.wikipedia.org/wiki/Monkey_patch

http://wiki.zope.org/zope2/MonkeyPatch

http://everything2.com/title/monkey%20patch

If the names of superclasses is resolved when classes are instantiated,
the patching is easy. If, as I would suspect, the names are resolved
when the classes are created, before the module becomes available to the
importing code, then much more careful and extensive patching would be
required, if it is even possible. (Objects in tuples cannot be
replaced, and some attributes are not writable.)

tjr

 

[Michele Simionato]

Try this:

class Base(object):
pass

class C(Base):
pass

class NewBase(object):
pass

C.__bases__ = (NewBase,)

help(C)

 

[Peter Otten]

Hi everybody,

let's assume I have a module with loads of classes inheriting from one
class, from the same module, i.e.: [...]
Now, let's also assume that myFile.py cannot be changed or it's
impractical to do so. Is there a way to replace the SuperClass at
runtime, so that when I instantiate one of the subclasses NewSuperClass
is used instead of the original SuperClass provided by the first module
module?

That's called "monkey patching" or "duck punching".

http://en.wikipedia.org/wiki/Monkey_patch

http://wiki.zope.org/zope2/MonkeyPatch

http://everything2.com/title/monkey%20patch

If the names of superclasses is resolved when classes are instantiated,
the patching is easy. If, as I would suspect, the names are resolved
when the classes are created, before the module becomes available to the
importing code, then much more careful and extensive patching would be
required, if it is even possible. (Objects in tuples cannot be
replaced, and some attributes are not writable.)

It may be sufficient to patch the subclasses:

$ cat my_file.py
class Super(object):
def __str__(self):
return "old"

class Sub(Super):
def __str__(self):
return "Sub(%s)" % super(Sub, self).__str__()

class Other(object):
pass

class SubSub(Sub, Other):
def __str__(self):
return "SubSub(%s)" % super(SubSub, self).__str__()

if __name__ == "__main__":
print Sub()

$ cat main2.py
import my_file
OldSuper = my_file.Super

class NewSuper(OldSuper):
def __str__(self):
return "new" + super(NewSuper, self).__str__()

my_file.Super = NewSuper
for n, v in vars(my_file).iteritems():
if v is not NewSuper:
try:
bases = v.__bases__
except AttributeError:
pass
else:
if OldSuper in bases:
print "patching", n
v.__bases__ = tuple(NewSuper if b is OldSuper else b
for b in bases)


print my_file.Sub()
print my_file.SubSub()
$ python main2.py
patching Sub
Sub(newold)
SubSub(Sub(newold))

Peter

 

[Emanuele D'Arrigo]

Wow, thank you all. Lots of ideas and things to try! I wish I knew
which one is going to work best. The module I'm trying to (monkey!)
patch is pxdom, and as it is a bit long (5700 lines of code.... in one
file!) I'm not quite sure if the simplest patching method will work or
the more complicated ones are necessary. Let me chew on it for a
little while. Luckilly pxdom has a test suite and I should be able to
find out quite quickly if a patch has broken something. Let me try and
chew on it, I'll report back.

Thank you all!

Manu

 

[Terry Reedy]

Terry Reedy wrote:


It may be sufficient to patch the subclasses:

I was not sure if __bases__ is writable or not. There is also __mro__
to consider.