Challenge supporting custom deepcopy with inheritance

[Michael H. Goldwasser]

I've been playing around recently with customizing the deepcopy
semantics within an inheritance hierarchy, and run across the
following hurdle.

Assume that class B inherits from class A, and that class A has
legitimately customized its deepcopy semantics (but in a way that is
not known to class B). If we want a deepcopy of B to be defined so
that relevant state inherited from A is copied as would be done for
class A, and with B able to control how to deepcopy the extra state
that it introduces. I cannot immediately find a general way to
properly implement the deepcopy of B.

To make the discussion tangible, I include an outrageously artificial
example in the code fragment below. In this code, class A implements
__deepcopy__ to intentionally create a clone that has a reversed
version of a list instance. The beginning of the main test
demonstrates that an instance of A can successfully be cloned with this
semantics. But the attempt to deepcopy an instance of B fails (the
current failure is because B has not implemented __deepcopy__ and the
inherited version of A.__deepcopy__ assumes the wrong constructor
signature).

The question is how class B should defined so that its instances can
"inherit" this deepcopy semantic. Ideally, I'd like to find a recipe
for class B that works regardless of how A accomplishes its semantics.
That is, I can find ways to rewrite A to be more supportive of B's
goal, but I'd like to know if there is a general solution that could
be used if the author of B doesn't have control over A's
implementation.

Here are some initial thoughts:

* I'm only concerned with single inheritance (thankfully)

* If A had relied on use of __getstate__ and __setstate__ to affect
the desired change, then the inheritance works as desired. But I
have an application where I want to leave the default pickling
behavior alone, and only to change what happens with clones.

* If we did not use inheritance, but instead had a class B that had
an A instance in its state, this is easy. We would call deepcopy
on the A instance to get an acceptable copy of that instance for
the new B. But in the example below, we insist on B being a
subtype of A.

* If A.__deepcopy__ had started
dup = A(self.__aTag)
rather than
dup = self.__class__(self.__aTag)

then the following perverse approach for B.__deepcopy__ succeeds:

def __deepcopy__(self, memo={}):
tempBTag = self.__bTag
tempBList = self.__bList
del self.__bTag # hide these attributes to avoid risk
del self.__bList # that they affect A.__deepcopy__
dupA = A.__deepcopy__(self, memo)
self.__bTag = tempBTag
self.__bList = tempBList
dup = B.__new__(B)
memo[id(self)] = dup
dup.__dict__ = dupA.__dict__
dup.__bTag = copy.deepcopy(self.__bTag, memo)
dup.__bList = copy.deepcopy(self.__bList, memo)
return dup

but this again presumes that we have some control over the
strategy employed by class A, it relies on the fact that
A.__deepcopy__ succeeded when the first parameter was actually an
instance of class B rather than class A, and it seems like a
perverse strategy on the whole, even if it does succeed on this
example. Also, there could be a flaw n that dupA may recursively
have included a reference back to that instance, yet we'd need
such a reference to be to dup not dupA in the end.

I'm hoping that I've just missed a cleaner way to do this.
I welcome any comments, discussion, or suggestions.

With regard,
Michael

-----------------------------------------------------------
import copy

class A(object):
def __init__(self, aTag):
self.__aTag = aTag
self.__aList = []

def addA(self, val):
self.__aList.append(val)

def __repr__(self):
return (
"aTag: " + self.__aTag +
"\naList ("+str(id(self.__aList))+"): " + repr(self.__aList)
)

def __deepcopy__(self, memo={}):
dup = self.__class__(self.__aTag)
memo[id(self)] = dup
dup.__aList = copy.deepcopy(self.__aList, memo)
dup.__aList.reverse()
return dup

class B(A):
def __init__(self, aTag, bTag):
A.__init__(self, aTag)
self.__bTag = bTag
self.__bList = []

def addB(self, val):
self.__bList.append(val)

def __repr__(self):
return (
A.__repr__(self) +
"\nbTag: " + self.__bTag +
"\nbList ("+str(id(self.__bList))+"): " + repr(self.__bList)
)

# How can B support a deepcopy that provides deepcopy of bTag and
# bList while getting the semantics of A's deepcopy for
# attributes defined by A?


if __name__ == '__main__':
print "Test of class A\n==============="
foo = A('alice')
foo.addA(1)
foo.addA(2)

bar = copy.deepcopy(foo)

print "\nOriginal Foo:"
print foo
print "\nOriginal (deepcopy) Bar:"
print bar

bar.addA(0)

print "\n(mutated) Bar:"
print bar
print "Original Foo:"
print foo

print "Test of class B\n==============="
foo = B('alice', 'bob')
foo.addA(1)
foo.addA(2)
foo.addB('hello')
foo.addB('goodbye')

bar = copy.deepcopy(foo) # crashes

print "\nOriginal Foo:"
print foo
print "\nOriginal (deepcopy) Bar:"
print bar

bar.addA(0)
bar.addB('adios')

print "\n(mutated) Bar:"
print bar
print "Original Foo:"
print foo
-----------------------------------------------------------

 

[Aahz]

Assume that class B inherits from class A, and that class A has
legitimately customized its deepcopy semantics (but in a way that is
not known to class B). If we want a deepcopy of B to be defined so
that relevant state inherited from A is copied as would be done for
class A, and with B able to control how to deepcopy the extra state
that it introduces. I cannot immediately find a general way to
properly implement the deepcopy of B.

[...]

class A(object):
def __init__(self, aTag):
self.__aTag = aTag
self.__aList = []

IMO, your problem starts right here. Not only are you using customized
attributes for each class, you're using class-private identifiers. You
would vastly simplify your work if you switch to single-underscore
attributes.

 

[Michael H. Goldwasser]

class A(object):
def __init__(self, aTag):
self.__aTag = aTag
self.__aList = []

IMO, your problem starts right here. Not only are you using customized
attributes for each class, you're using class-private identifiers. You
would vastly simplify your work if you switch to single-underscore
attributes.

Hi Aahz,

I intentionally chose the class-private identifiers in my artificial
example to emphasize that I was looking for a solution in which
class B did not have to rely on particular knowledge of class A's
implementation. That said, switching to single-underscores does not
address the issue raised in the original post.

Michael

 

[Aahz]

Michael Goldwasser:

class A(object):
def __init__(self, aTag):
self.__aTag = aTag
self.__aList = []

IMO, your problem starts right here. Not only are you using customized
attributes for each class, you're using class-private identifiers. You
would vastly simplify your work if you switch to single-underscore
attributes.

I intentionally chose the class-private identifiers in my artificial
example to emphasize that I was looking for a solution in which
class B did not have to rely on particular knowledge of class A's
implementation. That said, switching to single-underscores does not
address the issue raised in the original post.

Not directly, but it does simplify possible solutions. For example, you
could use a straightforward getattr() approach where the class contains
an attribute listing all the deep-copyable attributes. You could even
use name-munging so that attributes can have an accompanying
ATTR_deepcopy() method for custom code so that your main __deepcopy__
method stays the same in subclasses.

(Obviously, this trick does in fact still work if you use private
attributes and do the name-mangling yourself, but I find that distasteful
for production code unless it's absolutely required.)
--
Aahz ([email protected]) <*> http://www.pythoncraft.com/

"Given that C++ has pointers and typecasts, it's really hard to have a
serious conversation about type safety with a C++ programmer and keep a
straight face. It's kind of like having a guy who juggles chainsaws
wearing body armor arguing with a guy who juggles rubber chickens wearing
a T-shirt about who's in more danger." --Roy Smith, c.l.py, 2004.05.23