pickle/unpickle class which has changed

[Neal Becker]

What happens if I pickle a class, and later unpickle it where the class now has
added some new attributes?

 

[Steven D'Aprano]

What happens if I pickle a class, and later unpickle it where the class
now has added some new attributes?

Why don't you try it?

py> import pickle
py> class C:
.... a = 23
....
py> c = C()
py> pickled = pickle.dumps(c)
py> C.b = 42 # add a new class attribute
py> d = pickle.loads(pickled)
py> d.a
23
py> d.b
42


Unless you mean something different from this, adding attributes to the
class is perfectly fine.

But... why are you dynamically adding attributes to the class? Isn't that
rather unusual?

 

[Peter Otten]

Why don't you try it?

py> import pickle
py> class C:
... a = 23
...
py> c = C()
py> pickled = pickle.dumps(c)
py> C.b = 42 # add a new class attribute
py> d = pickle.loads(pickled)
py> d.a
23
py> d.b
42


Unless you mean something different from this, adding attributes to the
class is perfectly fine.

But... why are you dynamically adding attributes to the class? Isn't that
rather unusual?

The way I understand the problem is that an apparently backwards-compatible
change like adding a third dimension to a point with an obvious default
breaks when you restore an "old" instance in a script with the "new"
implementation:
.... def __init__(self, x, y):
.... self.x = x
.... self.y = y
.... def r2(self):
.... return self.x*self.x + self.y*self.y
........ def __init__(self, x, y, z=0):
.... self.x = x
.... self.y = y
.... self.z = z
.... def r2(self):
.... return self.x*self.x + self.y*self.y + self.z*self.z
....Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 7, in r2
AttributeError: 'P' object has no attribute 'z'

By default pickle doesn't invoke __init__() and updates __dict__ directly.
As pointed out in my previous post one way to fix the problem is to
implement a __setstate__() method:
.... def __init__(self, x, y, z=0):
.... self.x = x
.... self.y = y
.... self.z = z
.... def r2(self):
.... return self.x*self.x + self.y*self.y + self.z*self.z
.... def __setstate__(self, state):
.... self.__dict__["z"] = 42 # stupid default
.... self.__dict__.update(state)
....1777

This keeps working with pickles of the new implementation of P:
50

 

[Neal Becker]

Why don't you try it?

py> import pickle
py> class C:
... a = 23
...
py> c = C()
py> pickled = pickle.dumps(c)
py> C.b = 42 # add a new class attribute
py> d = pickle.loads(pickled)
py> d.a
23
py> d.b
42


Unless you mean something different from this, adding attributes to the
class is perfectly fine.

But... why are you dynamically adding attributes to the class? Isn't that
rather unusual?

The way I understand the problem is that an apparently backwards-compatible
change like adding a third dimension to a point with an obvious default
breaks when you restore an "old" instance in a script with the "new"
implementation:
... def __init__(self, x, y):
... self.x = x
... self.y = y
... def r2(self):
... return self.x*self.x + self.y*self.y
...... def __init__(self, x, y, z=0):
... self.x = x
... self.y = y
... self.z = z
... def r2(self):
... return self.x*self.x + self.y*self.y + self.z*self.z
...Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 7, in r2
AttributeError: 'P' object has no attribute 'z'

By default pickle doesn't invoke __init__() and updates __dict__ directly.
As pointed out in my previous post one way to fix the problem is to
implement a __setstate__() method:
... def __init__(self, x, y, z=0):
... self.x = x
... self.y = y
... self.z = z
... def r2(self):
... return self.x*self.x + self.y*self.y + self.z*self.z
... def __setstate__(self, state):
... self.__dict__["z"] = 42 # stupid default
... self.__dict__.update(state)
...1777

This keeps working with pickles of the new implementation of P:
50

So if in my new class definition there are now some new attributes, and if I did
not add a __setstate__ to set the new attributes, I guess then when unpickled
the instance of the class will simply lack those attributes?

 

[Peter Otten]

On Tue, 06 Mar 2012 07:34:34 -0500, Neal Becker wrote:

What happens if I pickle a class, and later unpickle it where the class
now has added some new attributes?

Why don't you try it?

py> import pickle
py> class C:
... a = 23
...
py> c = C()
py> pickled = pickle.dumps(c)
py> C.b = 42 # add a new class attribute
py> d = pickle.loads(pickled)
py> d.a
23
py> d.b
42


Unless you mean something different from this, adding attributes to the
class is perfectly fine.

But... why are you dynamically adding attributes to the class? Isn't
that rather unusual?

The way I understand the problem is that an apparently
backwards-compatible change like adding a third dimension to a point with
an obvious default breaks when you restore an "old" instance in a script
with the "new" implementation:

import pickle
class P(object):

... def __init__(self, x, y):
... self.x = x
... self.y = y
... def r2(self):
... return self.x*self.x + self.y*self.y
...

p = P(2, 3)
p.r2() 13
s = pickle.dumps(p)
class P(object):

... def __init__(self, x, y, z=0):
... self.x = x
... self.y = y
... self.z = z
... def r2(self):
... return self.x*self.x + self.y*self.y + self.z*self.z
...

p = P(2, 3)
p.r2() 13
pickle.loads(s).r2()

Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 7, in r2
AttributeError: 'P' object has no attribute 'z'

By default pickle doesn't invoke __init__() and updates __dict__
directly. As pointed out in my previous post one way to fix the problem
is to implement a __setstate__() method:

class P(object):

... def __init__(self, x, y, z=0):
... self.x = x
... self.y = y
... self.z = z
... def r2(self):
... return self.x*self.x + self.y*self.y + self.z*self.z
... def __setstate__(self, state):
... self.__dict__["z"] = 42 # stupid default
... self.__dict__.update(state)
...

pickle.loads(s).r2()

1777

This keeps working with pickles of the new implementation of P:

q = P(3, 4, 5)
pickle.loads(pickle.dumps(q)).r2()

50

So if in my new class definition there are now some new attributes, and if
I did not add a __setstate__ to set the new attributes, I guess then when
unpickled the instance of the class will simply lack those attributes?

I don't know. If you don't trust the demo try it yourself with the actual
code you have. Throwing in

obj = pickle.load(...)
print vars(obj)

should help.