Unpickle error -- "object has no attribute ...."

[luvspython]

I have an application that needs to keep a history of the values of
several attributes of each of many instances of many classes. The
history-keeping logic is in a helper class, HistoryKeeper, that's
inherited by classes like Vehicle in the example below.

Pickling an instance of Vehicle works, but unpickling fails with:
"Vehicle object has no attribute
'_orderedArgNames'" (_orderedArgNames is an attribute in
HistoryKeeper that tells the attributes for which history must be
kept.)

During unpickling, the exception occurs at the 2nd line in the
__getattribute__ method:
if item not in object.__getattribute__(self,
'_orderedArgNames'):

FWIW, cPickle fails the same way.

Below is a stripped-down example that fails in unpickling.

Can anyone explain why it fails and what I can do to fix it?

MANY thanks.

=========================================================

import datetime, bisect
from collections import OrderedDict



# define a class which helps keep date-history of attribute settings
in inheriting classes
class HistoryKeeper(object):
""" Class to maintain a dated history of attribute settings in
inheriting classes.
The initialization arguments are in an OrderedDict."""
def __init__(self, orderedArgs):
super(HistoryKeeper, self).__setattr__('_orderedArgNames',
orderedArgs.keys()) #remember the order of unnamed args
for arg, value in orderedArgs.items():
if arg != 'self':
self.Set(arg, value)


""" Get the current value of an attribute, optionally returning
its entire history."""
def __getattribute__(self, item, returnHistory=False):
value = object.__getattribute__(self, item)
if item not in object.__getattribute__(self,
'_orderedArgNames'):
return value # not an attribute for which we
maintain a change history
elif returnHistory:
return value # return the entire history
else:
return value[-1][0] # return only the latest
value


""" Set an attribute by appending the new value and date to
existing history of that attribute.
Unless a setting-date is supplied, default to today.
Set the value only if it's different than the chronological
immediately preceding value."""
def __setattr__(self, item, value, date=None):

# avoid history keeping if this item isn't among those
declared to require it
if item not in self._orderedArgNames:
super(HistoryKeeper, self).__setattr__(item, value)
else:
if not date:
date = datetime.date.today()
# if this attribute has already been set, add this value
to that history
try:
history = self.__getattribute__(item,
returnHistory=True)
# if a date was supplied, ensure the history remains
in chronological order
dates = [val[1] for val in history]
index = bisect.bisect_right(dates, date)
# insert this value into the history unless it doesn't
change an existing setting
if index == 0 or history[index-1][0] != value:
history.insert(index, (value,date))
except:
history = [(value, date)]
super(HistoryKeeper, self).__setattr__(item, history)

def Set(self, item, value):
self.__setattr__(item, value)




class Vehicle(HistoryKeeper):
def __init__(self, tag, make, model):
argDict = OrderedDict([('tag',tag),('make',make),
('model',model)])
super(Vehicle, self).__init__(argDict)

if __name__ == "__main__":
car = Vehicle('TAG123', 'FORD', 'Model A')

import pickle
pFile = open('car.pk1', 'wb')
pickle.dump(car, pFile, -1)
pFile.close()
print "car pickled OK"

pFile = open('car.pk1', 'rb')
community = pickle.load(pFile)
pFile.close()

 

[Peter Otten]

I have an application that needs to keep a history of the values of
several attributes of each of many instances of many classes. The
history-keeping logic is in a helper class, HistoryKeeper, that's
inherited by classes like Vehicle in the example below.

Pickling an instance of Vehicle works, but unpickling fails with:
"Vehicle object has no attribute
'_orderedArgNames'" (_orderedArgNames is an attribute in
HistoryKeeper that tells the attributes for which history must be
kept.)

During unpickling, the exception occurs at the 2nd line in the
__getattribute__ method:
if item not in object.__getattribute__(self,
'_orderedArgNames'):

FWIW, cPickle fails the same way.

Below is a stripped-down example that fails in unpickling.

Can anyone explain why it fails and what I can do to fix it?

By default unpickling an object does *not* invoke its __init__() method;
instead it creates an instance and then updates the __dict__ attribute of
that instance. You intercept attribute access with __getattribute__, so to
get hold of __dict__ you need to know __dict__["_orderedArgNames"] first, i.
e. you run into a bootstrap problem.

To fix the error I'd try special-casing "__dict__"

def __getattribute__(self, item, ...):
if item == "__dict__":
return super(HistoryKeeper, self).__getattribute__(item)
...

or making _orderedArgNames a class attribute:

class HistoryKeeper(object):
def __init__(self, orderedArgs):
for arg, value in orderedArgs.items():
if arg != 'self':
self.Set(arg, value)
...

class Vehicle(HistoryKeeper):
_orderedArgNames = "tag", "make", "model"
...

If that doesn't work out you can write your own __reduce_ex__() method to
customise pickling, see
http://docs.python.org/library/pickle.html#object.__reduce_ex__

By the way, docstrings belong below the def not above:

def f():
"""Explain f() here"""

 

[luvspython]

I have an application that needs to keep a history of the values of
several attributes of each of many instances of many classes.  The
history-keeping logic is in a helper class, HistoryKeeper, that's
inherited by classes like Vehicle in the example below.

Pickling an instance of Vehicle works, but unpickling fails with:
 "Vehicle object has no attribute
'_orderedArgNames'"   (_orderedArgNames is an attribute in
HistoryKeeper that tells the attributes for which history must be
kept.)

During unpickling, the exception occurs at the 2nd line in the
__getattribute__ method:
        if item not in object.__getattribute__(self,
'_orderedArgNames'):

FWIW, cPickle fails the same way.

Below is a stripped-down example that fails in unpickling.

Can anyone explain why it fails and what I can do to fix it?

By default unpickling an object does *not* invoke its __init__() method;
instead it creates an instance and then updates the __dict__ attribute of
that instance. You intercept attribute access with __getattribute__, so to
get hold of __dict__ you need to know __dict__["_orderedArgNames"] first,i.
e. you run into a bootstrap problem.

To fix the error I'd try special-casing "__dict__"

def __getattribute__(self, item, ...):
    if item == "__dict__":
        return super(HistoryKeeper, self).__getattribute__(item)
    ...

or making _orderedArgNames a class attribute:

class HistoryKeeper(object):
    def __init__(self, orderedArgs):
        for arg, value in orderedArgs.items():
            if arg != 'self':
                self.Set(arg, value)
    ...

class Vehicle(HistoryKeeper):
    _orderedArgNames = "tag", "make", "model"
    ...

If that doesn't work out you can write your own __reduce_ex__() method to
customise pickling, seehttp://docs.python.org/library/pickle.html#object.__reduce_ex__

By the way, docstrings belong below the def not above:

def f():
    """Explain f() here"""- Hide quoted text -

- Show quoted text -

THANK YOU! Special-casing "__dict__" did the trick. Not sure I'd
have ever figured that out, which leads to a different question:

I can figure out most things, though perhaps very slowly and
painfully, if I can trace through code. I use WingIDE (love it), but
the execution
of the C code is of course hidden, which helped stymie on this
problem. Is there another tool y'all might use and you can suggest
that deals with that problem and would have helped me with this case?
Or is one's ability to figure out this sort of problem largely
dependent on really understanding the system's internals?

 

[Chris Angelico]

I can figure out most things, though perhaps very slowly and
painfully, if I can trace through code.  I use WingIDE (love it), but
the execution
of the C code is of course hidden, which helped stymie on this
problem.  Is there another tool y'all might use and you can suggest
that deals with that problem and would have helped me with this case?
Or is one's ability to figure out this sort of problem largely
dependent on really understanding the system's internals?

In terms of debugging, it's hard to ignore the old favorite stand-by:
If In Doubt, Print It Out. Pepper your code with console-output calls
and manually trace your code using those. Every other debugging tool
you'll ever use is a bonus on top of that; if you accustom yourself to
IIDPIO debugging, you'll be able to solve problems in any environment.

Python does have a number of other debugging tools, though. I'd
recommend looking at pylint, for a start. It's not technically a
debugger, but it may be of value.

ChrisA

 

[Peter Otten]

THANK YOU! Special-casing "__dict__" did the trick. Not sure I'd
have ever figured that out, which leads to a different question:

I can figure out most things, though perhaps very slowly and
painfully, if I can trace through code. I use WingIDE (love it), but
the execution
of the C code is of course hidden, which helped stymie on this
problem. Is there another tool y'all might use and you can suggest
that deals with that problem and would have helped me with this case?
Or is one's ability to figure out this sort of problem largely
dependent on really understanding the system's internals?

I'm with Chris here, I make do with some understanding of Python and a
generous amount of print statements. My other secret weapon is that I try to
keep the complexity of my code low

As Python is yet another C program you can of course step through its code
(debug build) with an appropriate debugger, but I've never done that. My
guess is that you wouldn't have seen the forest for the trees.