J
Jean-Luc
Hi
I am grafing pickling functionality on a big dictionary of object
instances and I am having numerous problems. I think understand the
notion of what can/can't be pickled and I know how to use __getstate__
/ __setstate__ to weed out offending references. But I am finding it
very hard to identify _which_ reference is causing exceptions like:
File "C:\UserApps\Python23\Lib\copy_reg.py", line 69, in _reduce_ex
raise TypeError, "can't pickle %s objects" % base.__name__
TypeError: can't pickle function objects
PrettyPrinting the exception gets me:
{'args': ("can't pickle function objects",)}.
The exception includes no references to the guilty party! How do I
know _which_ object caused the issue? Though my current objects (i.e.
the stuff I am trying to pickle) are new, its instances hold
references to a lot of instances of existing utility classes of mine
that are database aware. I would not like to modify those utility
classes any more than I _have_ to since I use them elsewhere. I have
already stripped out database Connection references, as well as File
references. Those were obvious.
How do I identify the remaining issues? If I work with a smaller data
set, I often miss problems entirely, so reducing the pickling scope is
not an easy option to implement. The best I have managed so far is to
open the pickle file and look near the end, where the exception
presumably occurred. But I would expect there to be a more obvious
way to troubleshoot this stuff.
Cheers
JLuc
I am grafing pickling functionality on a big dictionary of object
instances and I am having numerous problems. I think understand the
notion of what can/can't be pickled and I know how to use __getstate__
/ __setstate__ to weed out offending references. But I am finding it
very hard to identify _which_ reference is causing exceptions like:
File "C:\UserApps\Python23\Lib\copy_reg.py", line 69, in _reduce_ex
raise TypeError, "can't pickle %s objects" % base.__name__
TypeError: can't pickle function objects
PrettyPrinting the exception gets me:
{'args': ("can't pickle function objects",)}.
The exception includes no references to the guilty party! How do I
know _which_ object caused the issue? Though my current objects (i.e.
the stuff I am trying to pickle) are new, its instances hold
references to a lot of instances of existing utility classes of mine
that are database aware. I would not like to modify those utility
classes any more than I _have_ to since I use them elsewhere. I have
already stripped out database Connection references, as well as File
references. Those were obvious.
How do I identify the remaining issues? If I work with a smaller data
set, I often miss problems entirely, so reducing the pickling scope is
not an easy option to implement. The best I have managed so far is to
open the pickle file and look near the end, where the exception
presumably occurred. But I would expect there to be a more obvious
way to troubleshoot this stuff.
Cheers
JLuc