pickle problem - frexp() out of range

[ahaldar]

Hi:

I have some large data structure objects in memory, and when I attempt
to pickle them, I get the following error:

SystemError: frexp() out of range

Are there some objects that are just too large to serialize, and if
so, is there an easy workaround without breaking up the object and
reconstructing it during deserialization?

Here's the code I use to pickle the object:

f = open(dir+file, "w+b")
pickle.dump(structure, f, protocol=2) # throws error
f.close()

- abhra

 

[Irmen de Jong]

Hi:

I have some large data structure objects in memory, and when I attempt
to pickle them, I get the following error:

SystemError: frexp() out of range

Are there some objects that are just too large to serialize, and if
so, is there an easy workaround without breaking up the object and
reconstructing it during deserialization?

Here's the code I use to pickle the object:

f = open(dir+file, "w+b")
pickle.dump(structure, f, protocol=2) # throws error
f.close()

- abhra

Could it be that your data contains floating point numbers,
where at least one of them is Inf or NaN?
I think these floats cannot be pickled reliably.

--Irmen

 

[Ziga Seilnacht]

Hi:

I have some large data structure objects in memory, and when I attempt
to pickle them, I get the following error:

SystemError: frexp() out of range

Are there some objects that are just too large to serialize, and if
so, is there an easy workaround without breaking up the object and
reconstructing it during deserialization?

Here's the code I use to pickle the object:

f = open(dir+file, "w+b")
pickle.dump(structure, f, protocol=2) # throws error
f.close()

- abhra

You are probably trying to pickle Inf or NaN. This was fixed
in Python 2.5, see this revision:
http://svn.python.org/view?rev=38893&view=rev
and this patch:
http://www.python.org/sf/1181301

Ziga