pickle and infinity

[Bart Ogryczak]

Hello,
I´ve got this problem with pickle, it seems it doesn´t handle
correctly infinite values (nor does Python return overflow/underflow
error). What could I do about it? Example code:
[...]
ValueError: invalid literal for float(): 1.#INF

 

[Bart Ogryczak]

To make things more interesting -- Solaris version:
[...]
SystemError: frexp() result out of range

 

[Grant Edwards]

I´ve got this problem with pickle, it seems it doesn´t handle
correctly infinite values (nor does Python return
overflow/underflow error). What could I do about it?

Here's what I did. I'm sure it'll fall down on some systems,
but it works on Linux and Windows.

______________________________________________________________________
# unpickle can't handle nan/inf/ind floats, so we need to
# handle that ourselves

def myload_float(self):
s = self.readline()[:-1]
try:
f = float(s)
except ValueError:
s = s.upper()
if s in ["1.#INF", "INF"]:
f = 1e300*1e300
elif s in ["-1.#INF", "-INF"]:
f = -1e300*1e300
elif s in ["NAN","1.#QNAN","-1.#QNAN","QNAN","1.#IND","IND","-1.#IND"]:
f = (1e300*1e300)/(1e300*1e300)
else:
raise ValueError, "Don't know what to do with "+`s`
self.append(f)

# unpickle routine that overrides the float load method

def unpickle(f):
unpickler = pickle.Unpickler(f)
unpickler.dispatch[pickle.FLOAT] = myload_float
return unpickler.load()
______________________________________________________________________

 

[Fredrik Lundh]

I´ve got this problem with pickle, it seems it doesn´t handle
correctly infinite values (nor does Python return overflow/underflow
error).

Python 2.X relies on the C library to serialize floats, and, as you've
noticed, some C libraries can produce values that they themselves cannot
read.

</F>

 

[Bart Ogryczak]

Python 2.X relies on the C library to serialize floats, and, as you've
noticed, some C libraries can produce values that they themselves cannot
read.

Actually I´d be very happy, if Python would throw FloatPointExeption,
rather then allow infinite values. Haven´t got idea how to do it.
fpectl would do the trick, but it´s not present nither on Solaris, nor
on Windows platform. And I´d need it on both.

 

[Grant Edwards]

Actually I´d be very happy, if Python would throw FloatPointExeption,
rather then allow infinite values.

I'd be very unhappy. I use both NaNs and infinities in many
programs, so it needs to be selectable should that feature be
added.

Haven´t got idea how to do it. fpectl would do the trick, but
it´s not present nither on Solaris, nor on Windows platform.
And I´d need it on both.

In my experience anything than needs to be portable and
involves NaN, inf, or denormals is rather painful.

 

[Bart Ogryczak]

I'd be very unhappy. I use both NaNs and infinities in many
programs, so it needs to be selectable should that feature be
added.

Like I´ve said. I´d be satisfied with something like fpectl, which
gives you option to either throw FloatPointExeption or go on with NaNs
and infs. NaNs and infinities are pain in the a** if you´re
integrating with C libraries or you´re using pickle, pyro, whatever.
The problem is, that the error pops out in the middle of the code that
really has nothing to do with the actual erroneous calculation. Its
impossible to trace back where the hell the NaN or infinite value comes
from.

In my experience anything than needs to be portable and
involves NaN, inf, or denormals is rather painful.

That´s not very comforting :-/