A struct for 2.4 that supports float's inf and nan?

[Joshua J. Kugler]

I'm trying to put some values into a struct. Some of these values are NaN
and Inf due to the nature of the data. As you well may know, struct (and
other things) in Python <= 2.4 doesn't support inf and nan float values.
You get the dreaded "SystemError: frexp() result out of range" error.

Before I go and write my own little wrapper, has anyone out there written
an "extended" struct that supports the inf and nan values? I know this is
fixed in 2.5, but using that isn't an option at this point, as users will
be running older versions of python.

I suppose I could get the relevant source from the 2.5 source and compile it
as a custom package, but that wouldn't be very transparent for my users,
and would probably be getting in way over my head.

Ideas? Suggestions?

j

 

[Paul Hankin]

I'm trying to put some values into a struct. Some of these values are NaN
and Inf due to the nature of the data. As you well may know, struct (and
other things) in Python <= 2.4 doesn't support inf and nan float values.
You get the dreaded "SystemError: frexp() result out of range" error.

Before I go and write my own little wrapper, has anyone out there written
an "extended" struct that supports the inf and nan values? I know this is
fixed in 2.5, but using that isn't an option at this point, as users will
be running older versions of python.

I suppose I could get the relevant source from the 2.5 source and compile it
as a custom package, but that wouldn't be very transparent for my users,
and would probably be getting in way over my head.

Ideas? Suggestions?

Here's a wrapped pack:

import struct

pack_double_workaround = {
'inf': struct.pack('Q', 0x7ff0000000000000L),
'-inf': struct.pack('Q', 0xfff0000000000000L),
'nan': struct.pack('Q', 0x7ff8000000000000L)
}

def pack_one_safe(f, a):
if f == 'd' and str(f) in pack_double_workaround:
return pack_double_workaround[str(f)]
return struct.pack(f, a)

def pack(fmt, *args):
return ''.join(pack_one_safe(f, a) for f, a in zip(fmt, args))

Unpacking is similar: unpack doubles with 'Q' and test the
long for equality with +-inf, and find nan's by checking bits
52 to 62. If the number's ok, unpack again using 'd'.

You can get python values for nan, -inf and inf by using
float('nan'), float('-inf'), float('inf').

I've not been able to properly test this, as struct seems
to work fine in Python 2.3 and 2.4 on MacOS X.

HTH

 

[Paul Hankin]

Both str(f) should be str(a) in pack_one_safe.

 

[Joshua J. Kugler]

I suppose I could get the relevant source from the 2.5 source and compile
it as a custom package, but that wouldn't be very transparent for my
users, and would probably be getting in way over my head.

Ideas? Suggestions?

Here's a wrapped pack:

import struct

pack_double_workaround = {
'inf': struct.pack('Q', 0x7ff0000000000000L),
'-inf': struct.pack('Q', 0xfff0000000000000L),
'nan': struct.pack('Q', 0x7ff8000000000000L)
}

def pack_one_safe(f, a):
if f == 'd' and str(f) in pack_double_workaround:
return pack_double_workaround[str(f)]
return struct.pack(f, a)

def pack(fmt, *args):
return ''.join(pack_one_safe(f, a) for f, a in zip(fmt, args))

Unpacking is similar: unpack doubles with 'Q' and test the
long for equality with +-inf, and find nan's by checking bits
52 to 62. If the number's ok, unpack again using 'd'.

You can get python values for nan, -inf and inf by using
float('nan'), float('-inf'), float('inf').

I've not been able to properly test this, as struct seems
to work fine in Python 2.3 and 2.4 on MacOS X.

Thanks for the ideas, Paul! I came up with something that works for me, but
this has a few ideas that I'm going to implement in my wrapper to make for
cleaner code.

As to testing it on MacOS X: yeah, it can be a somewhat system-dependent
problem, so may not show up on all architectures.

Thanks for the tips!

j

 

[Grant Edwards]

import struct

pack_double_workaround = {
'inf': struct.pack('Q', 0x7ff0000000000000L),
'-inf': struct.pack('Q', 0xfff0000000000000L),
'nan': struct.pack('Q', 0x7ff8000000000000L)
}

def pack_one_safe(f, a):
if f == 'd' and str(f) in pack_double_workaround:
return pack_double_workaround[str(f)]
return struct.pack(f, a)

def pack(fmt, *args):
return ''.join(pack_one_safe(f, a) for f, a in zip(fmt, args))

NB: the strings returned by str() when passed a NaN or Inf are
system dependent and aren't guaranteed to be consistent from
one day to the next unless you've overridden the floating point
object's __repr__ method to make sure.

Here are the tests I use for 32-bit work:

def isNaN(u):
return ((u & 0x7f800000) == 0x7f800000) and (u & 0x7fffff)
def isInf(u):
return ((u & 0x7f800000) == 0x7f800000) and ((u & 0x7fffff)==0)
def isNeg(u):
return (u & 0x80000000)

Thanks for the ideas, Paul! I came up with something that
works for me, but this has a few ideas that I'm going to
implement in my wrapper to make for cleaner code.

As to testing it on MacOS X: yeah, it can be a somewhat
system-dependent problem,

It shouldn't be, but unfortunately it is. If you're careful,
you can come up with something that's fairly portable (I've got
a wrapped pickle/unpickle that's nan/inf aware and works on
both Win32 and Linux. Holler if you want it.