Extracting bit fields from an IEEE-784 float

S

Steven D'Aprano

I wish to extract the bit fields from a Python float, call it x. First I
cast the float to 8-bytes:

s = struct.pack('=d', x)
i = struct.unpack('=q', s)[0]

Then I extract the bit fields from the int, e.g. to grab the sign bit:

(i & 0x8000000000000000) >> 63


Questions:

1) Are there any known implementations or platforms where Python floats
are not C doubles? If so, what are they?

2) If the platform byte-order is reversed, do I need to take any special
action? I don't think I do, because even though the float is reversed, so
will be the bit mask. Is this correct?

3) Any other problems with the way I am doing this?



Thanks in advance,



Steven
 
D

Dan Sommers

I wish to extract the bit fields from a Python float, call it x. First I
cast the float to 8-bytes:

s = struct.pack('=d', x)
i = struct.unpack('=q', s)[0]

Then I extract the bit fields from the int, e.g. to grab the sign bit:

(i & 0x8000000000000000) >> 63
3) Any other problems with the way I am doing this?

No, but perhaps this would be clearer:

import math
sign = math.copysign(1.0, x)

There are solutions that use math.frexp, too, but IMO they're more
obtuse.

HTH,
Dan
 
C

Chris Gonnerman

I've been making some minor updates to the PollyReports module I
announced a while back, and I've noticed that when I upload it to PyPI,
my changelog (CHANGES.txt) doesn't appear to be integrated into the site
at all. Do I have to put the changes into the README, or have I missed
something here? It seems that there should be some automatic method
whereby PyPI users could easily see what I've changed without
downloading it first.
 
T

Terry Reedy

I wish to extract the bit fields from a Python float, call it x. First I
cast the float to 8-bytes:

s = struct.pack('=d', x)
i = struct.unpack('=q', s)[0]

Then I extract the bit fields from the int, e.g. to grab the sign bit:

(i & 0x8000000000000000) >> 63


Questions:

1) Are there any known implementations or platforms where Python floats
are not C doubles? If so, what are they?

CPython floats are C doubles, which should be IEEE doubles. Other
implementations have a different to probably the same thing.
2) If the platform byte-order is reversed, do I need to take any special
action? I don't think I do, because even though the float is reversed, so
will be the bit mask. Is this correct?

The math modules functions to disassemble floats will not care.
 
U

Ulrich Eckhardt

Am 30.07.2012 02:44, schrieb Steven D'Aprano:
I wish to extract the bit fields from a Python float, call it x. First I
cast the float to 8-bytes:

s = struct.pack('=d', x)
i = struct.unpack('=q', s)[0]

Then I extract the bit fields from the int, e.g. to grab the sign bit:

(i & 0x8000000000000000) >> 63


Questions:

1) Are there any known implementations or platforms where Python floats
are not C doubles? If so, what are they?

The struct docs refer to C's double type, so it depends on that type
probably. However, regardless of C's double type, the same docs refer to
the IEEE form when packed into a byte array. Is it just the
representation you are after or some specific behaviour?

2) If the platform byte-order is reversed, do I need to take any special
action? I don't think I do, because even though the float is reversed, so
will be the bit mask. Is this correct?

Yes, the code is fine. If you have doubts, I have a big-endian system at
home (Linux/PowerPC) where I could run tests.

3) Any other problems with the way I am doing this?

Python docs refer to IEEE-754, not 784? Typo?


Uli
 
M

Mark Dickinson

1) Are there any known implementations or platforms where Python floats
are not C doubles? If so, what are they?

Well, IronPython is presumably using .NET Doubles, while Jython will be using Java Doubles---in either case, that's specified to be the IEEE 754 binary64 type.

For CPython, and I guess PyPy too, we're using C doubles, which in theory are in whatever format the platform provides, but in practice are always IEEE 754 binary64 again.

So you're pretty safe assuming IEEE 754 binary64 format. If you ever meet a current Python running on a system that *doesn't* use IEEE 754 for its C doubles, please let me know---there are a lot of interesting questions thatwould come up in that case.
2) If the platform byte-order is reversed, do I need to take any special

action? I don't think I do, because even though the float is reversed, so

will be the bit mask. Is this correct?

True; on almost all current platforms, the endianness of int types matches the endianness of float types. But to be safe, why not use '<d' and '<q'in your formats instead of '=d' and '=q'? That way you don't have to worry.
3) Any other problems with the way I am doing this?

You might consider whether you want to use '<q' or '<Q' --- i.e. whether you want a signed integer or an unsigned integer to be returned. For grabbing bits, '<Q' seems a bit cleaner, while '<q' has the nice property that youcan tell the sign of the original double by looking at the sign of the integer.
 
D

Dieter Maurer

Chris Gonnerman said:
I've been making some minor updates to the PollyReports module I
announced a while back, and I've noticed that when I upload it to
PyPI, my changelog (CHANGES.txt) doesn't appear to be integrated into
the site at all. Do I have to put the changes into the README, or
have I missed something here? It seems that there should be some
automatic method whereby PyPI users could easily see what I've changed
without downloading it first.

"CHANGES.txt" is not automatically presented.
If necessary, you must integrate it into the "long description".

However, personally, I am not interested in all the details (typically
found in "CHANGES.txt") but some (often implicit) information is
sufficient for me: something like "major API change", "minor bug
fixes". Thus, think carefully what you put on the overview page.

I find it very stupid to see several window scrolls of changes for
a package but to learn how to install the package, I have to download its
source...
 
C

Chris Gonnerman

Your post is showing up as a reply to a thread about IEEE-784 floats,
because you created your message as a reply. Consequently, it's rather
confusing why you suddenly start talking about PollyReports. If you
want to attract attention to an unrelated topic, it's best if you
don't reply to an existing thread; instead, start a new thread by
composing a new message to the forum.
My apologies. I did not consider that headers I can't see might be
being sent along.
 
G

Grant Edwards

1) Are there any known implementations or platforms where Python floats
are not C doubles? If so, what are they?

And the question you didn't ask: are there any platforms where a C
double isn't IEEE-754?

The last ones I worked on that where the FP format wasn't IEEE were
the DEC VAX and TI's line if 32-bit floating-point DSPs. I don't
think Python runs on the latter, but it might on the former.
 
R

Roy Smith

Grant Edwards said:
The last ones I worked on that where the FP format wasn't IEEE were
the DEC VAX

According to http://en.wikipedia.org/wiki/Vax#History, the last VAX was
produced 7 years ago. I'm sure there's still more than a few chugging
away in corporate data centers and manufacturing floors, but as an
architecture, it's pretty much a dead parrot.

IEEE floating point is as near to a universal standard as it gets in the
computer world. About the only thing that has it beat for market
penetration and longevity are 2's complement integers and 8-bit bytes.
 
M

Mark Lawrence

And the question you didn't ask: are there any platforms where a C
double isn't IEEE-754?

The last ones I worked on that where the FP format wasn't IEEE were
the DEC VAX and TI's line if 32-bit floating-point DSPs. I don't
think Python runs on the latter, but it might on the former.

Support for Python on VMS has been dropped for v3.3 see
http://bugs.python.org/issue11918
 
M

Mark Dickinson

The last ones I worked on that where the FP format wasn't IEEE were

the DEC VAX and TI's line if 32-bit floating-point DSPs. I don't

think Python runs on the latter, but it might on the former.

For current hardware, there's also IBM big iron: the IBM System z10 apparently has hardware support for IBM's hexadecimal floating-point format in addition to IEEE 754 binary *and* decimal floating-point. But IIUC, a typical Linux installation on one of these machines uses the IEEE 754 stuff, not the hexadecimal bits. So unlikely to be an issue for Python.
 
R

Roy Smith

Grant Edwards said:
I imagine that VAXes running Unix went extinct in the wild long before
VAXes running VMS.

Of course they did. VMS is all about vendor lock-in. People who
continue to run VAXen don't do so because they're wedded to the
hardware. They do so because they're wedded to some specific VMS
application (and the business processes which depend on it).
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

Forum statistics

Threads
473,731
Messages
2,569,432
Members
44,835
Latest member
KetoRushACVBuy

Latest Threads

Top