Extracting bit fields from an IEEE-784 float

Discussion in 'Python' started by Steven D'Aprano, Jul 30, 2012.

  1. I wish to extract the bit fields from a Python float, call it x. First I
    cast the float to 8-bytes:

    s = struct.pack('=d', x)
    i = struct.unpack('=q', s)[0]

    Then I extract the bit fields from the int, e.g. to grab the sign bit:

    (i & 0x8000000000000000) >> 63


    Questions:

    1) Are there any known implementations or platforms where Python floats
    are not C doubles? If so, what are they?

    2) If the platform byte-order is reversed, do I need to take any special
    action? I don't think I do, because even though the float is reversed, so
    will be the bit mask. Is this correct?

    3) Any other problems with the way I am doing this?



    Thanks in advance,



    Steven
     
    Steven D'Aprano, Jul 30, 2012
    #1
    1. Advertising

  2. Steven D'Aprano

    Dan Sommers Guest

    On 2012-07-30 at 00:44:04 +0000,
    Steven D'Aprano <> wrote:

    > I wish to extract the bit fields from a Python float, call it x. First I
    > cast the float to 8-bytes:
    >
    > s = struct.pack('=d', x)
    > i = struct.unpack('=q', s)[0]
    >
    > Then I extract the bit fields from the int, e.g. to grab the sign bit:
    >
    > (i & 0x8000000000000000) >> 63


    > 3) Any other problems with the way I am doing this?


    No, but perhaps this would be clearer:

    import math
    sign = math.copysign(1.0, x)

    There are solutions that use math.frexp, too, but IMO they're more
    obtuse.

    HTH,
    Dan
     
    Dan Sommers, Jul 30, 2012
    #2
    1. Advertising

  3. PyPI question, or, maybe I'm just stupid

    I've been making some minor updates to the PollyReports module I
    announced a while back, and I've noticed that when I upload it to PyPI,
    my changelog (CHANGES.txt) doesn't appear to be integrated into the site
    at all. Do I have to put the changes into the README, or have I missed
    something here? It seems that there should be some automatic method
    whereby PyPI users could easily see what I've changed without
    downloading it first.
     
    Chris Gonnerman, Jul 30, 2012
    #3
  4. Steven D'Aprano

    Terry Reedy Guest

    On 7/29/2012 8:44 PM, Steven D'Aprano wrote:
    > I wish to extract the bit fields from a Python float, call it x. First I
    > cast the float to 8-bytes:
    >
    > s = struct.pack('=d', x)
    > i = struct.unpack('=q', s)[0]
    >
    > Then I extract the bit fields from the int, e.g. to grab the sign bit:
    >
    > (i & 0x8000000000000000) >> 63
    >
    >
    > Questions:
    >
    > 1) Are there any known implementations or platforms where Python floats
    > are not C doubles? If so, what are they?


    CPython floats are C doubles, which should be IEEE doubles. Other
    implementations have a different to probably the same thing.

    > 2) If the platform byte-order is reversed, do I need to take any special
    > action? I don't think I do, because even though the float is reversed, so
    > will be the bit mask. Is this correct?


    The math modules functions to disassemble floats will not care.

    --
    Terry Jan Reedy
     
    Terry Reedy, Jul 30, 2012
    #4
  5. Am 30.07.2012 02:44, schrieb Steven D'Aprano:
    > I wish to extract the bit fields from a Python float, call it x. First I
    > cast the float to 8-bytes:
    >
    > s = struct.pack('=d', x)
    > i = struct.unpack('=q', s)[0]
    >
    > Then I extract the bit fields from the int, e.g. to grab the sign bit:
    >
    > (i & 0x8000000000000000) >> 63
    >
    >
    > Questions:
    >
    > 1) Are there any known implementations or platforms where Python floats
    > are not C doubles? If so, what are they?


    The struct docs refer to C's double type, so it depends on that type
    probably. However, regardless of C's double type, the same docs refer to
    the IEEE form when packed into a byte array. Is it just the
    representation you are after or some specific behaviour?


    > 2) If the platform byte-order is reversed, do I need to take any special
    > action? I don't think I do, because even though the float is reversed, so
    > will be the bit mask. Is this correct?


    Yes, the code is fine. If you have doubts, I have a big-endian system at
    home (Linux/PowerPC) where I could run tests.


    > 3) Any other problems with the way I am doing this?


    Python docs refer to IEEE-754, not 784? Typo?


    Uli
     
    Ulrich Eckhardt, Jul 30, 2012
    #5
  6. On Monday, July 30, 2012 1:44:04 AM UTC+1, Steven D'Aprano wrote:
    > 1) Are there any known implementations or platforms where Python floats
    > are not C doubles? If so, what are they?


    Well, IronPython is presumably using .NET Doubles, while Jython will be using Java Doubles---in either case, that's specified to be the IEEE 754 binary64 type.

    For CPython, and I guess PyPy too, we're using C doubles, which in theory are in whatever format the platform provides, but in practice are always IEEE 754 binary64 again.

    So you're pretty safe assuming IEEE 754 binary64 format. If you ever meet a current Python running on a system that *doesn't* use IEEE 754 for its C doubles, please let me know---there are a lot of interesting questions thatwould come up in that case.

    > 2) If the platform byte-order is reversed, do I need to take any special
    >
    > action? I don't think I do, because even though the float is reversed, so
    >
    > will be the bit mask. Is this correct?


    True; on almost all current platforms, the endianness of int types matches the endianness of float types. But to be safe, why not use '<d' and '<q'in your formats instead of '=d' and '=q'? That way you don't have to worry.

    > 3) Any other problems with the way I am doing this?


    You might consider whether you want to use '<q' or '<Q' --- i.e. whether you want a signed integer or an unsigned integer to be returned. For grabbing bits, '<Q' seems a bit cleaner, while '<q' has the nice property that youcan tell the sign of the original double by looking at the sign of the integer.

    --
    Mark
     
    Mark Dickinson, Jul 30, 2012
    #6
  7. Re: PyPI question, or, maybe I'm just stupid

    Chris Gonnerman <> writes:

    > I've been making some minor updates to the PollyReports module I
    > announced a while back, and I've noticed that when I upload it to
    > PyPI, my changelog (CHANGES.txt) doesn't appear to be integrated into
    > the site at all. Do I have to put the changes into the README, or
    > have I missed something here? It seems that there should be some
    > automatic method whereby PyPI users could easily see what I've changed
    > without downloading it first.


    "CHANGES.txt" is not automatically presented.
    If necessary, you must integrate it into the "long description".

    However, personally, I am not interested in all the details (typically
    found in "CHANGES.txt") but some (often implicit) information is
    sufficient for me: something like "major API change", "minor bug
    fixes". Thus, think carefully what you put on the overview page.

    I find it very stupid to see several window scrolls of changes for
    a package but to learn how to install the package, I have to download its
    source...
     
    Dieter Maurer, Jul 30, 2012
    #7
  8. Re: [Python] Re: PyPI question, or, maybe I'm just stupid

    On 07/29/2012 11:00 PM, Ben Finney wrote:
    > Your post is showing up as a reply to a thread about IEEE-784 floats,
    > because you created your message as a reply. Consequently, it's rather
    > confusing why you suddenly start talking about PollyReports. If you
    > want to attract attention to an unrelated topic, it's best if you
    > don't reply to an existing thread; instead, start a new thread by
    > composing a new message to the forum.

    My apologies. I did not consider that headers I can't see might be
    being sent along.
     
    Chris Gonnerman, Jul 30, 2012
    #8
  9. On 2012-07-30, Steven D'Aprano <> wrote:

    > 1) Are there any known implementations or platforms where Python floats
    > are not C doubles? If so, what are they?


    And the question you didn't ask: are there any platforms where a C
    double isn't IEEE-754?

    The last ones I worked on that where the FP format wasn't IEEE were
    the DEC VAX and TI's line if 32-bit floating-point DSPs. I don't
    think Python runs on the latter, but it might on the former.

    --
    Grant Edwards grant.b.edwards Yow! I was born in a
    at Hostess Cupcake factory
    gmail.com before the sexual
    revolution!
     
    Grant Edwards, Jul 30, 2012
    #9
  10. Steven D'Aprano

    Roy Smith Guest

    In article <jv64v5$g2n$>,
    Grant Edwards <> wrote:

    > The last ones I worked on that where the FP format wasn't IEEE were
    > the DEC VAX


    According to http://en.wikipedia.org/wiki/Vax#History, the last VAX was
    produced 7 years ago. I'm sure there's still more than a few chugging
    away in corporate data centers and manufacturing floors, but as an
    architecture, it's pretty much a dead parrot.

    IEEE floating point is as near to a universal standard as it gets in the
    computer world. About the only thing that has it beat for market
    penetration and longevity are 2's complement integers and 8-bit bytes.
     
    Roy Smith, Jul 30, 2012
    #10
  11. On 30/07/2012 15:16, Grant Edwards wrote:
    > On 2012-07-30, Steven D'Aprano <> wrote:
    >
    >> 1) Are there any known implementations or platforms where Python floats
    >> are not C doubles? If so, what are they?

    >
    > And the question you didn't ask: are there any platforms where a C
    > double isn't IEEE-754?
    >
    > The last ones I worked on that where the FP format wasn't IEEE were
    > the DEC VAX and TI's line if 32-bit floating-point DSPs. I don't
    > think Python runs on the latter, but it might on the former.
    >


    Support for Python on VMS has been dropped for v3.3 see
    http://bugs.python.org/issue11918

    --
    Cheers.

    Mark Lawrence.
     
    Mark Lawrence, Jul 30, 2012
    #11
  12. On 2012-07-30, Mark Lawrence <> wrote:
    > On 30/07/2012 15:16, Grant Edwards wrote:
    >> On 2012-07-30, Steven D'Aprano <> wrote:
    >>
    >>> 1) Are there any known implementations or platforms where Python floats
    >>> are not C doubles? If so, what are they?

    >>
    >> And the question you didn't ask: are there any platforms where a C
    >> double isn't IEEE-754?
    >>
    >> The last ones I worked on that where the FP format wasn't IEEE were
    >> the DEC VAX and TI's line if 32-bit floating-point DSPs. I don't
    >> think Python runs on the latter, but it might on the former.
    >>

    >
    > Support for Python on VMS has been dropped for v3.3 see
    > http://bugs.python.org/issue11918


    I imagine that VAXes running Unix went extinct in the wild long before
    VAXes running VMS.

    --
    Grant Edwards grant.b.edwards Yow! Did YOU find a
    at DIGITAL WATCH in YOUR box
    gmail.com of VELVEETA?
     
    Grant Edwards, Jul 30, 2012
    #12
  13. On Monday, July 30, 2012 3:16:05 PM UTC+1, Grant Edwards wrote:
    > The last ones I worked on that where the FP format wasn't IEEE were
    >
    > the DEC VAX and TI's line if 32-bit floating-point DSPs. I don't
    >
    > think Python runs on the latter, but it might on the former.


    For current hardware, there's also IBM big iron: the IBM System z10 apparently has hardware support for IBM's hexadecimal floating-point format in addition to IEEE 754 binary *and* decimal floating-point. But IIUC, a typical Linux installation on one of these machines uses the IEEE 754 stuff, not the hexadecimal bits. So unlikely to be an issue for Python.

    --
    Mark
     
    Mark Dickinson, Jul 30, 2012
    #13
  14. Steven D'Aprano

    Roy Smith Guest

    In article <jv6ab7$jne$>,
    Grant Edwards <> wrote:

    > I imagine that VAXes running Unix went extinct in the wild long before
    > VAXes running VMS.


    Of course they did. VMS is all about vendor lock-in. People who
    continue to run VAXen don't do so because they're wedded to the
    hardware. They do so because they're wedded to some specific VMS
    application (and the business processes which depend on it).
     
    Roy Smith, Jul 30, 2012
    #14
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Ernst Murnleitner

    float: IEEE, big endian, little endian

    Ernst Murnleitner, Jan 13, 2004, in forum: C++
    Replies:
    0
    Views:
    868
    Ernst Murnleitner
    Jan 13, 2004
  2. bd
    Replies:
    0
    Views:
    632
  3. Andy

    32-bit IEEE float multiplication

    Andy, Dec 3, 2003, in forum: C Programming
    Replies:
    54
    Views:
    1,757
  4. franzkowiak
    Replies:
    9
    Views:
    951
    Tim Peters
    Jan 16, 2005
  5. Carsten Fuchs
    Replies:
    45
    Views:
    1,553
    James Kanze
    Oct 8, 2009
Loading...

Share This Page