interpret 4 byte as 32-bit float (IEEE-754)

Discussion in 'Python' started by franzkowiak, Jan 15, 2005.

  1. franzkowiak

    franzkowiak Guest

    Hello,

    I've read some bytes from a file and just now I can't interpret 4 bytes
    in this dates like a real value.

    An extract from my program

    def l32(c):
    return ord(c[0]) + (ord(c[1])<<8) + (ord(c[2])<<16) + (ord(c[3])<<24)

    ....

    value = l32(f.read(4)) <--- 3F 8C CC CD should be 1.11



    Anybody an answer ?



    regards
    gf
     
    franzkowiak, Jan 15, 2005
    #1
    1. Advertising

  2. franzkowiak wrote:
    > I've read some bytes from a file and just now I can't interpret 4 bytes
    > in this dates like a real value. An extract from my program:
    > def l32(c):
    > return ord(c[0]) + (ord(c[1])<<8) + (ord(c[2])<<16) + (ord(c[3])<<24)
    > ...
    > value = l32(f.read(4)) <--- 3F 8C CC CD should be 1.11
    >

    OK, here's the skinny (I used blocks & views to get the answer):

    import struct
    bytes = ''.join(chr(int(txt, 16)) for txt in '3F 8C CC CD'.split())
    struct.unpack('>f', bytes)

    I was suspicious of that first byte, thought it might be an exponent,
    since it seemed to have too many on bits in a row to be part of 1.11.

    -Scott David Daniels
     
    Scott David Daniels, Jan 15, 2005
    #2
    1. Advertising

  3. On 2005-01-15, Scott David Daniels <> wrote:

    >> I've read some bytes from a file and just now I can't interpret 4 bytes
    >> in this dates like a real value.


    > OK, here's the skinny (I used blocks & views to get the answer):
    >
    > import struct
    > bytes = ''.join(chr(int(txt, 16)) for txt in '3F 8C CC CD'.split())
    > struct.unpack('>f', bytes)


    Just be careful. That doesn't work for all 32-bit IEEE floating
    point values:

    >>> import struct
    >>> bytes = '\xff\xff\xff\xff'
    >>> print struct.unpack('>f',bytes)

    (-6.8056469327705772e+38,)

    0xffffff is _not_ -6.8...e38. It's a NaN.

    IIRC, it doesn't work for infinities either. I haven't tried
    denormals.

    --
    Grant Edwards grante Yow! It's hard being
    at an ARTIST!!
    visi.com
     
    Grant Edwards, Jan 15, 2005
    #3
  4. Scott David Daniels schrieb:
    > franzkowiak wrote:
    >
    >> I've read some bytes from a file and just now I can't interpret 4
    >> bytes in this dates like a real value. An extract from my program:
    >> def l32(c):
    >> return ord(c[0]) + (ord(c[1])<<8) + (ord(c[2])<<16) +
    >> (ord(c[3])<<24)
    >> ...
    >> value = l32(f.read(4)) <--- 3F 8C CC CD should be 1.11
    >>

    > OK, here's the skinny (I used blocks & views to get the answer):
    >
    > import struct
    > bytes = ''.join(chr(int(txt, 16)) for txt in '3F 8C CC CD'.split())
    > struct.unpack('>f', bytes)
    >
    > I was suspicious of that first byte, thought it might be an exponent,
    > since it seemed to have too many on bits in a row to be part of 1.11.
    >
    > -Scott David Daniels
    >


    Ok, I the string exist with "mystr = f.read(4)" and the solution for
    this case is in your line "struct.unpack('>f', bytes)"
    But what can I do when I want the interpret the content from the Integer
    myInt (*myInt = 0x3F8CCCCD) like 4-byte-real ?
    This was stored with an othes system in a binary file to
    CD CC 8C 3F and now is it in python in value. The conversion is not
    possible. It's right... one of this bytes is an exponent.
    I want copy the memory content from the "value address" to "myReal
    address" and use print "%f" %myReal.
    Is myReal then the right format ?
    What can I do with python, in FORTH is it simple
    ( >f f. )

    gf
     
    G.Franzkowiak, Jan 15, 2005
    #4
  5. G.Franzkowiak wrote:
    > Scott David Daniels schrieb:
    >
    >> franzkowiak wrote:
    >>
    >>> I've read some bytes from a file and just now I can't interpret 4
    >>> bytes in this dates like a real value. An extract from my program:
    >>> def l32(c):
    >>> return ord(c[0]) + (ord(c[1])<<8) + (ord(c[2])<<16) +
    >>> (ord(c[3])<<24)
    >>> ...
    >>> value = l32(f.read(4)) <--- 3F 8C CC CD should be 1.11
    >>>

    >> OK, here's the skinny (I used blocks & views to get the answer):
    >>
    >> import struct
    >> bytes = ''.join(chr(int(txt, 16)) for txt in '3F 8C CC CD'.split())
    >> struct.unpack('>f', bytes)
    >>
    >> I was suspicious of that first byte, thought it might be an exponent,
    >> since it seemed to have too many on bits in a row to be part of 1.11.
    >>
    >> -Scott David Daniels
    >>

    >
    >
    > Ok, I the string exist with "mystr = f.read(4)" and the solution for
    > this case is in your line "struct.unpack('>f', bytes)"
    > But what can I do when I want the interpret the content from the Integer
    > myInt (*myInt = 0x3F8CCCCD) like 4-byte-real ?
    > This was stored with an othes system in a binary file to
    > CD CC 8C 3F and now is it in python in value. The conversion is not
    > possible. It's right... one of this bytes is an exponent.
    > I want copy the memory content from the "value address" to "myReal
    > address" and use print "%f" %myReal.
    > Is myReal then the right format ?
    > What can I do with python, in FORTH is it simple
    > ( >f f. )
    >
    > gf
    >
    >
    >

    If you really want to do this kind of byte fiddling:
    http://members.dsl-only.net/~daniels/block.html

    Then:
    from block import Block, View
    b = Block(4) # enough space for one float (more is fine)
    iv = View('i', b) # getting to it as an integer
    fv = View('f', b) # same memory as floating point
    iv[0] = 0x3F8CCCCD # Here is a sample just using the integer
    print fv[0]

    On an Intel/Amd/Generic "PC" machine, you should get 1.1

    -Scott David Daniels
     
    Scott David Daniels, Jan 15, 2005
    #5
  6. franzkowiak

    Terry Reedy Guest

    "Scott David Daniels" <> wrote in message
    news:41e952b9$...
    > franzkowiak wrote:
    >> I've read some bytes from a file and just now I can't interpret 4 bytes
    >> in this dates like a real value. An extract from my program:
    >> def l32(c):
    >> return ord(c[0]) + (ord(c[1])<<8) + (ord(c[2])<<16) +
    >> (ord(c[3])<<24)
    >> ...
    >> value = l32(f.read(4)) <--- 3F 8C CC CD should be 1.11
    >>

    > OK, here's the skinny (I used blocks & views to get the answer):
    >
    > import struct
    > bytes = ''.join(chr(int(txt, 16)) for txt in '3F 8C CC CD'.split())
    > struct.unpack('>f', bytes)
    >
    > I was suspicious of that first byte, thought it might be an exponent,
    > since it seemed to have too many on bits in a row to be part of 1.11.


    I believe exponents are typically stored as a positive offset from the
    largest negative exponent. 3F8 is about half of 7FF, so that seems about
    right for an actual exponent of 0.

    Terry J. Reedy
     
    Terry Reedy, Jan 15, 2005
    #6
  7. Scott David Daniels schrieb:
    > G.Franzkowiak wrote:
    >
    >> Scott David Daniels schrieb:
    >>
    >>> franzkowiak wrote:
    >>>
    >>>> I've read some bytes from a file and just now I can't interpret 4
    >>>> bytes in this dates like a real value. An extract from my program:
    >>>> def l32(c):
    >>>> return ord(c[0]) + (ord(c[1])<<8) + (ord(c[2])<<16) +
    >>>> (ord(c[3])<<24)
    >>>> ...
    >>>> value = l32(f.read(4)) <--- 3F 8C CC CD should be 1.11
    >>>>
    >>> OK, here's the skinny (I used blocks & views to get the answer):
    >>>
    >>> import struct
    >>> bytes = ''.join(chr(int(txt, 16)) for txt in '3F 8C CC CD'.split())
    >>> struct.unpack('>f', bytes)
    >>>
    >>> I was suspicious of that first byte, thought it might be an exponent,
    >>> since it seemed to have too many on bits in a row to be part of 1.11.
    >>>
    >>> -Scott David Daniels
    >>>

    >>
    >>
    >>
    >> Ok, I the string exist with "mystr = f.read(4)" and the solution for
    >> this case is in your line "struct.unpack('>f', bytes)"
    >> But what can I do when I want the interpret the content from the
    >> Integer myInt (*myInt = 0x3F8CCCCD) like 4-byte-real ?
    >> This was stored with an othes system in a binary file to
    >> CD CC 8C 3F and now is it in python in value. The conversion is not
    >> possible. It's right... one of this bytes is an exponent.
    >> I want copy the memory content from the "value address" to "myReal
    >> address" and use print "%f" %myReal.
    >> Is myReal then the right format ?
    >> What can I do with python, in FORTH is it simple
    >> ( >f f. )
    >>
    >> gf
    >>
    >>
    >>

    > If you really want to do this kind of byte fiddling:
    > http://members.dsl-only.net/~daniels/block.html
    >
    > Then:
    > from block import Block, View
    > b = Block(4) # enough space for one float (more is fine)
    > iv = View('i', b) # getting to it as an integer
    > fv = View('f', b) # same memory as floating point
    > iv[0] = 0x3F8CCCCD # Here is a sample just using the integer
    > print fv[0]
    >
    > On an Intel/Amd/Generic "PC" machine, you should get 1.1
    >
    > -Scott David Daniels
    >


    That's good :))
    I'm missing the makefile ;-)
    I'm using the other world... right

    Thank you
     
    G.Franzkowiak, Jan 15, 2005
    #7
  8. G.Franzkowiak wrote:
    > Scott David Daniels schrieb:
    >> If you really want to do this kind of byte fiddling:
    >> http://members.dsl-only.net/~daniels/block.html
    >> Then:
    >> from block import Block, View
    >> b = Block(4) # enough space for one float (more is fine)
    >> iv = View('i', b) # getting to it as an integer
    >> fv = View('f', b) # same memory as floating point
    >> iv[0] = 0x3F8CCCCD # Here is a sample just using the integer
    >> print fv[0]
    >> On an Intel/Amd/Generic "PC" machine, you should get 1.1

    >
    > That's good :))
    > I'm missing the makefile ;-)
    > I'm using the other world... right

    There's a lot more than one other world. distlib is your friend.

    There is no makefile. If you are not on a windows box,
    get the source, extract the files from the zip, and run:

    python setup.py install

    -Scott David Daniels
     
    Scott David Daniels, Jan 15, 2005
    #8
  9. On Sat, 15 Jan 2005 11:00:36 -0800, Scott David Daniels <> wrote:

    >G.Franzkowiak wrote:
    >> Scott David Daniels schrieb:
    >>
    >>> franzkowiak wrote:
    >>>
    >>>> I've read some bytes from a file and just now I can't interpret 4
    >>>> bytes in this dates like a real value. An extract from my program:
    >>>> def l32(c):
    >>>> return ord(c[0]) + (ord(c[1])<<8) + (ord(c[2])<<16) +
    >>>> (ord(c[3])<<24)
    >>>> ...
    >>>> value = l32(f.read(4)) <--- 3F 8C CC CD should be 1.11
    >>>>
    >>> OK, here's the skinny (I used blocks & views to get the answer):
    >>>
    >>> import struct
    >>> bytes = ''.join(chr(int(txt, 16)) for txt in '3F 8C CC CD'.split())
    >>> struct.unpack('>f', bytes)
    >>>
    >>> I was suspicious of that first byte, thought it might be an exponent,
    >>> since it seemed to have too many on bits in a row to be part of 1.11.
    >>>
    >>> -Scott David Daniels
    >>>

    >>
    >>
    >> Ok, I the string exist with "mystr = f.read(4)" and the solution for
    >> this case is in your line "struct.unpack('>f', bytes)"
    >> But what can I do when I want the interpret the content from the Integer
    >> myInt (*myInt = 0x3F8CCCCD) like 4-byte-real ?
    >> This was stored with an othes system in a binary file to
    >> CD CC 8C 3F and now is it in python in value. The conversion is not
    >> possible. It's right... one of this bytes is an exponent.
    >> I want copy the memory content from the "value address" to "myReal
    >> address" and use print "%f" %myReal.
    >> Is myReal then the right format ?
    >> What can I do with python, in FORTH is it simple
    >> ( >f f. )
    >>
    >> gf
    >>
    >>
    >>

    >If you really want to do this kind of byte fiddling:
    > http://members.dsl-only.net/~daniels/block.html
    >
    >Then:
    > from block import Block, View
    > b = Block(4) # enough space for one float (more is fine)
    > iv = View('i', b) # getting to it as an integer
    > fv = View('f', b) # same memory as floating point
    > iv[0] = 0x3F8CCCCD # Here is a sample just using the integer
    > print fv[0]
    >
    >On an Intel/Amd/Generic "PC" machine, you should get 1.1
    >

    Ok, for most bit patterns (except QNANs), you can do it in pure python:

    ----< i32as_single.py >------------------------------------
    class NoQNAN(ValueError): pass # use to flag inability to return QNANs

    def i32as_single(i):
    """
    (UIAM in my interpretation of a 1995 Pentium Processor Developer's Manual)
    This converts bits in a 32-bit integer as if they were bits
    of a single-precision IEEE 754 floating point number to python float

    +---+---+---+---+---+---+---+---+---+---+---+---+ ... +---+---+---+---+
    | s | e e e e e e e eb| b b b ... b b b b0 |
    +---+---+---+---^---+---+---+---^---+---+---+---^ ... ^---+---+---+---+
    31 30 29 28 27 26 25 24 23 22 21 20 3 2 1 0

    where s is the sign bit, and is the only thing that changes between + and -
    and e..eb is an 8-bit biased (by 127) exponent, and eb is the hidden
    unit 1 bit followed by 23 b..b0 significant "fraction" bits",
    normalized so that the most significant bit is always 1 and therefore
    doesn't have to be stored at eb, except that when all but the sign bit
    are zero, eb is ignored and the value is then a signed zero value.
    The binary fraction starting bit is after the hidden '1' bit eb at 23,
    so viewing bits 0..23 as an integer, we have to divide by 2**23 (or
    adjust the exponent) to get the 1.xxxx values when the official unbiased
    exponent is zero (offset value 127). Thus 0x3f800000 is zero unbiased
    exponent and no other bits, for a value of 1.0

    A biased exponent of 255 signifies a NaN, and a biased exponent of
    zero signifies a denormal (which doesn't have a hidden bit, and whose
    unit bit is bit 22). Single precision denormals can be normalized
    in python (double) float format, which is done for the return value.
    """
    signbit = i&0x80000000
    if not i&0x7fffffff: return signbit and -0.0 or 0.0 # if -0.0 available
    biased_exp = (i>>23) & 0xff # bits 30-23
    unitbit = biased_exp and 0x800000 or 0 # bit 23, or 0 for denormals
    significand = i & 0x7fffff # bits 22-0
    if biased_exp == 255:
    if significand:
    raise NoQNAN, "Sorry, can't generate QNAN from %08x" % i
    return signbit and -1e9999 or 1e9999 # XXX s/b INF's for sure??
    adjexp = (biased_exp or 1) - 127 - 23 # adjusted for denormal
    posvalue = (significand + unitbit)*2.0**adjexp
    return signbit and -posvalue or posvalue

    def test():
    import struct
    num = 0
    for sign in (0, 2*(-2**30)):
    for i3 in xrange(128):
    num3 = i3<<24
    for i2 in xrange(256):
    print '\r%08x'%(num,), # show progress
    num2 = i2<<16
    # skip mid byte of significand, make 0
    # and twiddle only a few bits at the bottom
    for num0 in xrange(8):
    num = sign+num3+num2+num0
    s = ''.join(map(chr,(num0, 0, i2,((sign and 0x80 or 0)+i3))))
    try: ti32as = i32as_single(num)
    except NoQNAN: continue
    tstruct = struct.unpack('f', s)[0] # XXX '<f' => no INF ??
    if ti32as != tstruct:
    print '\n%x =>%r\n%r => %r' % (num, ti32as, s, tstruct)

    if __name__ == '__main__':
    test()
    -----------------------------------------------------------


    [21:47] C:\pywk\clp>i32as_single.py
    C:\pywk\clp\i32as_single.py:31: FutureWarning: hex/oct constants > sys.maxint will return positi
    ve values in Python 2.4 and up
    signbit = i&0x80000000
    7fff0007C:\pywk\clp\i32as_single.py:51: FutureWarning: %u/%o/%x/%X of negative int will return a
    signed string in Python 2.4 and up
    print '\r%08x'%(num,), # show progress
    ff7f0007C:\pywk\clp\i32as_single.py:38: FutureWarning: %u/%o/%x/%X of negative int will return a
    signed string in Python 2.4 and up
    raise NoQNAN, "Sorry, can't generate QNAN from %08x" % i
    fffe0007

    Of course, the test() gives a clue how you might write the whole thing using struct
    by just summing the four chr-ed extracted bytes from the input i ;-)

    Anyway, a couple things (how to make a QNAN in python without calling struct, and '<' in struct):

    >>> from i32as_single import i32as_single as i32f
    >>> i32f(0x3f8ccccd)

    1.1000000238418579
    >>> i32f(0x00000000)

    0.0
    >>> i32f(0x7f800000)

    1.#INF
    >>> i32f(0xff800000)

    -1.#INF

    But I don't know how to build QNaNs:

    >>> i32f(0x7f800001)

    Traceback (most recent call last):
    File "<stdin>", line 1, in ?
    File "i32as_single.py", line 38, in i32as_single
    raise NoQNAN, "Sorry, can't generate QNAN from %08x" % i
    i32as_single.NoQNAN: Sorry, can't generate QNAN from 7f800001

    Whereas struct does:

    >>> import struct
    >>> struct.unpack('f','\x00\x00\x80\x7f')[0]

    1.#INF
    >>> struct.unpack('f','\x01\x00\x80\x7f')[0]

    1.#QNAN

    BTW, your example:
    >>> struct.unpack('f','\xcd\xcc\x8c\x3f')[0]

    1.1000000238418579
    >>> i32f(0x3f8ccccd)

    1.1000000238418579

    But what does this mean? (I wanted to test with little endian unpacking, since
    that is what I knew I created, but that isn't what '<' means?? ...)

    >>> struct.unpack('f','\x00\x00\x80\x7f')[0]

    1.#INF
    >>> struct.unpack('<f','\x00\x00\x80\x7f')[0]

    3.4028236692093846e+038

    Regards,
    Bengt Richter
     
    Bengt Richter, Jan 16, 2005
    #9
  10. franzkowiak

    Tim Peters Guest

    [Bengt Richter]
    ....
    > But I don't know how to build QNaNs:


    You can subtract infinity from infinity. While all Python behavior in
    the presence of NaNs, infinities, and signed zeroes is a
    platform-dependent accident, it you're on a box that has such things,
    and figure out some (accidental!) way to spell infinity, then inf-inf
    should return a NaN (although on a WinTel box, it's most likely to be
    spelled "-1.#IND" when converted to string).

    ....

    > Whereas struct does:


    No, it doesn't. All Python behavior in the presence of NaNs,
    infinities, and signed zeroes is a platform-dependent accident.

    > >>> import struct
    > >>> struct.unpack('f','\x00\x00\x80\x7f')[0]

    > 1.#INF


    An accident (both that you got back an infinity, and the string
    representation of an infinity).

    > >>> struct.unpack('f','\x01\x00\x80\x7f')[0]

    > 1.#QNAN


    Ditto.

    Those specific accidents are reliable on the box you're using.

    > BTW, your example:
    > >>> struct.unpack('f','\xcd\xcc\x8c\x3f')[0]

    > 1.1000000238418579
    > >>> i32f(0x3f8ccccd)

    > 1.1000000238418579
    >
    > But what does this mean?


    Mostly just that behavior is predictable when you're *not* using
    infinities, NaNs or signed zeroes from Python.

    > (I wanted to test with little endian unpacking, since
    > that is what I knew I created, but that isn't what '<' means?? ...)


    Little-endian is part of what '<' means. '<' also means "use standard
    alignment" and "use standard data size" and "use standard bit
    representation". As soon as you force anything like that, Python has
    to try to interpret the bits itself. In the absence of "<", ">" and
    "!", plain "f" asks for a wholly platform-native result. In that
    case, Python doesn't have to know anything about what the bits might
    mean: it stores the bytes into a native sizeof(float)-byte memory
    area natively aligned for a float, and you get back whatever the
    native C thinks that means.

    > >>> struct.unpack('f','\x00\x00\x80\x7f')[0]

    > 1.#INF


    Entirely determined by the platform C compiler and hardware.

    > >>> struct.unpack('<f','\x00\x00\x80\x7f')[0]

    > 3.4028236692093846e+038


    Entirely determined by what Python thinks the bits should mean, but
    Python doesn't know anything about infinities, NaNs, or signed zeroes,
    and C89 library routines Python calls to construct the value (like
    ldexp()) have no portably defined behavior in their presence either.
     
    Tim Peters, Jan 16, 2005
    #10
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Auronc

    IEEE-754 double class??

    Auronc, Sep 1, 2004, in forum: C++
    Replies:
    0
    Views:
    458
    Auronc
    Sep 1, 2004
  2. Lionel B
    Replies:
    0
    Views:
    1,071
    Lionel B
    Sep 23, 2004
  3. bd
    Replies:
    0
    Views:
    634
  4. Dale Huffman

    IEEE 754 floats

    Dale Huffman, Sep 14, 2004, in forum: Python
    Replies:
    5
    Views:
    508
    Robert Kern
    Sep 15, 2004
  5. Grant Edwards
    Replies:
    11
    Views:
    639
    Grant Edwards
    Jul 15, 2005
Loading...

Share This Page