Endianness conversion

Discussion in 'Python' started by Toby, Feb 24, 2007.

  1. Toby

    Toby Guest

    As part of a program I'm writing, I need to save to disk big amounts of
    data (hundreds of MB, in 8kB chunks) swapping every couple of bytes.

    I obviously cannot do it in a Python loop.

    Is there a function I could use in the standard library, or do I have to
    write my own C extension?


    Toby
     
    Toby, Feb 24, 2007
    #1
    1. Advertisements

  2. Toby

    Dan Sommers Guest

    You could try the struct module. If your input comes in fixed sized
    chunks, just call struct.unpack and struct.pack once per chunk.

    HTH,
    Dan
     
    Dan Sommers, Feb 24, 2007
    #2
    1. Advertisements

  3. Toby

    Toby Guest

    Thanks, but it was a bit awkward to use for big chunks.

    I ended up writing my own byteswapper in Pyrex:


    def swapbytes(data):
    "Swap every two bytes of a even-sized python string, in place"
    cdef int i
    cdef char t, *p
    p = data
    for i from 0 <= i < len(data) / 2:
    t = p[0]
    p[0] = p[1]
    p[1] = t
    p = p + 2


    Toby
     
    Toby, Feb 24, 2007
    #3
  4. Try the using the array module. array objects provide a byteswap
    method which reverses endianness.

    -Daniel
     
    Daniel Harding, Feb 24, 2007
    #4
  5. Toby

    Toby Guest

    Thanks!


    Toby
     
    Toby, Feb 24, 2007
    #5
  6. You can use the byteswap method of arrays:
    'BAdc65'
     
    Gabriel Genellina, Feb 25, 2007
    #6
  7. Toby

    Dan Sommers Guest

    def swapper( bytestring ):
    someHs = len( bytestring ) / 2 * "H" # could check for odd-lengths?
    unpackfmt = "<" + someHs
    packfmt = ">" + someHs
    return struct.pack( packfmt, *struct.unpack( unpackfmt, bytestring )
     
    Dan Sommers, Feb 25, 2007
    #7
    1. Advertisements

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments (here). After that, you can post your question and our members will help you out.