Endianness conversion

[Toby]

As part of a program I'm writing, I need to save to disk big amounts of
data (hundreds of MB, in 8kB chunks) swapping every couple of bytes.

I obviously cannot do it in a Python loop.

Is there a function I could use in the standard library, or do I have to
write my own C extension?


Toby

 

[Dan Sommers]

As part of a program I'm writing, I need to save to disk big amounts of
data (hundreds of MB, in 8kB chunks) swapping every couple of bytes.

I obviously cannot do it in a Python loop.

Is there a function I could use in the standard library, or do I have to
write my own C extension?

You could try the struct module. If your input comes in fixed sized
chunks, just call struct.unpack and struct.pack once per chunk.

HTH,
Dan

 

[Toby]

You could try the struct module. If your input comes in fixed sized
chunks, just call struct.unpack and struct.pack once per chunk.

Thanks, but it was a bit awkward to use for big chunks.

I ended up writing my own byteswapper in Pyrex:


def swapbytes(data):
"Swap every two bytes of a even-sized python string, in place"
cdef int i
cdef char t, *p
p = data
for i from 0 <= i < len(data) / 2:
t = p[0]
p[0] = p[1]
p[1] = t
p = p + 2


Toby

 

[Daniel Harding]

As part of a program I'm writing, I need to save to disk big amounts of
data (hundreds of MB, in 8kB chunks) swapping every couple of bytes.

I obviously cannot do it in a Python loop.

Is there a function I could use in the standard library, or do I have to
write my own C extension?

Try the using the array module. array objects provide a byteswap
method which reverses endianness.

-Daniel

 

[Toby]

Try the using the array module. array objects provide a byteswap
method which reverses endianness.

Thanks!


Toby

 

[Gabriel Genellina]

I ended up writing my own byteswapper in Pyrex:

You can use the byteswap method of arrays:
'BAdc65'

 

[Dan Sommers]

Thanks, but it was a bit awkward to use for big chunks.

def swapper( bytestring ):
someHs = len( bytestring ) / 2 * "H" # could check for odd-lengths?
unpackfmt = "<" + someHs
packfmt = ">" + someHs
return struct.pack( packfmt, *struct.unpack( unpackfmt, bytestring )