finding byte order

[biner]

Hello,

I am using a program that has to read binary data from files coming
from different machines. The file are always written with big endian.
I am using the struct module to read the data and it is fine because I
can specify in the format if the data are to be read with big or small
endian convention.

I would like to use the array module instead of struct because it is
suppose to be faster for big arrays. However, this module does not
provide a format specifier to say if the data are writtent with big or
small endian. The result is that it works on big-endian machine and
not on small-endian machine.

I would like to have a test to tell me if the current machine is
using big or small endian, this way I could use the array module in
the first case and the *slower* struct module on the second. I looked
but did not find. Is there a python function to know that?

Thanks!

 

[Scott David Daniels]

> I would like to have a test to tell me if the current machine is
> using big or small endian, this way I could use the array module in
> the first case and the *slower* struct module on the second. I looked
> but did not find. Is there a python function to know that?
>
> Thanks!

How about sys.byteorder?

At least if you are using array.array, note the "byteswap" method:

>>> import array
>>> v = array.array('h',range(8))
>>> v array('h', [0, 1, 2, 3, 4, 5, 6, 7])
>>> v.byteswap()
>>> v

array('h', [0, 256, 512, 768, 1024, 1280, 1536, 1792])

--Scott David Daniels
(e-mail address removed)

 

[Diez B. Roggisch]

How about sys.byteorder?

This doesn't help, as he wants to read files from varying endianess - what
the _current_ endianess is doesn't matter here.

 

[Diez B. Roggisch]

I would like to have a test to tell me if the current machine is

using big or small endian, this way I could use the array module in
the first case and the *slower* struct module on the second. I looked
but did not find. Is there a python function to know that?

There is no such test, as it's domain-specific if there _can_ be such a test
or not. If your data is composed in a way that you can infer the endianess
by reading e.g. a header at the beginning that has well-known fields, then
it might work. But only you can do that.

For a pure binary file, there is absolutely no way of telling the endianess.

 

[Scott David Daniels]

> I am using a program that has to read binary data from files coming
> from different machines. The file are always written with big endian.

Diez B. Roggisch wrote:
[Scott David Daniels wrote]

This doesn't help, as he wants to read files from varying endianess - what
the _current_ endianess is doesn't matter here.

But, in fact, he says the files are always big endian. So, code like
the following should address his problem. Note I use type 'h' as an
example so I can easily read samples.

import sys, array
f =open('huge.dat')
v = array.array('h') # Or whatever data type
v.fromfile(f, 4096)
f.close()
if sys.byteorder == 'little':
v.byteswap()

--Scott David Daniels
(e-mail address removed)

 

[Diez B. Roggisch]

But, in fact, he says the files are always big endian. So, code like

the following should address his problem. Note I use type 'h' as an
example so I can easily read samples.

I'm sorry, I confused that he asked for machine endianess. Then of course
you are right.

 

[biner.sebastien]

I am using a program that has to read binary data from files coming
from different machines. The file are always written with big

endian.

Diez B. Roggisch wrote:
[Scott David Daniels wrote]

This doesn't help, as he wants to read files from varying endianess - what
the _current_ endianess is doesn't matter here.

But, in fact, he says the files are always big endian. So, code like
the following should address his problem. Note I use type 'h' as an
example so I can easily read samples.

import sys, array
f =open('huge.dat')
v = array.array('h') # Or whatever data type
v.fromfile(f, 4096)
f.close()
if sys.byteorder == 'little':
v.byteswap()

--Scott David Daniels
(e-mail address removed)

This seems to do the what I want. I did not know about array.byteswap
and sys.byteorder.
Thanks for taking the time to answer my silly question.

Ciao!