endianess

G

Guest

Well! Maybe I wrote this word incorrect but its not included im my
e-dictionary!

I have a structure with many int, short etc
I execute this function:
fread(&my_struct, sizeof(struct), 1, filepointer);
The problem is this:
In different endianess platforms from same file will be loaded different
data in the class.

Is there a method to load whole structure at once without this problem?

Until now my approach is:
load whole structure from file.
Instead of my_struct.integer_no_1 I use load_integer(my_struct.integer_no_1)

with load_integer:
--------------------------------
// swap char of an 4 bytes long if machine has "LSB last" storage
//
unsigned long load_integer(unsigned long a)
{
const static short sample = 0x1234;
const static char *test = (char*) &sample;
if (test[0] == 0x34) return a;
return (a << 24) + ((a & 0xff00) << 8) + ((a & 0xff0000) >> 8) + (a >> 24);
}
 
J

Jason

....
Is there a method to load whole structure at once without this problem?
....

No there isn't really. It looks like you are doing the right kind of thing.
Note that if you emit data to some medium in a standard way. i.e. use the <<
and && to store the data in the first place, you can read it in likewise
with << and && without having to check or know anything about endianess.

It is a bit of a problem. It's quite common to implement a function to
convert external data to a structure, and have another function to do the
opposite. This type of thing is mainly necessary for communicable data
sorted in binary form. For example, in the implementation of a portable
IP-stack, or for making data formats portable.

It is usually possible to make a compiler produce "packed" data structures
so that they can be written to some medium in the desired form, but this
still leaves the problem of endianess and also make a program less portable.
 
T

Thomas Matthews

Well! Maybe I wrote this word incorrect but its not included im my
e-dictionary!

I have a structure with many int, short etc
I execute this function:
fread(&my_struct, sizeof(struct), 1, filepointer);
The problem is this:
In different endianess platforms from same file will be loaded different
data in the class.

Is there a method to load whole structure at once without this problem?
No.


Until now my approach is:
load whole structure from file.
Instead of my_struct.integer_no_1 I use load_integer(my_struct.integer_no_1)

with load_integer:
--------------------------------
// swap char of an 4 bytes long if machine has "LSB last" storage
//
unsigned long load_integer(unsigned long a)
{
const static short sample = 0x1234;
const static char *test = (char*) &sample;
if (test[0] == 0x34) return a;
return (a << 24) + ((a & 0xff00) << 8) + ((a & 0xff0000) >> 8) + (a >> 24);
}
-------------------------------

but it is very funny approach....
well! another problem is if long is not 4 bytes, but don't do it so
diffucult!...

The common process is to load the data, then convert to the proper
endianness.
The Endian conversion functions should be in a separate module so that
different conversions can be linked in according to the target platform.
 
E

Eric Sosman

Well! Maybe I wrote this word incorrect but its not included im my
e-dictionary!

I have a structure with many int, short etc
I execute this function:
fread(&my_struct, sizeof(struct), 1, filepointer);
The problem is this:
In different endianess platforms from same file will be loaded different
data in the class.

Is there a method to load whole structure at once without this problem?

This is Question 2.11 in the comp.lang.c Frequently
Asked Questions (FAQ) list

http://www.eskimo.com/~scs/C-faq/top.html

.... and you should also study the other Questions the
answer points to.
 
G

Glen Herrmannsfeldt

Well! Maybe I wrote this word incorrect but its not included im my
e-dictionary!

I have a structure with many int, short etc
I execute this function:
fread(&my_struct, sizeof(struct), 1, filepointer);
The problem is this:
In different endianess platforms from same file will be loaded different
data in the class.

Is there a method to load whole structure at once without this problem?

It is not part of standard C, but RFC 1832 defines XDR which is designed to
solve this problem.

It was put into the public domain by Sun so that it could be freely
implemented.

Note that besides endiannes, structures can also have implementation
dependent padding in them.

-- glen
 
M

Marco de Boer

Well! Maybe I wrote this word incorrect but its not included im my
e-dictionary!

I have a structure with many int, short etc
I execute this function:
fread(&my_struct, sizeof(struct), 1, filepointer);
The problem is this:
In different endianess platforms from same file will be loaded different
data in the class.

Is there a method to load whole structure at once without this problem?

Until now my approach is:
load whole structure from file.
Instead of my_struct.integer_no_1 I use load_integer(my_struct.integer_no_1)

with load_integer:
--------------------------------
// swap char of an 4 bytes long if machine has "LSB last" storage
//
unsigned long load_integer(unsigned long a)
{
const static short sample = 0x1234;
const static char *test = (char*) &sample;
if (test[0] == 0x34) return a;
return (a << 24) + ((a & 0xff00) << 8) + ((a & 0xff0000) >> 8) + (a >> 24);
}
-------------------------------

but it is very funny approach....
well! another problem is if long is not 4 bytes, but don't do it so
diffucult!...
Hi,

Here is a example for Little/Big Endian machines/files.
NOT for middle endian machines/files (PDP).

Marco

/*
** This program works ONLY for
** Little-Endian and Big-Endian files/processors
** NOT for Middle-Endian
*/

#include <stdio.h>

typedef enum {
ENDIAN_NO_INFORMATION,
ENDIAN_LITTLE,
ENDIAN_MIDDLE,
ENDIAN_BIG
} ENDIAN_TYPE;

static int FileWrite(const char*,ENDIAN_TYPE,ENDIAN_TYPE);
static int FileRead(const char*,ENDIAN_TYPE,ENDIAN_TYPE);
static ENDIAN_TYPE UtilEndianType(void);
static void *ReverseBytesInArray(void*,size_t);
static void DumpArray(const void*,size_t);

int main()
{
ENDIAN_TYPE ProcType,FileType;
char *EndianTypeName[]={"NO Information","Little","Middle","Big"};

ProcType=UtilEndianType();
(void)printf("This is a %s-Endian processor.\n",EndianTypeName[ProcType]);
FileType=ENDIAN_LITTLE;
if ( FileWrite("Test.bin",ProcType,FileType)!=0 ) return 1;
FileType=ENDIAN_LITTLE;
if ( FileRead("Test.bin",ProcType,FileType)!=0 ) return 1;
return 0;
}

static int FileWrite(const char *cFile,ENDIAN_TYPE ProcType,ENDIAN_TYPE
FileType)
{
FILE *File;
short sTest;
long lTest;
float fTest;
double dTest;

if ( (File=fopen(cFile,"wb"))==NULL ) return 1;

(void)printf("Writing file '%s'\n",cFile);

sTest=0x1234;
(void)printf("%04X\n",sTest);
if ( ProcType!=FileType ) (void)ReverseBytesInArray(&sTest,sizeof(short));
if ( fwrite(&sTest,sizeof(short),1,File)!=1 ) return 1;

lTest=0x12345678L;
(void)printf("%08X\n",lTest);
if ( ProcType!=FileType ) (void)ReverseBytesInArray(&lTest,sizeof(long));
if ( fwrite(&lTest,sizeof(long),1,File)!=1 ) return 1;

fTest=63.0F;
(void)printf("%.8f\n",fTest);
if ( ProcType!=FileType ) (void)ReverseBytesInArray(&fTest,sizeof(float));
if ( fwrite(&fTest,sizeof(float),1,File)!=1 ) return 1;

dTest=1.234567890123456789;
(void)printf("%.18lf\n",dTest);
if ( ProcType!=FileType ) (void)ReverseBytesInArray(&dTest,sizeof(double));
if ( fwrite(&dTest,sizeof(double),1,File)!=1 ) return 1;

if ( fclose(File)!=0 ) return 1;

return 0;
}

static int FileRead(const char *cFile,ENDIAN_TYPE ProcType,ENDIAN_TYPE
FileType)
{
FILE *File;
short sTest;
long lTest;
float fTest;
double dTest;

if ( (File=fopen(cFile,"rb"))==NULL ) return 1;

(void)printf("Reading file '%s'\n",cFile);

if ( fread(&sTest,sizeof(short),1,File)!=1 ) return 1;
DumpArray(&sTest,sizeof(short));
if ( ProcType!=FileType ) (void)ReverseBytesInArray(&sTest,sizeof(short));
(void)printf("%04X\n",sTest);

if ( fread(&lTest,sizeof(long),1,File)!=1 ) return 1;
DumpArray(&lTest,sizeof(long));
if ( ProcType!=FileType ) (void)ReverseBytesInArray(&lTest,sizeof(long));
(void)printf("%08X\n",lTest);

if ( fread(&fTest,sizeof(float),1,File)!=1 ) return 1;
DumpArray(&fTest,sizeof(float));
if ( ProcType!=FileType ) (void)ReverseBytesInArray(&fTest,sizeof(float));
(void)printf("%.8f\n",fTest);

if ( fread(&dTest,sizeof(double),1,File)!=1 ) return 1;
DumpArray(&dTest,sizeof(double));
if ( ProcType!=FileType ) (void)ReverseBytesInArray(&dTest,sizeof(double));
(void)printf("%.18lf\n",dTest);

if ( fclose(File)!=0 ) return 1;

return 0;
}

static ENDIAN_TYPE UtilEndianType()
{
ENDIAN_TYPE EndianType=ENDIAN_NO_INFORMATION;
unsigned long Value=0x12345678;
unsigned char *cPtr=(unsigned char*)&Value;

if ( *cPtr==0x12 && *(cPtr+1)==0x34 && *(cPtr+2)==0x56 && *(cPtr+3)==0x78 )
EndianType=ENDIAN_BIG;
else if ( *cPtr==0x78 && *(cPtr+1)==0x56 && *(cPtr+2)==0x34 &&
*(cPtr+3)==0x12 )
EndianType=ENDIAN_LITTLE;
else if ( *cPtr==0x34 && *(cPtr+1)==0x12 && *(cPtr+2)==0x78 &&
*(cPtr+3)==0x56 )
EndianType=ENDIAN_MIDDLE;
return EndianType;
}

static void *ReverseBytesInArray(void *Buffer,size_t Size)
{
unsigned char *cPtr0,*cPtr1,cTmp;

cPtr0=Buffer;
cPtr1=cPtr0+Size;
while ( cPtr0<--cPtr1 ) {
cTmp=*cPtr0;
*cPtr0++=*cPtr1;
*cPtr1=cTmp;
}
return Buffer;
}

static void DumpArray(const void *Array,size_t Size)
{
const unsigned char *cPtr=Array;
size_t i;

for ( i=0; i<Size; i++ )
(void)printf("%02X%c",*cPtr++,(i+1<Size)?' ':',');
return;
}
 
M

Malcolm

I have a structure with many int, short etc
I execute this function:
fread(&my_struct, sizeof(struct), 1, filepointer);
The problem is this:
In different endianess platforms from same file will be loaded different
data in the class.

Is there a method to load whole structure at once without this problem?
fread() should be banned for this reason.
There is no single call which will load a C structure portably. As well as
endianess, you have problems of padding, float formats, and even character
set.

However you can write an int get32be(FILE *fp) function which loads a 32-bit
big-endian integer from a file.

int get32be(FILE *fp)
{
int answer = 0;

answer = (fgetc(fp) & 0xFF) << 24;
answer |= (fgetc(fp) & 0xFF) << 16;
answer |= (fgetc(fp) & 0xFF) << 8;
answer |= (fgetc(fp) & 0xFF);

return answer;
}

The calling function needs to call ferror() at some time to check for EOF.

Writing an fget32le() - get 32bit integer little endian and write functions,
you can implement a portable read/write binary interface.
 
E

Emmanuel Delahaye

<- Chameleon -> said:
I have a structure with many int, short etc
I execute this function:
fread(&my_struct, sizeof(struct), 1, filepointer);

You meant:

fread (&my_struct, sizeof my_struct, 1, filepointer);

This is not portable. You can't map a byte stream on a structure. Don't do
that. Never.
The problem is this:
In different endianess platforms from same file will be loaded different
data in the class.

Sure, because what you are doing is not portable. It's called lazy
programming. It can work under certain circumstances, but it's not portable
because:

- The layout of a structure is implementation-dependent
- The size of the types is implementation-dependent
- The endianness is implementation-dependent
- The bit order in bitfields is implementation-dependent

A stream is just a sequence of octets (called 'bytes' in a C context). Hence
you must treat them the like:

unsigned char buffer[WHAT_YA_NEED];

fread (buffer, sizeof buffer, 1, filepointer);
Is there a method to load whole structure at once without this problem?

No. You must pick the bytes one by one, and put them into the members of your
/internal/ structure:

according to the stream specifications (often MSB-LSB, aka network order)

e.g.:

/* byte */
mystruct.my_byte = buffer[0];

/* 'word' */
mystruct.my_word = buffer[1] << 8 /* MSB */
| buffer[2] << 0; /* LSB */
/* 'long' */
mystruct.my_long = buffer[3] << 24 /* MSB */
| buffer[4] << 16
| buffer[5] << 8
| buffer[6] << 0; /* LSB */
 
K

Kevin Easton

Malcolm said:
fread() should be banned for this reason.
There is no single call which will load a C structure portably. As well as
endianess, you have problems of padding, float formats, and even character
set.

However you can write an int get32be(FILE *fp) function which loads a 32-bit
big-endian integer from a file.

int get32be(FILE *fp)
{

If you make this "unsigned long" (int only has to have at least 16 value
bits), then you have a portable getu32be (You should also check for
fgetc returing EOF, and cast the checked return value to unsigned long
before shifting).

Reading signed integers is harder - you'll need to check the sign bit and
act on it later. Of course, portably reading and writing floating point
values is harder still.

- Kevin.
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

No members online now.

Forum statistics

Threads
473,744
Messages
2,569,484
Members
44,903
Latest member
orderPeak8CBDGummies

Latest Threads

Top