endianess

Discussion in 'C Programming' started by Guest, Jul 18, 2003.

  1. Guest

    Guest Guest

    Well! Maybe I wrote this word incorrect but its not included im my
    e-dictionary!

    I have a structure with many int, short etc
    I execute this function:
    fread(&my_struct, sizeof(struct), 1, filepointer);
    The problem is this:
    In different endianess platforms from same file will be loaded different
    data in the class.

    Is there a method to load whole structure at once without this problem?

    Until now my approach is:
    load whole structure from file.
    Instead of my_struct.integer_no_1 I use load_integer(my_struct.integer_no_1)

    with load_integer:
    --------------------------------
    // swap char of an 4 bytes long if machine has "LSB last" storage
    //
    unsigned long load_integer(unsigned long a)
    {
    const static short sample = 0x1234;
    const static char *test = (char*) &sample;
    if (test[0] == 0x34) return a;
    return (a << 24) + ((a & 0xff00) << 8) + ((a & 0xff0000) >> 8) + (a >> 24);
    }
    -------------------------------

    but it is very funny approach....
    well! another problem is if long is not 4 bytes, but don't do it so
    diffucult!...
    Guest, Jul 18, 2003
    #1
    1. Advertising

  2. Guest

    Jason Guest

    "<- Chameleon ->" <> wrote in message
    news:bf8pnt$sps$...
    ....
    > Is there a method to load whole structure at once without this problem?

    ....

    No there isn't really. It looks like you are doing the right kind of thing.
    Note that if you emit data to some medium in a standard way. i.e. use the <<
    and && to store the data in the first place, you can read it in likewise
    with << and && without having to check or know anything about endianess.

    It is a bit of a problem. It's quite common to implement a function to
    convert external data to a structure, and have another function to do the
    opposite. This type of thing is mainly necessary for communicable data
    sorted in binary form. For example, in the implementation of a portable
    IP-stack, or for making data formats portable.

    It is usually possible to make a compiler produce "packed" data structures
    so that they can be written to some medium in the desired form, but this
    still leaves the problem of endianess and also make a program less portable.
    Jason, Jul 18, 2003
    #2
    1. Advertising

  3. <- Chameleon -> wrote:

    > Well! Maybe I wrote this word incorrect but its not included im my
    > e-dictionary!
    >
    > I have a structure with many int, short etc
    > I execute this function:
    > fread(&my_struct, sizeof(struct), 1, filepointer);
    > The problem is this:
    > In different endianess platforms from same file will be loaded different
    > data in the class.
    >
    > Is there a method to load whole structure at once without this problem?


    No.


    > Until now my approach is:
    > load whole structure from file.
    > Instead of my_struct.integer_no_1 I use load_integer(my_struct.integer_no_1)
    >
    > with load_integer:
    > --------------------------------
    > // swap char of an 4 bytes long if machine has "LSB last" storage
    > //
    > unsigned long load_integer(unsigned long a)
    > {
    > const static short sample = 0x1234;
    > const static char *test = (char*) &sample;
    > if (test[0] == 0x34) return a;
    > return (a << 24) + ((a & 0xff00) << 8) + ((a & 0xff0000) >> 8) + (a >> 24);
    > }
    > -------------------------------
    >
    > but it is very funny approach....
    > well! another problem is if long is not 4 bytes, but don't do it so
    > diffucult!...


    The common process is to load the data, then convert to the proper
    endianness.
    The Endian conversion functions should be in a separate module so that
    different conversions can be linked in according to the target platform.

    --
    Thomas Matthews
    C Faq: http://www.eskimo.com/~scs/c-faq/top.html
    alt.comp.lang.learn.c-c++ faq:
    http://www.raos.demon.uk/acllc-c /faq.html
    Thomas Matthews, Jul 18, 2003
    #3
  4. Guest

    Eric Sosman Guest

    "<- Chameleon ->" wrote:
    >
    > Well! Maybe I wrote this word incorrect but its not included im my
    > e-dictionary!
    >
    > I have a structure with many int, short etc
    > I execute this function:
    > fread(&my_struct, sizeof(struct), 1, filepointer);
    > The problem is this:
    > In different endianess platforms from same file will be loaded different
    > data in the class.
    >
    > Is there a method to load whole structure at once without this problem?


    This is Question 2.11 in the comp.lang.c Frequently
    Asked Questions (FAQ) list

    http://www.eskimo.com/~scs/C-faq/top.html

    .... and you should also study the other Questions the
    answer points to.

    --
    Eric Sosman, Jul 18, 2003
    #4
  5. "<- Chameleon ->" <> wrote in message
    news:bf8pnt$sps$...
    > Well! Maybe I wrote this word incorrect but its not included im my
    > e-dictionary!
    >
    > I have a structure with many int, short etc
    > I execute this function:
    > fread(&my_struct, sizeof(struct), 1, filepointer);
    > The problem is this:
    > In different endianess platforms from same file will be loaded different
    > data in the class.
    >
    > Is there a method to load whole structure at once without this problem?


    It is not part of standard C, but RFC 1832 defines XDR which is designed to
    solve this problem.

    It was put into the public domain by Sun so that it could be freely
    implemented.

    Note that besides endiannes, structures can also have implementation
    dependent padding in them.

    -- glen
    Glen Herrmannsfeldt, Jul 18, 2003
    #5
  6. "<- Chameleon ->" <> wrote in message
    news:bf8pnt$sps$...
    > Well! Maybe I wrote this word incorrect but its not included im my
    > e-dictionary!
    >
    > I have a structure with many int, short etc
    > I execute this function:
    > fread(&my_struct, sizeof(struct), 1, filepointer);
    > The problem is this:
    > In different endianess platforms from same file will be loaded different
    > data in the class.
    >
    > Is there a method to load whole structure at once without this problem?
    >
    > Until now my approach is:
    > load whole structure from file.
    > Instead of my_struct.integer_no_1 I use

    load_integer(my_struct.integer_no_1)
    >
    > with load_integer:
    > --------------------------------
    > // swap char of an 4 bytes long if machine has "LSB last" storage
    > //
    > unsigned long load_integer(unsigned long a)
    > {
    > const static short sample = 0x1234;
    > const static char *test = (char*) &sample;
    > if (test[0] == 0x34) return a;
    > return (a << 24) + ((a & 0xff00) << 8) + ((a & 0xff0000) >> 8) + (a >>

    24);
    > }
    > -------------------------------
    >
    > but it is very funny approach....
    > well! another problem is if long is not 4 bytes, but don't do it so
    > diffucult!...
    >
    >

    Hi,

    Here is a example for Little/Big Endian machines/files.
    NOT for middle endian machines/files (PDP).

    Marco

    /*
    ** This program works ONLY for
    ** Little-Endian and Big-Endian files/processors
    ** NOT for Middle-Endian
    */

    #include <stdio.h>

    typedef enum {
    ENDIAN_NO_INFORMATION,
    ENDIAN_LITTLE,
    ENDIAN_MIDDLE,
    ENDIAN_BIG
    } ENDIAN_TYPE;

    static int FileWrite(const char*,ENDIAN_TYPE,ENDIAN_TYPE);
    static int FileRead(const char*,ENDIAN_TYPE,ENDIAN_TYPE);
    static ENDIAN_TYPE UtilEndianType(void);
    static void *ReverseBytesInArray(void*,size_t);
    static void DumpArray(const void*,size_t);

    int main()
    {
    ENDIAN_TYPE ProcType,FileType;
    char *EndianTypeName[]={"NO Information","Little","Middle","Big"};

    ProcType=UtilEndianType();
    (void)printf("This is a %s-Endian processor.\n",EndianTypeName[ProcType]);
    FileType=ENDIAN_LITTLE;
    if ( FileWrite("Test.bin",ProcType,FileType)!=0 ) return 1;
    FileType=ENDIAN_LITTLE;
    if ( FileRead("Test.bin",ProcType,FileType)!=0 ) return 1;
    return 0;
    }

    static int FileWrite(const char *cFile,ENDIAN_TYPE ProcType,ENDIAN_TYPE
    FileType)
    {
    FILE *File;
    short sTest;
    long lTest;
    float fTest;
    double dTest;

    if ( (File=fopen(cFile,"wb"))==NULL ) return 1;

    (void)printf("Writing file '%s'\n",cFile);

    sTest=0x1234;
    (void)printf("%04X\n",sTest);
    if ( ProcType!=FileType ) (void)ReverseBytesInArray(&sTest,sizeof(short));
    if ( fwrite(&sTest,sizeof(short),1,File)!=1 ) return 1;

    lTest=0x12345678L;
    (void)printf("%08X\n",lTest);
    if ( ProcType!=FileType ) (void)ReverseBytesInArray(&lTest,sizeof(long));
    if ( fwrite(&lTest,sizeof(long),1,File)!=1 ) return 1;

    fTest=63.0F;
    (void)printf("%.8f\n",fTest);
    if ( ProcType!=FileType ) (void)ReverseBytesInArray(&fTest,sizeof(float));
    if ( fwrite(&fTest,sizeof(float),1,File)!=1 ) return 1;

    dTest=1.234567890123456789;
    (void)printf("%.18lf\n",dTest);
    if ( ProcType!=FileType ) (void)ReverseBytesInArray(&dTest,sizeof(double));
    if ( fwrite(&dTest,sizeof(double),1,File)!=1 ) return 1;

    if ( fclose(File)!=0 ) return 1;

    return 0;
    }

    static int FileRead(const char *cFile,ENDIAN_TYPE ProcType,ENDIAN_TYPE
    FileType)
    {
    FILE *File;
    short sTest;
    long lTest;
    float fTest;
    double dTest;

    if ( (File=fopen(cFile,"rb"))==NULL ) return 1;

    (void)printf("Reading file '%s'\n",cFile);

    if ( fread(&sTest,sizeof(short),1,File)!=1 ) return 1;
    DumpArray(&sTest,sizeof(short));
    if ( ProcType!=FileType ) (void)ReverseBytesInArray(&sTest,sizeof(short));
    (void)printf("%04X\n",sTest);

    if ( fread(&lTest,sizeof(long),1,File)!=1 ) return 1;
    DumpArray(&lTest,sizeof(long));
    if ( ProcType!=FileType ) (void)ReverseBytesInArray(&lTest,sizeof(long));
    (void)printf("%08X\n",lTest);

    if ( fread(&fTest,sizeof(float),1,File)!=1 ) return 1;
    DumpArray(&fTest,sizeof(float));
    if ( ProcType!=FileType ) (void)ReverseBytesInArray(&fTest,sizeof(float));
    (void)printf("%.8f\n",fTest);

    if ( fread(&dTest,sizeof(double),1,File)!=1 ) return 1;
    DumpArray(&dTest,sizeof(double));
    if ( ProcType!=FileType ) (void)ReverseBytesInArray(&dTest,sizeof(double));
    (void)printf("%.18lf\n",dTest);

    if ( fclose(File)!=0 ) return 1;

    return 0;
    }

    static ENDIAN_TYPE UtilEndianType()
    {
    ENDIAN_TYPE EndianType=ENDIAN_NO_INFORMATION;
    unsigned long Value=0x12345678;
    unsigned char *cPtr=(unsigned char*)&Value;

    if ( *cPtr==0x12 && *(cPtr+1)==0x34 && *(cPtr+2)==0x56 && *(cPtr+3)==0x78 )
    EndianType=ENDIAN_BIG;
    else if ( *cPtr==0x78 && *(cPtr+1)==0x56 && *(cPtr+2)==0x34 &&
    *(cPtr+3)==0x12 )
    EndianType=ENDIAN_LITTLE;
    else if ( *cPtr==0x34 && *(cPtr+1)==0x12 && *(cPtr+2)==0x78 &&
    *(cPtr+3)==0x56 )
    EndianType=ENDIAN_MIDDLE;
    return EndianType;
    }

    static void *ReverseBytesInArray(void *Buffer,size_t Size)
    {
    unsigned char *cPtr0,*cPtr1,cTmp;

    cPtr0=Buffer;
    cPtr1=cPtr0+Size;
    while ( cPtr0<--cPtr1 ) {
    cTmp=*cPtr0;
    *cPtr0++=*cPtr1;
    *cPtr1=cTmp;
    }
    return Buffer;
    }

    static void DumpArray(const void *Array,size_t Size)
    {
    const unsigned char *cPtr=Array;
    size_t i;

    for ( i=0; i<Size; i++ )
    (void)printf("%02X%c",*cPtr++,(i+1<Size)?' ':',');
    return;
    }
    Marco de Boer, Jul 19, 2003
    #6
  7. Guest

    Malcolm Guest

    "<- Chameleon ->" <> wrote in message
    news:bf8pnt$sps$...
    >
    > I have a structure with many int, short etc
    > I execute this function:
    > fread(&my_struct, sizeof(struct), 1, filepointer);
    > The problem is this:
    > In different endianess platforms from same file will be loaded different
    > data in the class.
    >
    > Is there a method to load whole structure at once without this problem?
    >

    fread() should be banned for this reason.
    There is no single call which will load a C structure portably. As well as
    endianess, you have problems of padding, float formats, and even character
    set.

    However you can write an int get32be(FILE *fp) function which loads a 32-bit
    big-endian integer from a file.

    int get32be(FILE *fp)
    {
    int answer = 0;

    answer = (fgetc(fp) & 0xFF) << 24;
    answer |= (fgetc(fp) & 0xFF) << 16;
    answer |= (fgetc(fp) & 0xFF) << 8;
    answer |= (fgetc(fp) & 0xFF);

    return answer;
    }

    The calling function needs to call ferror() at some time to check for EOF.

    Writing an fget32le() - get 32bit integer little endian and write functions,
    you can implement a portable read/write binary interface.
    Malcolm, Jul 19, 2003
    #7
  8. In 'comp.lang.c', "<- Chameleon ->" <> wrote:

    > I have a structure with many int, short etc
    > I execute this function:
    > fread(&my_struct, sizeof(struct), 1, filepointer);


    You meant:

    fread (&my_struct, sizeof my_struct, 1, filepointer);

    This is not portable. You can't map a byte stream on a structure. Don't do
    that. Never.

    > The problem is this:
    > In different endianess platforms from same file will be loaded different
    > data in the class.


    Sure, because what you are doing is not portable. It's called lazy
    programming. It can work under certain circumstances, but it's not portable
    because:

    - The layout of a structure is implementation-dependent
    - The size of the types is implementation-dependent
    - The endianness is implementation-dependent
    - The bit order in bitfields is implementation-dependent

    A stream is just a sequence of octets (called 'bytes' in a C context). Hence
    you must treat them the like:

    unsigned char buffer[WHAT_YA_NEED];

    fread (buffer, sizeof buffer, 1, filepointer);

    > Is there a method to load whole structure at once without this problem?


    No. You must pick the bytes one by one, and put them into the members of your
    /internal/ structure:

    according to the stream specifications (often MSB-LSB, aka network order)

    e.g.:

    /* byte */
    mystruct.my_byte = buffer[0];

    /* 'word' */
    mystruct.my_word = buffer[1] << 8 /* MSB */
    | buffer[2] << 0; /* LSB */
    /* 'long' */
    mystruct.my_long = buffer[3] << 24 /* MSB */
    | buffer[4] << 16
    | buffer[5] << 8
    | buffer[6] << 0; /* LSB */

    --
    -ed- [remove YOURBRA before answering me]
    The C-language FAQ: http://www.eskimo.com/~scs/C-faq/top.html
    <blank line>
    FAQ de f.c.l.c : http://www.isty-info.uvsq.fr/~rumeau/fclc/
    Emmanuel Delahaye, Jul 19, 2003
    #8
  9. Guest

    Kevin Easton Guest

    Malcolm <> wrote:
    >
    > "<- Chameleon ->" <> wrote in message
    > news:bf8pnt$sps$...
    >>
    >> I have a structure with many int, short etc
    >> I execute this function:
    >> fread(&my_struct, sizeof(struct), 1, filepointer);
    >> The problem is this:
    >> In different endianess platforms from same file will be loaded different
    >> data in the class.
    >>
    >> Is there a method to load whole structure at once without this problem?
    >>

    > fread() should be banned for this reason.
    > There is no single call which will load a C structure portably. As well as
    > endianess, you have problems of padding, float formats, and even character
    > set.
    >
    > However you can write an int get32be(FILE *fp) function which loads a 32-bit
    > big-endian integer from a file.
    >
    > int get32be(FILE *fp)
    > {


    If you make this "unsigned long" (int only has to have at least 16 value
    bits), then you have a portable getu32be (You should also check for
    fgetc returing EOF, and cast the checked return value to unsigned long
    before shifting).

    Reading signed integers is harder - you'll need to check the sign bit and
    act on it later. Of course, portably reading and writing floating point
    values is harder still.

    - Kevin.
    Kevin Easton, Jul 20, 2003
    #9
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. hantheman

    Portable endianess

    hantheman, Dec 20, 2003, in forum: C++
    Replies:
    6
    Views:
    4,366
    Nick Hounsome
    Dec 28, 2003
  2. Endianess...

    , Feb 2, 2005, in forum: C++
    Replies:
    4
    Views:
    479
    Howard
    Feb 3, 2005
  3. SpOiLeR
    Replies:
    5
    Views:
    1,346
    SpOiLeR
    Mar 16, 2005
  4. Oliver Knoll
    Replies:
    12
    Views:
    434
    RoSsIaCrIiLoIA
    Sep 16, 2004
  5. jlara

    Checking endianess in compile time

    jlara, Jun 6, 2005, in forum: C Programming
    Replies:
    7
    Views:
    381
    Michael Mair
    Jun 7, 2005
Loading...

Share This Page