char data[0]

Discussion in 'C Programming' started by aarklon@gmail.com, Oct 10, 2006.

  1. Guest

    Hi all,

    this is a question which i saw in a book

    typedef struct mall_li_header_ {
    int refcnt;
    uchar pool;
    uchar flag;
    ushort magic_no;
    char data[0];

    } mall_li_header_t;

    What is the use of data[0] here ?
    to which answer i found from my colleagues as

    this is a very old (and common) idea used by a lot of game programmers
    from yesteryears.
    The answer for completeness sake - common use is : allocate arbitrary
    size - anything above the size of the struct can be referenced as
    "data".

    to what extent this is true?????
     
    , Oct 10, 2006
    #1
    1. Advertising

  2. wrote:

    > this is a question which i saw in a book


    (snip struct hack)

    > What is the use of data[0] here ?
    > to which answer i found from my colleagues as


    (snip)

    What your colleagues said is similar to the FAQ's content:
    http://c-faq.com/struct/structhack.html

    --
    C. Benson Manica | I *should* know what I'm talking about - if I
    cbmanica(at)gmail.com | don't, I need to know. Flames welcome.
     
    Christopher Benson-Manica, Oct 10, 2006
    #2
    1. Advertising

  3. Jack Klein Guest

    On 10 Oct 2006 11:56:12 -0700, wrote in comp.lang.c:

    > Hi all,
    >
    > this is a question which i saw in a book
    >
    > typedef struct mall_li_header_ {
    > int refcnt;
    > uchar pool;
    > uchar flag;
    > ushort magic_no;
    > char data[0];
    >
    > } mall_li_header_t;
    >
    > What is the use of data[0] here ?
    > to which answer i found from my colleagues as
    >
    > this is a very old (and common) idea used by a lot of game programmers
    > from yesteryears.
    > The answer for completeness sake - common use is : allocate arbitrary
    > size - anything above the size of the struct can be referenced as
    > "data".
    >
    > to what extent this is true?????


    What it is actually for is testing for broken compilers, since an
    array declaration with a size of 0 has never been allowed in any
    version of standard C.

    --
    Jack Klein
    Home: http://JK-Technology.Com
    FAQs for
    comp.lang.c http://c-faq.com/
    comp.lang.c++ http://www.parashift.com/c -faq-lite/
    alt.comp.lang.learn.c-c++
    http://www.contrib.andrew.cmu.edu/~ajo/docs/FAQ-acllc.html
     
    Jack Klein, Oct 11, 2006
    #3
  4. Old Wolf Guest

    Jack Klein wrote:
    > wrote:
    > >
    > > typedef struct mall_li_header_ {
    > > char data[0];

    >
    > What it is actually for is testing for broken compilers, since an
    > array declaration with a size of 0 has never been allowed in any
    > version of standard C.


    Compilers are allowed to offer extensions that don't alter the
    behaviour of any conforming program.
     
    Old Wolf, Oct 11, 2006
    #4
  5. jmcgill Guest

    wrote:

    > to what extent this is true?????


    "none."
     
    jmcgill, Oct 11, 2006
    #5
  6. jmcgill Guest

    wrote:

    > this is a very old (and common) idea used by a lot of game programmers
    > from yesteryears.


    I remember a lot of yesteryears' games failing with bizarre memory
    problems, especially when running them on later OS versions, with
    different memory managers, etc.

    If you want an incomplete type, just put a void * in your struct and
    allocate that properly. Would that kill you?
     
    jmcgill, Oct 11, 2006
    #6
  7. "Old Wolf" <> writes:
    > Jack Klein wrote:
    >> wrote:
    >> >
    >> > typedef struct mall_li_header_ {
    >> > char data[0];

    >>
    >> What it is actually for is testing for broken compilers, since an
    >> array declaration with a size of 0 has never been allowed in any
    >> version of standard C.

    >
    > Compilers are allowed to offer extensions that don't alter the
    > behaviour of any conforming program.


    Yes, but if the expression is constant and doesn't have a value
    greater than zero, it's a constraint violation requiring a diagnostic.
    A compiler is free to do what it likes after issuing the diagnostic.
    (And of course it's free not to issue the diagnostic in non-conforming
    mode; the standard by definition cannot constrain implementations that
    don't conform to it.)

    --
    Keith Thompson (The_Other_Keith) <http://www.ghoti.net/~kst>
    San Diego Supercomputer Center <*> <http://users.sdsc.edu/~kst>
    We must do something. This is something. Therefore, we must do this.
     
    Keith Thompson, Oct 11, 2006
    #7
  8. jmcgill <> writes:
    > wrote:
    >
    >> this is a very old (and common) idea used by a lot of game programmers
    >> from yesteryears.

    >
    > I remember a lot of yesteryears' games failing with bizarre memory
    > problems, especially when running them on later OS versions, with
    > different memory managers, etc.
    >
    > If you want an incomplete type, just put a void * in your struct and
    > allocate that properly. Would that kill you?


    The struct hack has the advantage that the structure and the array can
    be allocated contiguously. It's certainly not legal if you declare
    the array with [0], but with [1] it's arguably legal and de facto
    portable. It's sufficiently popular that C99 provided a well-defined
    version, flexible array members.

    See the comp.lang.c FAQ, <http://www.c-faq.com/>, question 2.6.

    --
    Keith Thompson (The_Other_Keith) <http://www.ghoti.net/~kst>
    San Diego Supercomputer Center <*> <http://users.sdsc.edu/~kst>
    We must do something. This is something. Therefore, we must do this.
     
    Keith Thompson, Oct 11, 2006
    #8
  9. Bill Reid Guest

    Keith Thompson <> wrote in message
    news:...
    > jmcgill <> writes:
    > > wrote:
    > >
    > >> this is a very old (and common) idea used by a lot of game programmers
    > >> from yesteryears.

    > >
    > > I remember a lot of yesteryears' games failing with bizarre memory
    > > problems, especially when running them on later OS versions, with
    > > different memory managers, etc.
    > >
    > > If you want an incomplete type, just put a void * in your struct and
    > > allocate that properly. Would that kill you?

    >
    > The struct hack has the advantage that the structure and the array can
    > be allocated contiguously.


    Of course, you can allocate a "data" pointer contiguously in a struct
    in any event, right?

    typedef struct {
    unsigned data_type;
    unsigned data_size;
    unsigned *data;
    } contiguous_data_struct;

    contiguous_data_struct *my_contiguous_data_struct;

    void create_struct(unsigned data_type,unsigned data_size) {
    my_contiguous_data_struct=
    malloc(sizeof(contiguous_data_struct)+(data_size*sizeof(unsigned));
    my_contiguous_data_struct->data_type=data_type;
    my_contiguous_data_struct->data_size=data_size;
    my_contiguous_data_struct->data=
    my_contiguous_data_struct+sizeof(contiguous_data_struct);
    }

    And now you may write the data to my_contiguous_data_struct->data(++).

    Of course, you wouldn't want to do this for certain types of struct
    uses, like an array of structs that you intend to search for data of a
    certain type quickly using pointer arithmetic. However, it does allow
    you to free the entire schmear using one free().

    Right?

    ---
    William Ernest Reid
     
    Bill Reid, Oct 11, 2006
    #9
  10. Chris Torek Guest

    >Keith Thompson <> wrote in message
    >news:...
    >> The struct hack has the advantage that the structure and the array can
    >> be allocated contiguously.


    In article <zY6Xg.51911$>
    Bill Reid <> wrote:
    >Of course, you can allocate a "data" pointer contiguously in a struct
    >in any event, right?


    Yes; but one crucial fix (and one style nit):

    >typedef struct {
    > unsigned data_type;
    > unsigned data_size;
    > unsigned *data;
    > } contiguous_data_struct;
    >
    >contiguous_data_struct *my_contiguous_data_struct;
    >
    >void create_struct(unsigned data_type,unsigned data_size) {
    > my_contiguous_data_struct=
    > malloc(sizeof(contiguous_data_struct)+(data_size*sizeof(unsigned));


    It is "better style" (at least according to those who think it is
    better :) ) to write this as:

    my_contiguous_data_struct =
    malloc(sizeof *my_contiguous_data_struct +
    data_size * sizeof *my_contiguous_data_struct->data);

    That is, apply sizeof to the object(s) being allocated, rather
    than their types. That way, if you make a small change to the
    data types -- e.g., if you decide to save space by using
    "unsigned short" instead of "unsigned int" for the data --
    the code automatically follows the change.

    > my_contiguous_data_struct->data_type=data_type;
    > my_contiguous_data_struct->data_size=data_size;
    > my_contiguous_data_struct->data=
    > my_contiguous_data_struct+sizeof(contiguous_data_struct);


    This last line needs to add 1, not "sizeof(contiguous_data_struct)"
    (nor sizeof *my_contiguous_data_struct). The result has type
    "pointer to contiguous_data_struct" -- the same as the type of
    the left-hand operand -- so a cast is also required (to
    "unsigned *", or perhaps "void *").

    Note that, if the compiler is not able to "see" that the
    "data" pointer points just past the structure itself, on most
    machines you will get some extra (and slightly slower) code
    to access the data elements. That is, without the "struct hack":

    my_contiguous_data_struct->data

    tends to compile into:

    - obtain value of my_contiguous_data_struct pointer
    - use that value to obtain pointer value my_contiguous_data_struct->data
    - obtain value of i, scale if needed, and add to previous
    - follow this last pointer to the data

    or in machine-code terms:

    # assumes "i" is in register r2 at this point
    load r1, my_contiguous_data_struct # first step above
    load r1, 12(r1) # 2nd step, assuming 4-byte "int"
    sll r3, r2, 2 # r3 = r2 * 4 (scale)
    add r3, r1, r3 # r3 = &...->data
    # now r3 points to my_contiguous_data_struct->data

    load r4, (r3) # (if we want to read it)

    Using the "struct hack" (or C99's flexible members), on the other
    hand, we get to skip one "load" step, perhaps at the cost of an
    extra "add", and the machine code becomes more like:

    load r1, my_contiguous_data_struct
    sll r3, r2, 2
    add r3, r1, r3
    # now r3 points 12 bytes below my_contiguous_data_struct

    load r4, 12(r3)

    (this assumes there is a "constant(reg)" offset addressing mode,
    but no reg(reg) mode; many machines have both; some even offer
    scaling for one of the "reg"s, which can eliminate the shift
    instruction).
    --
    In-Real-Life: Chris Torek, Wind River Systems
    Salt Lake City, UT, USA (40°39.22'N, 111°50.29'W) +1 801 277 2603
    email: forget about it http://web.torek.net/torek/index.html
    Reading email is like searching for food in the garbage, thanks to spammers.
     
    Chris Torek, Oct 11, 2006
    #10
  11. "Bill Reid" <> writes:
    > Keith Thompson <> wrote in message
    > news:...
    >> jmcgill <> writes:
    >> > wrote:
    >> >
    >> >> this is a very old (and common) idea used by a lot of game programmers
    >> >> from yesteryears.
    >> >
    >> > I remember a lot of yesteryears' games failing with bizarre memory
    >> > problems, especially when running them on later OS versions, with
    >> > different memory managers, etc.
    >> >
    >> > If you want an incomplete type, just put a void * in your struct and
    >> > allocate that properly. Would that kill you?

    >>
    >> The struct hack has the advantage that the structure and the array can
    >> be allocated contiguously.

    >
    > Of course, you can allocate a "data" pointer contiguously in a struct
    > in any event, right?
    >
    > typedef struct {
    > unsigned data_type;
    > unsigned data_size;
    > unsigned *data;
    > } contiguous_data_struct;
    >
    > contiguous_data_struct *my_contiguous_data_struct;
    >
    > void create_struct(unsigned data_type,unsigned data_size) {
    > my_contiguous_data_struct=
    > malloc(sizeof(contiguous_data_struct)+(data_size*sizeof(unsigned));
    > my_contiguous_data_struct->data_type=data_type;
    > my_contiguous_data_struct->data_size=data_size;
    > my_contiguous_data_struct->data=
    > my_contiguous_data_struct+sizeof(contiguous_data_struct);
    > }
    >
    > And now you may write the data to my_contiguous_data_struct->data(++).

    [...]

    I see two problems with this.

    First, there's no guarantee that
    my_contiguous_data_struct+sizeof(contiguous_data_struct)
    is properly aligned.

    Second, if you make a copy of the structure (even if you properly
    allow for the additional size), the data member will still point to
    the old copy.

    The struct hack (or in C99, a flexible array member) avoids both these
    problems.

    --
    Keith Thompson (The_Other_Keith) <http://www.ghoti.net/~kst>
    San Diego Supercomputer Center <*> <http://users.sdsc.edu/~kst>
    We must do something. This is something. Therefore, we must do this.
     
    Keith Thompson, Oct 11, 2006
    #11
  12. wrote:
    > ...
    > this is a question which i saw in a book
    >
    > typedef struct mall_li_header_ {
    > int refcnt;
    > uchar pool;
    > uchar flag;
    > ushort magic_no;
    > char data[0];
    >
    > } mall_li_header_t;
    >
    > What is the use of data[0] here ?
    > to which answer i found from my colleagues as
    >
    > this is a very old (and common) idea used by a lot of game programmers
    > from yesteryears.
    > The answer for completeness sake - common use is : allocate arbitrary
    > size - anything above the size of the struct can be referenced as
    > "data".
    >
    > to what extent this is true?????
    > ...


    This is exactly true with one exception: in C89/90 the proper use of this idiom
    requires an array of size _greater_ _than_ _zero_ as the last member of the
    structure. Zero sized arrays (as the one in your quote) are illegal in C.
    Normally, an array of size 1 is used in C89/90.

    Provided the structure is declared as above but with the 'data' array of any
    non-zero size, the total amount of memory needed for a structure with a trailing
    'data' array of size 'n' can be calculated as follows

    size_t size;
    mall_li_header_t* p;
    ...
    size = offsetof(mall_li_header_t, data) + sizeof(*p->data) * n;

    or as

    size = sizeof(*p) - sizeof(p->data) + sizeof(*p->data) * n;

    Note, that since non-zero size of 'data' field is included into the total size
    of 'mall_li_header_t', we have to remember to somehow exclude the former from
    the latter before adding the actual size of the array. This can be done either
    by explicit subtraction (as in the second example) or by using the 'offsetof'
    macro (as in the first one). Unfortunately, some programmers are too lazy to do
    that and instead prefer to rely on non-standard extension provided by some
    compilers: their support for zero-sized arrays as struct members.

    Note also, that this idiom is so widely used and recognized that the latest C
    standard - C99 - added support for this specific idiom into the language. Now
    one can completely omit the array size in the declaration of the last member of
    the structure. A 'sizeof' applied to the structure will act as if the array
    member contributes 0 to the total size of the structure. This makes sense and
    makes things a bit easier, of course.

    --
    Best regards,
    Andrey Tarasevich
     
    Andrey Tarasevich, Oct 11, 2006
    #12
  13. jmcgill wrote:
    > ....
    > If you want an incomplete type, just put a void * in your struct and
    > allocate that properly. Would that kill you?
    > ...


    It wouldn't. But the problem is that would be a rather ugly solution to
    something that can be implemented by a good old, well-known and elegant idiom.
    Which now happens to be supported by the language itself. One just needs to
    learn to use it properly.

    --
    Best regards,
    Andrey Tarasevich
     
    Andrey Tarasevich, Oct 11, 2006
    #13
  14. Bill Reid Guest

    Chris Torek <> wrote in message
    news:...
    > >Keith Thompson <> wrote in message
    > >news:...

    > In article <zY6Xg.51911$>
    > Bill Reid <> wrote:


    > >> The struct hack has the advantage that the structure and the array can
    > >> be allocated contiguously.


    > >Of course, you can allocate a "data" pointer contiguously in a struct
    > >in any event, right?

    >
    > Yes; but one crucial fix (and one style nit):
    >
    > >typedef struct {
    > > unsigned data_type;
    > > unsigned data_size;
    > > unsigned *data;
    > > } contiguous_data_struct;
    > >
    > >contiguous_data_struct *my_contiguous_data_struct;
    > >
    > >void create_struct(unsigned data_type,unsigned data_size) {
    > > my_contiguous_data_struct=
    > > malloc(sizeof(contiguous_data_struct)+(data_size*sizeof(unsigned));

    >
    > It is "better style" (at least according to those who think it is
    > better :) ) to write this as:
    >
    > my_contiguous_data_struct =
    > malloc(sizeof *my_contiguous_data_struct +
    > data_size * sizeof *my_contiguous_data_struct->data);
    >
    > That is, apply sizeof to the object(s) being allocated, rather
    > than their types. That way, if you make a small change to the
    > data types -- e.g., if you decide to save space by using
    > "unsigned short" instead of "unsigned int" for the data --
    > the code automatically follows the change.
    >

    Yeah, I think how likely you are to make a mistake depends on how
    "separated" the type declaration and the taking the sizeof are. But in
    any event a mistake is possible so yeah, it's better to do it "your" way...

    > > my_contiguous_data_struct->data_type=data_type;
    > > my_contiguous_data_struct->data_size=data_size;
    > > my_contiguous_data_struct->data=
    > > my_contiguous_data_struct+sizeof(contiguous_data_struct);

    >
    > This last line needs to add 1, not "sizeof(contiguous_data_struct)"
    > (nor sizeof *my_contiguous_data_struct). The result has type
    > "pointer to contiguous_data_struct" -- the same as the type of
    > the left-hand operand -- so a cast is also required (to
    > "unsigned *", or perhaps "void *").
    >

    Uh, yeah, I was doing this from vague memory, so you're absolutely
    correct. Here's the real deal (from several separate libraries) that I know
    works but I'm not sure right now I actually use for anything (basically
    opens a "smart" text file buffer):

    typedef struct {
    unsigned type;
    unsigned long size;
    unsigned cols;
    unsigned rows;
    char *buffer;
    } CSV_EFILEB;

    char *csv_file_path="testcsv.txt";
    CSV_EFILEB *csv_efileb=NULL;

    /* allocates a structure with a contiguous data buffer */
    unsigned alloc_data_struct(void **ptr_struct_ptr,
    unsigned struct_size,unsigned long data_size) {
    void *struct_ptr=ptr_struct_ptr;

    if((struct_ptr=realloc(struct_ptr,struct_size+data_size))==NULL) {
    printf("\nCouldn't allocate structure buffer");
    return FALSE;
    }
    else *ptr_struct_ptr=struct_ptr;

    return TRUE;
    }

    unsigned open_csv_efileb(char *csv_file_path,CSV_EFILEB **csv_efileb) {
    unsigned struct_size=sizeof(CSV_EFILEB);
    unsigned long data_size;
    unsigned func_return=FALSE;

    if((csv_file=fopen(csv_file_path,"rt"))==NULL) {
    printf("\nCan't open CSV file: \n%s",csv_file_path);
    return func_return;
    }

    if((data_size=get_text_file_size(csv_file))==0) {
    printf("\nApparently, there's no data in the file");
    goto CloseFile;
    }

    if(!(alloc_data_struct((void **)csv_efileb,struct_size,data_size)))
    goto CloseFile;

    (*csv_efileb)->size=data_size;

    (*csv_efileb)->buffer=(void *)(*csv_efileb+1);

    if((fread((*csv_efileb)->buffer,data_size,1,csv_file))!=1) {
    printf("\nFailed to read file to buffer");
    goto CloseFile;
    }
    else func_return=TRUE;

    CloseFile :
    if((fclose(csv_file))!=SUCCESS) {
    printf("\nCan't close CSV file: \n%s",csv_file_path);
    return FALSE;
    }

    return TRUE;
    }

    open_csv_efileb(csv_file_path,&csv_efileb);

    > Note that, if the compiler is not able to "see" that the
    > "data" pointer points just past the structure itself, on most
    > machines you will get some extra (and slightly slower) code
    > to access the data elements. That is, without the "struct hack":
    >

    So, bottom line, is it entirely cool to use the "struct hack"
    (keeping in mind I don't believe my compiler allows zero-length
    arrays in structs)? Should I just read the FAQ on the issue?

    > my_contiguous_data_struct->data
    >
    > tends to compile into:
    >
    > - obtain value of my_contiguous_data_struct pointer
    > - use that value to obtain pointer value my_contiguous_data_struct->data
    > - obtain value of i, scale if needed, and add to previous
    > - follow this last pointer to the data
    >
    > or in machine-code terms:
    >
    > # assumes "i" is in register r2 at this point
    > load r1, my_contiguous_data_struct # first step above
    > load r1, 12(r1) # 2nd step, assuming 4-byte "int"
    > sll r3, r2, 2 # r3 = r2 * 4 (scale)
    > add r3, r1, r3 # r3 = &...->data
    > # now r3 points to my_contiguous_data_struct->data
    >
    > load r4, (r3) # (if we want to read it)
    >
    > Using the "struct hack" (or C99's flexible members), on the other
    > hand, we get to skip one "load" step, perhaps at the cost of an
    > extra "add", and the machine code becomes more like:
    >
    > load r1, my_contiguous_data_struct
    > sll r3, r2, 2
    > add r3, r1, r3
    > # now r3 points 12 bytes below my_contiguous_data_struct
    >
    > load r4, 12(r3)
    >
    > (this assumes there is a "constant(reg)" offset addressing mode,
    > but no reg(reg) mode; many machines have both; some even offer
    > scaling for one of the "reg"s, which can eliminate the shift
    > instruction).
    >

    Sure, but this is all true of any pointer to data in a struct, right?
    I mean, how often does that happen? I was only commenting
    on the VERY narrow issue of "contiguous" allocation...

    ---
    William Ernest Reid
     
    Bill Reid, Oct 12, 2006
    #14
  15. Bill Reid Guest

    Keith Thompson <> wrote in message
    news:...
    > "Bill Reid" <> writes:
    > > Keith Thompson <> wrote in message
    > > news:...
    > >> jmcgill <> writes:
    > >> > wrote:
    > >> >
    > >> >> this is a very old (and common) idea used by a lot of game

    programmers
    > >> >> from yesteryears.
    > >> >
    > >> > I remember a lot of yesteryears' games failing with bizarre memory
    > >> > problems, especially when running them on later OS versions, with
    > >> > different memory managers, etc.
    > >> >
    > >> > If you want an incomplete type, just put a void * in your struct and
    > >> > allocate that properly. Would that kill you?
    > >>
    > >> The struct hack has the advantage that the structure and the array can
    > >> be allocated contiguously.

    > >
    > > Of course, you can allocate a "data" pointer contiguously in a struct
    > > in any event, right?
    > >
    > > typedef struct {
    > > unsigned data_type;
    > > unsigned data_size;
    > > unsigned *data;
    > > } contiguous_data_struct;
    > >
    > > contiguous_data_struct *my_contiguous_data_struct;
    > >
    > > void create_struct(unsigned data_type,unsigned data_size) {
    > > my_contiguous_data_struct=
    > > malloc(sizeof(contiguous_data_struct)+(data_size*sizeof(unsigned));
    > > my_contiguous_data_struct->data_type=data_type;
    > > my_contiguous_data_struct->data_size=data_size;
    > > my_contiguous_data_struct->data=
    > > my_contiguous_data_struct+sizeof(contiguous_data_struct);
    > > }
    > >
    > > And now you may write the data to my_contiguous_data_struct->data(++).

    > [...]
    >
    > I see two problems with this.
    >

    Actually, as it turns out, there were at least a couple more, but
    who's counting?

    > First, there's no guarantee that
    > my_contiguous_data_struct+sizeof(contiguous_data_struct)


    s/be (void *)my_contiguous_data_struct+1

    > is properly aligned.
    >

    Now this confuses me...when you say "no guarantee" that the
    struct memory (?) is "properly aligned", what specifically are you
    talking about? The struct itself is aligned (within itself!), so doesn't
    that just leave any possible "alignment" of (in this case) a fundamental
    type? Is this just a pathological theoretical possibility, or something
    that could really happen?

    > Second, if you make a copy of the structure (even if you properly
    > allow for the additional size), the data member will still point to
    > the old copy.
    >

    Exactly, like memcpy() is a problem...but I openly stated that there
    were "caveats" concerning this "scheme"...I was just pointing out that
    you can "solve" the issue of "contiguous" memory and multiple free()s
    using it, that's all...

    > The struct hack (or in C99, a flexible array member) avoids both these
    > problems.
    >

    And if I don't have a C99 compiler, then that "hack" is just fine in all
    cases to use?

    ---
    William Ernest Reid
     
    Bill Reid, Oct 12, 2006
    #15
  16. jmcgill Guest

    Keith Thompson wrote:

    > Second, if you make a copy of the structure (even if you properly
    > allow for the additional size), the data member will still point to
    > the old copy.


    Take deep-copy semantics into account early in your design.
     
    jmcgill, Oct 12, 2006
    #16
  17. "Bill Reid" <> writes:
    > Keith Thompson <> wrote in message
    > news:...
    >> "Bill Reid" <> writes:

    [...]
    >> > Of course, you can allocate a "data" pointer contiguously in a struct
    >> > in any event, right?
    >> >
    >> > typedef struct {
    >> > unsigned data_type;
    >> > unsigned data_size;
    >> > unsigned *data;
    >> > } contiguous_data_struct;
    >> >
    >> > contiguous_data_struct *my_contiguous_data_struct;
    >> >
    >> > void create_struct(unsigned data_type,unsigned data_size) {
    >> > my_contiguous_data_struct=
    >> > malloc(sizeof(contiguous_data_struct)+(data_size*sizeof(unsigned));
    >> > my_contiguous_data_struct->data_type=data_type;
    >> > my_contiguous_data_struct->data_size=data_size;
    >> > my_contiguous_data_struct->data=
    >> > my_contiguous_data_struct+sizeof(contiguous_data_struct);
    >> > }
    >> >
    >> > And now you may write the data to my_contiguous_data_struct->data(++).

    >> [...]
    >>
    >> I see two problems with this.
    >>

    > Actually, as it turns out, there were at least a couple more, but
    > who's counting?
    >
    >> First, there's no guarantee that
    >> my_contiguous_data_struct+sizeof(contiguous_data_struct)

    >
    > s/be (void *)my_contiguous_data_struct+1
    >
    >> is properly aligned.


    It took me a while to realize that "s/be" meant "should be". It's
    worth the effort to use whole words.

    And it should really be (void*)(my_contiguous_data_struct+1);
    otherwise you're adding 1 to a value of type void*, which is illegal
    (but gcc will accept it with no warning by default -- another reason
    why that extension is a bad idea).

    > Now this confuses me...when you say "no guarantee" that the
    > struct memory (?) is "properly aligned", what specifically are you
    > talking about? The struct itself is aligned (within itself!), so doesn't
    > that just leave any possible "alignment" of (in this case) a fundamental
    > type? Is this just a pathological theoretical possibility, or something
    > that could really happen?


    It could really happen. I'll construct an example similar to what you
    wrote above:

    #include <stdlib.h>

    struct contiguous_data {
    unsigned data_type;
    unsigned data_size;
    double *data;
    };

    struct contiguous_data *create_struct(unsigned data_type, unsigned data_size)
    {
    struct contiguous_data *result
    = malloc(sizeof(struct contiguous_data) +
    data_size * sizeof(double));
    if (result == NULL) {
    return NULL;
    }
    result->data_type = data_type;
    result->data_size = data_size;
    result->data = (double*)(result + 1);
    return result;
    }

    Suppose types unsigned and (double*) are 4 bytes, requiring 4-byte
    alignment, and double is 8 bytes, requiring 8-byte alignment.
    Assuming struct contiguous_data has no gaps, its size is 12 bytes;
    let's say it requires 4-byte alignment. And suppose we call
    create_struct() with data_size == 2.

    Then create_struct() will malloc() 12 + 2*8 bytes, or 28 bytes. The
    base address of the malloc()ed block is guaranteed to be properly
    aligned for any type. We treat the first 12 bytes as a struct
    contiguous_data object, which is fine. We then treat the last 16
    bytes, starting at offset 12, as an array of 2 doubles -- but since 12
    is not a multiple of 8, it's not properly aligned to hold doubles.

    Misalignment might be less likely in your original example, but
    it's still possible.

    If you used the "struct hack", you'd declare:

    struct contiguous_data {
    unsigned data_type;
    unsigned data_size;
    double data[1];
    };

    (or "double data[];" if you use a C99 flexible array member). The
    compiler knows the required alignment of type double, so it inserts
    whatever padding is necessary. (We dropped the pointer, so it happens
    to be aligned anyway, but we could easily have an example where
    padding is necessary.)

    In your example, you placed the follow-on data manually without
    allowing for alignment issues. The compiler didn't have a chance to
    align it properly.

    >> Second, if you make a copy of the structure (even if you properly
    >> allow for the additional size), the data member will still point to
    >> the old copy.
    >>

    > Exactly, like memcpy() is a problem...but I openly stated that there
    > were "caveats" concerning this "scheme"...I was just pointing out that
    > you can "solve" the issue of "contiguous" memory and multiple free()s
    > using it, that's all...


    Sure, but the struct hack is more convenient, even if it's of somewhat
    questionable validity.

    >> The struct hack (or in C99, a flexible array member) avoids both these
    >> problems.
    >>

    > And if I don't have a C99 compiler, then that "hack" is just fine in all
    > cases to use?


    Maybe. Probably.

    Question 2.6 in the FAQ says:

    Despite its popularity, the technique is also somewhat notorious:
    Dennis Ritchie has called it ``unwarranted chumminess with the C
    implementation,'' and an official interpretation has deemed that
    it is not strictly conforming with the C Standard, although it
    does seem to work under all known implementations. (Compilers
    which check array bounds carefully might issue warnings.)

    If you don't trust the struct hack, you can always just allocate the
    data separately:

    #include <stdlib.h>

    struct contiguous_data {
    unsigned data_type;
    unsigned data_size;
    double *data;
    };

    struct contiguous_data *create_struct(unsigned data_type, unsigned data_size)
    {
    struct contiguous_data *result = malloc(sizeof *result);
    if (result == NULL) {
    return NULL;
    }
    result->data_type = data_type;
    result->data_size = data_size;
    result->data = malloc(data_size * sizeof(double));
    if (result->data == NULL) {
    free(result);
    return NULL;
    }
    return result;
    }

    This will require two calls to free() to deallocate the allocated
    memory.

    --
    Keith Thompson (The_Other_Keith) <http://www.ghoti.net/~kst>
    San Diego Supercomputer Center <*> <http://users.sdsc.edu/~kst>
    We must do something. This is something. Therefore, we must do this.
     
    Keith Thompson, Oct 12, 2006
    #17
  18. Keith Thompson wrote:
    > "Bill Reid" <> writes:
    > > Keith Thompson <> wrote in message
    > > news:...
    > >> "Bill Reid" <> writes:

    > [...]
    > >> > Of course, you can allocate a "data" pointer contiguously in a struct
    > >> > in any event, right?
    > >> >
    > >> > typedef struct {
    > >> > unsigned data_type;
    > >> > unsigned data_size;
    > >> > unsigned *data;
    > >> > } contiguous_data_struct;
    > >> >
    > >> > contiguous_data_struct *my_contiguous_data_struct;
    > >> >
    > >> > void create_struct(unsigned data_type,unsigned data_size) {
    > >> > my_contiguous_data_struct=
    > >> > malloc(sizeof(contiguous_data_struct)+(data_size*sizeof(unsigned));
    > >> > my_contiguous_data_struct->data_type=data_type;
    > >> > my_contiguous_data_struct->data_size=data_size;
    > >> > my_contiguous_data_struct->data=
    > >> > my_contiguous_data_struct+sizeof(contiguous_data_struct);
    > >> > }
    > >> >
    > >> > And now you may write the data to my_contiguous_data_struct->data(++).
    > >> [...]
    > >>
    > >> I see two problems with this.
    > >>

    > > Actually, as it turns out, there were at least a couple more, but
    > > who's counting?
    > >
    > >> First, there's no guarantee that
    > >> my_contiguous_data_struct+sizeof(contiguous_data_struct)

    > >
    > > s/be (void *)my_contiguous_data_struct+1
    > >
    > >> is properly aligned.

    >
    > It took me a while to realize that "s/be" meant "should be". It's
    > worth the effort to use whole words.
    >
    > And it should really be (void*)(my_contiguous_data_struct+1);
    > otherwise you're adding 1 to a value of type void*, which is illegal
    > (but gcc will accept it with no warning by default -- another reason
    > why that extension is a bad idea).
    >
    > > Now this confuses me...when you say "no guarantee" that the
    > > struct memory (?) is "properly aligned", what specifically are you
    > > talking about? The struct itself is aligned (within itself!), so doesn't
    > > that just leave any possible "alignment" of (in this case) a fundamental
    > > type? Is this just a pathological theoretical possibility, or something
    > > that could really happen?

    >
    > It could really happen. I'll construct an example similar to what you
    > wrote above:
    >
    > #include <stdlib.h>
    >
    > struct contiguous_data {
    > unsigned data_type;
    > unsigned data_size;
    > double *data;
    > };
    >
    > struct contiguous_data *create_struct(unsigned data_type, unsigned data_size)
    > {
    > struct contiguous_data *result
    > = malloc(sizeof(struct contiguous_data) +
    > data_size * sizeof(double));
    > if (result == NULL) {
    > return NULL;
    > }
    > result->data_type = data_type;
    > result->data_size = data_size;
    > result->data = (double*)(result + 1);
    > return result;
    > }
    >
    > Suppose types unsigned and (double*) are 4 bytes, requiring 4-byte
    > alignment, and double is 8 bytes, requiring 8-byte alignment.
    > Assuming struct contiguous_data has no gaps, its size is 12 bytes;
    > let's say it requires 4-byte alignment. And suppose we call
    > create_struct() with data_size == 2.
    >
    > Then create_struct() will malloc() 12 + 2*8 bytes, or 28 bytes. The
    > base address of the malloc()ed block is guaranteed to be properly
    > aligned for any type. We treat the first 12 bytes as a struct
    > contiguous_data object, which is fine. We then treat the last 16
    > bytes, starting at offset 12, as an array of 2 doubles -- but since 12
    > is not a multiple of 8, it's not properly aligned to hold doubles.
    >
    > Misalignment might be less likely in your original example, but
    > it's still possible.


    All structs are required to have the same alignment, right? Since the
    initial member of any struct can be of any type, isn't it then required
    that any struct is aligned properly for any type? And if that is, isn't
    one past the end of any struct required to be aligned properly for any
    type as well, in order for arrays of structs to work?

    (I know I'm probably overlooking something, but I don't know what.)
     
    =?utf-8?B?SGFyYWxkIHZhbiBExLNr?=, Oct 12, 2006
    #18
  19. "Harald van Dijk" <> writes:
    [...]
    > All structs are required to have the same alignment, right?


    No. The standard does say that

    All pointers to structure types shall have the same representation
    and alignment requirements as each other.

    but that refers to the alignment of pointer objects, not to the
    alignment of structure objects.

    For example, this type:

    struct foo {
    char c;
    };

    can reasonably have a size and alignment of 1 byte.

    --
    Keith Thompson (The_Other_Keith) <http://www.ghoti.net/~kst>
    San Diego Supercomputer Center <*> <http://users.sdsc.edu/~kst>
    We must do something. This is something. Therefore, we must do this.
     
    Keith Thompson, Oct 12, 2006
    #19
  20. Harald van Dijk wrote:
    > ...
    > All structs are required to have the same alignment, right? Since the
    > initial member of any struct can be of any type, isn't it then required
    > that any struct is aligned properly for any type?


    No. Why? The only thing that is required is that each given struct is
    properly aligned for its own and only its own members. The alignment
    requirement of the struct itself is indeed determined by its first
    member, since other members can be aligned independently by introducing
    internal padding.

    On a related note, the trailing padding (the one that follows the last
    member) is introduced in order achieve proper alignment of struct
    elements in an array and, therefore, can be thought of as dependent on
    all members of the structure. This means that if you want to use this
    technique to store an array of 'int's immediately after the struct in
    memory, you should be OK if the struct itself contains at least one
    'int' member (or a member with a stronger alignment requirement).
    Otherwise, there's no guarantee.

    --
    Best regards,
    Andrey Tarasevich
     
    Andrey Tarasevich, Oct 12, 2006
    #20
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. wwj
    Replies:
    7
    Views:
    558
  2. wwj
    Replies:
    24
    Views:
    2,520
    Mike Wahler
    Nov 7, 2003
  3. Ben Pfaff
    Replies:
    5
    Views:
    480
    Tristan Miller
    Jan 17, 2004
  4. Steffen Fiksdal

    void*, char*, unsigned char*, signed char*

    Steffen Fiksdal, May 8, 2005, in forum: C Programming
    Replies:
    1
    Views:
    589
    Jack Klein
    May 9, 2005
  5. lovecreatesbeauty
    Replies:
    1
    Views:
    1,061
    Ian Collins
    May 9, 2006
Loading...

Share This Page