dynamic buffer allocation at char buf[1]

Discussion in 'C Programming' started by Kok How Teh, Apr 10, 2010.

  1. Kok How Teh

    Kok How Teh Guest

    Hi;
    Is it possible to allocate 20-byte buffer at buf in the following
    structure?

    struct {
    int len;
    char buf[1];
    };
    If yes, how? Thanks.

    Regards.
     
    Kok How Teh, Apr 10, 2010
    #1
    1. Advertising

  2. Kok How Teh wrote:
    > Hi;
    > Is it possible to allocate 20-byte buffer at buf in the following
    > structure?
    >
    > struct {
    > int len;
    > char buf[1];
    > };
    > If yes, how? Thanks.


    No, at least not without breaking the abstraction that the C language
    provides. Anyway, why would you want to do that? And what is the purpose
    of the the field len?


    August
     
    August Karlstrom, Apr 10, 2010
    #2
    1. Advertising

  3. August Karlstrom <> writes:

    > Kok How Teh wrote:
    >> Hi;
    >> Is it possible to allocate 20-byte buffer at buf in the following
    >> structure?
    >>
    >> struct {
    >> int len;
    >> char buf[1];
    >> };
    >> If yes, how? Thanks.

    >
    > No, at least not without breaking the abstraction that the C language
    > provides. Anyway, why would you want to do that?


    Are you pointing out that the technique often called the "struct hack"
    involves, technically, undefined behaviour? If so, you should probably
    have been more explicit about that simply because it is such a commonly
    used device despite being undefined.

    So common, in fact, that there is an official C99 version:

    struct buffer {
    size_t len;
    char buf[];
    };

    were buf is called a "flexible array member". There can be only one and
    it must be the last member and there must be at least one other member.

    > And what is the purpose of the the field len?


    len will presumably be used to record the number of elements that can be
    stored and will be set after using malloc to allocate more than
    sizeof(struct buffer) bytes.

    --
    Ben.
     
    Ben Bacarisse, Apr 10, 2010
    #3
  4. Ben Bacarisse wrote:
    [...]
    > So common, in fact, that there is an official C99 version:
    >
    > struct buffer {
    > size_t len;
    > char buf[];
    > };
    >
    > were buf is called a "flexible array member". There can be only one and
    > it must be the last member and there must be at least one other member.


    Oh my, that's ugly. And some say C is a simple language... well I have
    my doubts.


    August
     
    August Karlstrom, Apr 11, 2010
    #4
  5. Kok How Teh

    Ian Collins Guest

    On 04/11/10 12:06 PM, August Karlstrom wrote:
    > Ben Bacarisse wrote:
    > [...]
    >> So common, in fact, that there is an official C99 version:
    >>
    >> struct buffer {
    >> size_t len;
    >> char buf[];
    >> };
    >>
    >> were buf is called a "flexible array member". There can be only one and
    >> it must be the last member and there must be at least one other member.

    >
    > Oh my, that's ugly. And some say C is a simple language... well I have
    > my doubts.


    Unless you can suggest a cleaner way to have variable length field in a
    struct, we're stuck with it. I've yet to come across a pretty hack!

    --
    Ian Collins
     
    Ian Collins, Apr 11, 2010
    #5
  6. Ian Collins wrote:
    > On 04/11/10 12:06 PM, August Karlstrom wrote:
    >> Ben Bacarisse wrote:
    >> [...]
    >>> So common, in fact, that there is an official C99 version:
    >>>
    >>> struct buffer {
    >>> size_t len;
    >>> char buf[];
    >>> };
    >>>
    >>> were buf is called a "flexible array member". There can be only one and
    >>> it must be the last member and there must be at least one other member.

    >>
    >> Oh my, that's ugly. And some say C is a simple language... well I have
    >> my doubts.

    >
    > Unless you can suggest a cleaner way to have variable length field in a
    > struct, we're stuck with it. I've yet to come across a pretty hack!
    >


    I don't quite get the advantage of the struct hack compared to just
    having a pointer field like in

    struct buffer {
    size_t len;
    char *buf;
    };

    Is it to avoid an extra call to malloc?


    August
     
    August Karlstrom, Apr 11, 2010
    #6
  7. Kok How Teh

    Willem Guest

    August Karlstrom wrote:
    ) I don't quite get the advantage of the struct hack compared to just
    ) having a pointer field like in
    )
    ) struct buffer {
    ) size_t len;
    ) char *buf;
    ) };
    )
    ) Is it to avoid an extra call to malloc?

    - It also avoids an extra call to free()
    - It avoids the extra allocated space for the pointer
    - It keeps the items together in memory which improves locality of
    reference (which is good for performance because of caching)
    - A lot of file formats consist of blocks with a fixed-size header,
    followed by a variable-size bit of data


    SaSW, Willem
    --
    Disclaimer: I am in no way responsible for any of the statements
    made in the above text. For all I know I might be
    drugged or something..
    No I'm not paranoid. You all think I'm paranoid, don't you !
    #EOT
     
    Willem, Apr 11, 2010
    #7
  8. Willem <> writes:

    > August Karlstrom wrote:
    > ) I don't quite get the advantage of the struct hack compared to just
    > ) having a pointer field like in
    > )
    > ) struct buffer {
    > ) size_t len;
    > ) char *buf;
    > ) };
    > )
    > ) Is it to avoid an extra call to malloc?
    >
    > - It also avoids an extra call to free()
    > - It avoids the extra allocated space for the pointer
    > - It keeps the items together in memory which improves locality of
    > reference (which is good for performance because of caching)
    > - A lot of file formats consist of blocks with a fixed-size header,
    > followed by a variable-size bit of data


    It's probably worth saying that in this specific instance (where the
    buffer elements are characters) one can combine some of the
    advantages of both (single allocation/free and some degree of locality)
    by doing this:

    struct buffer *bp = malloc(sizeof *bp + BUF_LENGTH);
    if (bp) {
    bp->len = BUF_LENGTH;
    bp->buf = (char *)bp + sizeof *bp;
    }

    Space is wasted and it it of no use if one is trying to match a file
    format, but it is well-defined where C99's solution is not available.

    Even so, I am not sure I'd bother.

    --
    Ben.
     
    Ben Bacarisse, Apr 11, 2010
    #8
  9. Kok How Teh

    Eric Sosman Guest

    On 4/11/2010 9:34 AM, Willem wrote:
    > August Karlstrom wrote:
    > ) I don't quite get the advantage of the struct hack compared to just
    > ) having a pointer field like in
    > )
    > ) struct buffer {
    > ) size_t len;
    > ) char *buf;
    > ) };
    > )
    > ) Is it to avoid an extra call to malloc?
    >
    > - It also avoids an extra call to free()
    > - It avoids the extra allocated space for the pointer
    > - It keeps the items together in memory which improves locality of
    > reference (which is good for performance because of caching)
    > - A lot of file formats consist of blocks with a fixed-size header,
    > followed by a variable-size bit of data


    For that last, a struct of any kind is risky. Padding,
    you know. (Also byte order and other such representational
    issues, but they're not unique to structs.)

    --
    Eric Sosman
    lid
     
    Eric Sosman, Apr 11, 2010
    #9
  10. Kok How Teh

    Willem Guest

    Eric Sosman wrote:
    ) On 4/11/2010 9:34 AM, Willem wrote:
    )> - A lot of file formats consist of blocks with a fixed-size header,
    )> followed by a variable-size bit of data
    )
    ) For that last, a struct of any kind is risky. Padding,
    ) you know. (Also byte order and other such representational
    ) issues, but they're not unique to structs.)

    I know. That's why a lot of compilers have extensions to force a
    struct to not be padded. Should have been added to the language IMO.

    Along with multi-character character initializers.
    For example: unsigned long exif_tag = 'EXIF';

    Unfortunately, that was never standardized. A pity.


    SaSW, Willem
    --
    Disclaimer: I am in no way responsible for any of the statements
    made in the above text. For all I know I might be
    drugged or something..
    No I'm not paranoid. You all think I'm paranoid, don't you !
    #EOT
     
    Willem, Apr 11, 2010
    #10
  11. On Sun, 11 Apr 2010, Willem wrote:

    > Along with multi-character character initializers.
    > For example: unsigned long exif_tag = 'EXIF';
    >
    > Unfortunately, that was never standardized. A pity.


    { little endian, big endian, ... } x { ascii, ebcdic, ... }

    OTOH, it is standardized to some degree:

    C89 6.1.3.4 Character constants

    ----v----
    [...] The value of an integer character constant containing more than one
    character, or containing a character or escape sequence not represented in
    the basic execution character set, is implementation-defined. [...]
    ----^----

    C99 6.4.4.4 Character constants p10

    ----v----
    The value of an integer character constant containing more than one
    character (e.g., 'ab'), or containing a character or escape sequence that
    does not map to a single-byte execution character, is
    implementation-defined.
    ----^----

    Cheers,
    lacos
     
    Ersek, Laszlo, Apr 11, 2010
    #11
  12. Kok How Teh

    Moi Guest

    On Sun, 11 Apr 2010 15:33:58 +0100, Ben Bacarisse wrote:

    > Willem <> writes:
    >
    >
    > It's probably worth saying that in this specific instance (where the
    > buffer elements are characters) one can combine some of the advantages
    > of both (single allocation/free and some degree of locality) by doing
    > this:
    >
    > struct buffer *bp = malloc(sizeof *bp + BUF_LENGTH); if (bp) {
    > bp->len = BUF_LENGTH;
    > bp->buf = (char *)bp + sizeof *bp;
    > }
    >


    Personally I would prefer:
    bp->buf = (char *) (bp+1);

    or maybe even...
    bp->buf = (char*) (&bp[1]);


    > Even so, I am not sure I'd bother.


    Me neither. I use the struct hack wherever I go.
    But that is not allowed here.

    AvK
     
    Moi, Apr 11, 2010
    #12
  13. Kok How Teh

    Eric Sosman Guest

    On 4/11/2010 11:54 AM, Willem wrote:
    > Eric Sosman wrote:
    > ) On 4/11/2010 9:34 AM, Willem wrote:
    > )> - A lot of file formats consist of blocks with a fixed-size header,
    > )> followed by a variable-size bit of data
    > )
    > ) For that last, a struct of any kind is risky. Padding,
    > ) you know. (Also byte order and other such representational
    > ) issues, but they're not unique to structs.)
    >
    > I know. That's why a lot of compilers have extensions to force a
    > struct to not be padded. Should have been added to the language IMO.


    To provide a still-inadequate non-solution for the
    benefit of lazy programmers? Great idea: Submit a proposal
    to the C1x Committee ...

    > Along with multi-character character initializers.
    > For example: unsigned long exif_tag = 'EXIF';
    >
    > Unfortunately, that was never standardized. A pity.


    Looks like the meat of another proposal.

    (In short: Bah!)

    --
    Eric Sosman
    lid
     
    Eric Sosman, Apr 11, 2010
    #13
  14. Kok How Teh

    Seebs Guest

    On 2010-04-11, Eric Sosman <> wrote:
    > On 4/11/2010 11:54 AM, Willem wrote:
    >> I know. That's why a lot of compilers have extensions to force a
    >> struct to not be padded. Should have been added to the language IMO.


    > To provide a still-inadequate non-solution for the
    > benefit of lazy programmers? Great idea: Submit a proposal
    > to the C1x Committee ...


    There are cases where it's useful in code which isn't expected to be
    portable to many machines, but yeah.

    >> Along with multi-character character initializers.


    >>
    >> Unfortunately, that was never standardized. A pity.


    > Looks like the meat of another proposal.


    There's a good reason not to standardize it, too, which is that its
    meaning is deeply ambiguous, and whichever answer you pick will surprise
    some people and break their code.

    -s
    --
    Copyright 2010, all wrongs reversed. Peter Seebach /
    http://www.seebs.net/log/ <-- lawsuits, religion, and funny pictures
    http://en.wikipedia.org/wiki/Fair_Game_(Scientology) <-- get educated!
     
    Seebs, Apr 11, 2010
    #14
  15. On 11 Apr, 16:54, Willem <> wrote:
    > Eric Sosman wrote:
    >
    > ) On 4/11/2010 9:34 AM, Willem wrote:
    > )> - A lot of file formats consist of blocks with a fixed-size header,
    > )>    followed by a variable-size bit of data
    > )
    > )      For that last, a struct of any kind is risky.  Padding,
    > ) you know.  (Also byte order and other such representational
    > ) issues, but they're not unique to structs.)
    >
    > I know.  That's why a lot of compilers have extensions to force a
    > struct to not be padded.  Should have been added to the language IMO.


    nooo!!

    on some architectures this becomes very inefficient. I think having
    the option tends to lead to its overuse.


    > Along with multi-character character initializers.
    > For example: unsigned long exif_tag = 'EXIF';
    >
    > Unfortunately, that was never standardized.  A pity.


    jolly good thing
     
    Nick Keighley, Apr 11, 2010
    #15
  16. Kok How Teh

    Seebs Guest

    On 2010-04-12, Michael Foukarakis <> wrote:
    > On Apr 11, 6:54 pm, Willem <> wrote:
    >> Along with multi-character character initializers.
    >> For example: unsigned long exif_tag = 'EXIF';


    > Posting that as if it were a piece of code actually encountered in the
    > wild and considered half-useful made me shed a little tear.


    It was an idiom very popular on pre-OS X macs, where it was used to
    create 32-bit values for use as "creator" and "type" tags.

    Not a big fan, myself, but it was a plausible idiom, because it allowed
    you to compare the 32-bit values rather than iterating through strings.

    -s
    --
    Copyright 2010, all wrongs reversed. Peter Seebach /
    http://www.seebs.net/log/ <-- lawsuits, religion, and funny pictures
    http://en.wikipedia.org/wiki/Fair_Game_(Scientology) <-- get educated!
     
    Seebs, Apr 12, 2010
    #16
  17. On Apr 11, 6:54 pm, Willem <> wrote:

    > Along with multi-character character initializers.
    > For example: unsigned long exif_tag = 'EXIF';


    Posting that as if it were a piece of code actually encountered in the
    wild and considered half-useful made me shed a little tear.
     
    Michael Foukarakis, Apr 12, 2010
    #17
  18. Kok How Teh

    Alan Curry Guest

    In article <>,
    Seebs <> wrote:
    |On 2010-04-12, Michael Foukarakis <> wrote:
    |> On Apr 11, 6:54 pm, Willem <> wrote:
    |>> Along with multi-character character initializers.
    |>> For example: unsigned long exif_tag = 'EXIF';
    |
    |> Posting that as if it were a piece of code actually encountered in the
    |> wild and considered half-useful made me shed a little tear.
    |
    |It was an idiom very popular on pre-OS X macs, where it was used to
    |create 32-bit values for use as "creator" and "type" tags.

    I suppose the byte order question didn't bother anyone, because everyone
    knew Macs would forever be big-endian.

    --
    Alan Curry
     
    Alan Curry, Apr 12, 2010
    #18
  19. Kok How Teh

    Willem Guest

    Alan Curry wrote:
    ) In article <>,
    ) Seebs <> wrote:
    )|On 2010-04-12, Michael Foukarakis <> wrote:
    )|> On Apr 11, 6:54?pm, Willem <> wrote:
    )|>> Along with multi-character character initializers.
    )|>> For example: unsigned long exif_tag = 'EXIF';
    )|
    )|> Posting that as if it were a piece of code actually encountered in the
    )|> wild and considered half-useful made me shed a little tear.
    )|
    )|It was an idiom very popular on pre-OS X macs, where it was used to
    )|create 32-bit values for use as "creator" and "type" tags.
    )
    ) I suppose the byte order question didn't bother anyone, because everyone
    ) knew Macs would forever be big-endian.

    For the record: I encountered it in C code for AmigaOS, for reading
    their standard image format (IFF). This was in the early nineties.


    SaSW, Willem
    --
    Disclaimer: I am in no way responsible for any of the statements
    made in the above text. For all I know I might be
    drugged or something..
    No I'm not paranoid. You all think I'm paranoid, don't you !
    #EOT
     
    Willem, Apr 12, 2010
    #19
  20. Kok How Teh

    Seebs Guest

    On 2010-04-12, Willem <> wrote:
    [in reference to:]
    > )|>> For example: unsigned long exif_tag = 'EXIF';


    > For the record: I encountered it in C code for AmigaOS, for reading
    > their standard image format (IFF). This was in the early nineties.


    Trivia point: IFF is not just an image format, but an all-purpose file
    format definition, used as well for many other types of data. It's actually
    still in use, somewhat, although the widespread existence of datums in
    excess of 4GB has reduced its appeal slightly.

    It's actually a really *good* format. I believe it was developed by
    Electronic Arts, although C= standardized on it.

    IFF is a great example of how to design a file format which is reasonably
    friendly to efficient use in C, and flexible/powerful enough to be a good
    fit for many tasks.

    -s
    --
    Copyright 2010, all wrongs reversed. Peter Seebach /
    http://www.seebs.net/log/ <-- lawsuits, religion, and funny pictures
    http://en.wikipedia.org/wiki/Fair_Game_(Scientology) <-- get educated!
     
    Seebs, Apr 12, 2010
    #20
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Replies:
    33
    Views:
    897
  2. lovecreatesbeauty
    Replies:
    1
    Views:
    1,099
    Ian Collins
    May 9, 2006
  3. Ken
    Replies:
    24
    Views:
    3,903
    Ben Bacarisse
    Nov 30, 2006
  4. chris
    Replies:
    6
    Views:
    1,007
    chris
    Oct 28, 2005
  5. Kok How Teh
    Replies:
    3
    Views:
    3,057
    Seebs
    Apr 10, 2010
Loading...

Share This Page