Dealing with struct padding using a dynamic element

Discussion in 'C++' started by mojumbo, Oct 3, 2008.

  1. mojumbo

    mojumbo Guest

    Problem:

    I have a structure which needs to store its data in contiguous memory
    by there is a dynamic element which can't be defined at compile time.
    It needs to be aligned along a 4 byte boundary. This is what my
    structure looks like:

    START

    struct tsBob
    {
    unsigned int fieldA;
    unsigned int fieldB;

    unsigned short varLen[0];
    }

    tsBob* myBob;

    myBob = (tsBob*)malloc(sizeof(tsBob) + sizeof(unsigned short) * 1);

    END

    myBob needs an additional 3 bytes alloc'd to it to be aligned. I
    don't want to use %

    Finally the question,

    Can I use a bitwise something to determine how many extra bytes I need
    and malloc that with the
    rest of the function call?
    mojumbo, Oct 3, 2008
    #1
    1. Advertising

  2. On 2008-10-03 16:47, Victor Bazarov wrote:
    > mojumbo wrote:
    >> Problem:
    >>
    >> I have a structure which needs to store its data in contiguous memory
    >> by there is a dynamic element which can't be defined at compile time.
    >> It needs to be aligned along a 4 byte boundary. This is what my
    >> structure looks like:


    Unless you can use some kind of struct/type which is 4 bytes (and make
    sure that the first element in the array is correctly aligned using
    padding) instead of the unsigned shorts you will have to take care when
    allocating the memory. Perhaps using a factory-function (which allocates
    the correct number of bytes) would be a good idea.

    >> struct tsBob
    >> {
    >> unsigned int fieldA;
    >> unsigned int fieldB;
    >>
    >> unsigned short varLen[0];

    >
    > I am not sure this is OK, you might consider giving it at least 1 element.


    It's not OK, the standard requires a number greater than zero, but some
    compilers (gcc for one) accepts this. We use this a lot at work when
    building messages to be sent over the network, and I find it helpful to
    have the zero-sized array as a way of describing the message layout. It
    usually looks something like this:

    struct Msg
    {
    unsigned nrFoo;
    unsigned nrBar;
    #if 0
    Foo foos[0];
    Bar bars[0];
    #endif
    };

    --
    Erik Wikström
    Erik Wikström, Oct 3, 2008
    #2
    1. Advertising

  3. mojumbo

    mojumbo Guest

    Thanks for the above comments but I will clarify:

    I am using an overloaded operator new in the structure which takes the
    number of elements just as you suggested, so the actual spec for the
    struct does look like this.

    struct tsBob
    {
    unsigned int fieldA;
    unsigned int fieldB;

    unsigned short varLen[0];

    void* operator new(size_t, int aNum)
    { return malloc(sizeof(tsBob) + sizeof(unsigned short)*num; }
    };

    This is exactly what I'm trying to solve - aNum can come in as any
    number (which is why I multiplied by one in my first explanation)

    "bitwise something" meaning -
    I also have another structure with float aFlts[0].
    My new looks like this: void* operator new(size_t, int aNum) { return
    sizeof(tsFltStr) + sizeof(float)* aNum + (aNum & 0x01) ? 1:0); }

    It's nice with 32-bits, just add another.
    Also, I thought zero length arrays were part of the C99 standard?
    mojumbo, Oct 3, 2008
    #3
  4. mojumbo

    James Kanze Guest

    On Oct 3, 4:47 pm, Victor Bazarov <> wrote:
    > mojumbo wrote:
    > > Problem:


    > > I have a structure which needs to store its data in
    > > contiguous memory by there is a dynamic element which can't
    > > be defined at compile time. It needs to be aligned along a
    > > 4 byte boundary. This is what my structure looks like:


    > > START


    > > struct tsBob
    > > {
    > > unsigned int fieldA;
    > > unsigned int fieldB;


    > > unsigned short varLen[0];


    > I am not sure this is OK, you might consider giving it at
    > least 1 element.


    It's not legal C++. Nor legal C, but in C, the last element may
    have an incomplete array type, e.g.:
    unsigned short varLen[] ;
    You can, of course, give the final element a length of 1 in
    either language, but then any array access with an index greater
    than 0 is undefined behavior.

    > > }

    > ;


    > > tsBob* myBob;


    > > myBob = (tsBob*)malloc(sizeof(tsBob) + sizeof(unsigned short) * 1);


    > This is better accomplished by the correctly implemented
    > 'operator new' in the class itself, and then you just do


    > tsBob* myBob = new (true_varLen_count) myBob;


    If alignment isn't an issue (and I don't think it can be in his
    exact case), then this should be accompanied with:

    struct tsBob
    {
    unsigned short* varLen()
    {
    return reinterpret_cast< unsigned short* >( this + 1 ) ;
    }
    } ;

    or
    struct tsBob
    {
    unsigned short& operator[]( std::size_t index ) ;
    {
    // bounds checking...
    return reinterpret_cast< unsigned short* >(
    this + 1)[ index ] ;
    }
    } ;

    for accessing the additional elements. I'd also do something to
    ensure that the actual length was correctly memorized somewhere.

    > > END


    > > myBob needs an additional 3 bytes alloc'd to it to be aligned. I
    > > don't want to use %


    > If the size in bytes is divisible by 4, it's going to be
    > aligned on the 4 byte boundary. That's the specification of
    > 'malloc', IIRC.


    No. All that malloc (or operator new()) guarantee is that the
    returned pointer is sufficiently aligned for any type. In his
    case, he's safe because it's not conceivable that unsigned short
    require more alignment than the unsigned int in his struct. In
    general, however, you do have to worry about alignment; the
    implementation of std::basic_string in g++ uses a similar trick,
    and core dumps if you try to use std::basic_string< double > (or
    probably std::basic_string< long long >). (Hmmm. I wonder if
    there are any platforms where wchar_t is equivalent to a long
    long. If so, g++ has a real problem.)

    --
    James Kanze (GABI Software) email:
    Conseils en informatique orientée objet/
    Beratung in objektorientierter Datenverarbeitung
    9 place Sémard, 78210 St.-Cyr-l'École, France, +33 (0)1 30 23 00 34
    James Kanze, Oct 4, 2008
    #4
  5. James Kanze wrote:
    >
    > No. All that malloc (or operator new()) guarantee is that the
    > returned pointer is sufficiently aligned for any type.


    I knew about malloc, but are you sure this is true for operator new? It
    should know the requirement of the needed type in advance so the largest
    aligment seems like a overspecification?

    -- Hrvoje Prge¹a
    Hrvoje Prge¹a, Oct 4, 2008
    #5
  6. On 3 Oct, 22:25, Victor Bazarov <> wrote:
    > mojumbo wrote:
    > > [..]
    > > Also, I thought zero length arrays were part of the C99 standard?

    >
    > They may have been, but what relevance does it have here?  We *are*
    > talking C++ here, aren't we?  Not C99.  Or are you unaware that they are
    > two different languages?


    Was it not the original intent of C++ to be compatible with C?

    --
    Max
    Maxim Yegorushkin, Oct 4, 2008
    #6
  7. Maxim Yegorushkin wrote:
    >
    > Was it not the original intent of C++ to be compatible with C?
    >


    It still is, but the c++ standard is always catching up with the c
    standard. Unspecified/flexible length arrays are from C99, the latest
    c++ standard is C++03 (which is an updated version of c++98). It'll
    probably get in the new standard, sometime in the future.

    For more about incompatibilities see:
    http://david.tribble.com/text/cdiffs.htm#C99-fam
    http://en.wikipedia.org/wiki/Compatibility_of_C_and_C++
    Hrvoje Prge¹a, Oct 4, 2008
    #7
  8. mojumbo

    Ian Collins Guest

    Hrvoje Prge¹a wrote:
    > James Kanze wrote:
    >>
    >> No. All that malloc (or operator new()) guarantee is that the
    >> returned pointer is sufficiently aligned for any type.

    >
    > I knew about malloc, but are you sure this is true for operator new? It
    > should know the requirement of the needed type in advance so the largest
    > aligment seems like a overspecification?
    >

    No, operator new does not know which type is being allocated, it is only
    passed the size.

    --
    Ian Collins.
    Ian Collins, Oct 4, 2008
    #8
  9. mojumbo

    James Kanze Guest

    On Oct 4, 3:57 pm, Hrvoje Prge?a <> wrote:
    > Maxim Yegorushkin wrote:


    > > Was it not the original intent of C++ to be compatible with C?


    Up to a certain point. The original C++ required function
    prototypes, for example, which didn't even exist in C at the
    time.

    > It still is, but the c++ standard is always catching up with
    > the c standard. Unspecified/flexible length arrays are from
    > C99, the latest c++ standard is C++03 (which is an updated
    > version of c++98). It'll probably get in the new standard,
    > sometime in the future.


    I don't think so. It would require at least a minimum of work
    (a formal proposal, etc.), and no one thought it worth the
    effort.

    --
    James Kanze (GABI Software) email:
    Conseils en informatique orientée objet/
    Beratung in objektorientierter Datenverarbeitung
    9 place Sémard, 78210 St.-Cyr-l'École, France, +33 (0)1 30 23 00 34
    James Kanze, Oct 6, 2008
    #9
  10. mojumbo

    James Kanze Guest

    On Oct 4, 3:20 pm, Hrvoje Prge?a <> wrote:
    > James Kanze wrote:


    > > No. All that malloc (or operator new()) guarantee is that the
    > > returned pointer is sufficiently aligned for any type.


    > I knew about malloc, but are you sure this is true for
    > operator new? It should know the requirement of the needed
    > type in advance so the largest aligment seems like a
    > overspecification?


    The operator new function doesn't know what type it is being
    called for, and must return a pointer suitably aligned for any
    function. (See §3.7.4.1.)

    --
    James Kanze (GABI Software) email:
    Conseils en informatique orientée objet/
    Beratung in objektorientierter Datenverarbeitung
    9 place Sémard, 78210 St.-Cyr-l'École, France, +33 (0)1 30 23 00 34
    James Kanze, Oct 6, 2008
    #10
  11. mojumbo

    James Kanze Guest

    On Oct 4, 10:14 pm, Ian Collins <> wrote:
    > Hrvoje Prge¹a wrote:
    > > James Kanze wrote:


    > >> No. All that malloc (or operator new()) guarantee is that
    > >> the returned pointer is sufficiently aligned for any type.


    > > I knew about malloc, but are you sure this is true for
    > > operator new? It should know the requirement of the needed
    > > type in advance so the largest aligment seems like a
    > > overspecification?


    > No, operator new does not know which type is being allocated,
    > it is only passed the size.


    There's a possibility of ambiguity with the expression "operator
    new": it can be interpreted to mean a "new expression" or the
    "new operator", or to mean the operator new function. In the
    first case, it does know the type.

    I specifically wrote "operator new()", with the parenthesis, to
    make it clear that I was talking about the function.
    Apparently, it wasn't clear enough.

    --
    James Kanze (GABI Software) email:
    Conseils en informatique orientée objet/
    Beratung in objektorientierter Datenverarbeitung
    9 place Sémard, 78210 St.-Cyr-l'École, France, +33 (0)1 30 23 00 34
    James Kanze, Oct 6, 2008
    #11
  12. mojumbo

    James Kanze Guest

    On Oct 7, 6:37 pm, (blargg) wrote:
    > In article
    > <>, James


    > Kanze <> wrote:
    > > On Oct 4, 3:20 pm, Hrvoje Prge?a <> wrote:
    > > > James Kanze wrote:


    > > > > No. All that malloc (or operator new()) guarantee is
    > > > > that the returned pointer is sufficiently aligned for
    > > > > any type.


    > > > I knew about malloc, but are you sure this is true for
    > > > operator new? It should know the requirement of the needed
    > > > type in advance so the largest aligment seems like a
    > > > overspecification?


    > > The operator new function doesn't know what type it is being
    > > called for, and must return a pointer suitably aligned for
    > > any function. (See =A73.7.4.1.)


    > Couldn't it do the same that new char [] and new unsigned char
    > [] (operator, not function) does (C++03 section 5.3.4
    > paragraph 10)? There, the memory is suitably aligned for any
    > object of that size or less, since one can't (legally) store
    > an object of greater size.


    The standard says (concerning the return value of an "allocation
    functionr", §3.7.3.1/2): "The pointer returned shall be suitably
    aligned so that it can be converted to a pointer of any complete
    object type and then used to access the object or array in the
    storage allocated (until the storage is explicitly deallocated
    by a call to a corresponding deallocation function)." The first
    part of the sentence is clear: it must be suitably aligned for
    any type. The second part makes it a little less clear; it's
    obvious that you can't use the pointer to access an object
    larger than the ammount of memory allocated.

    --
    James Kanze (GABI Software) email:
    Conseils en informatique orientée objet/
    Beratung in objektorientierter Datenverarbeitung
    9 place Sémard, 78210 St.-Cyr-l'École, France, +33 (0)1 30 23 00 34
    James Kanze, Oct 8, 2008
    #12
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Chris Fogelklou
    Replies:
    36
    Views:
    1,345
    Chris Fogelklou
    Apr 20, 2004
  2. edware

    struct padding

    edware, Apr 25, 2006, in forum: C Programming
    Replies:
    9
    Views:
    430
    Ed Vogel
    Apr 27, 2006
  3. Hallvard B Furuseth

    Padding bits and struct assignment

    Hallvard B Furuseth, Dec 27, 2006, in forum: C Programming
    Replies:
    5
    Views:
    628
    Eric Sosman
    Dec 27, 2006
  4. Replies:
    11
    Views:
    864
  5. Replies:
    13
    Views:
    341
    glen herrmannsfeldt
    May 5, 2013
Loading...

Share This Page