Dealing with struct padding using a dynamic element

[mojumbo]

Problem:

I have a structure which needs to store its data in contiguous memory
by there is a dynamic element which can't be defined at compile time.
It needs to be aligned along a 4 byte boundary. This is what my
structure looks like:

START

struct tsBob
{
unsigned int fieldA;
unsigned int fieldB;

unsigned short varLen[0];
}

tsBob* myBob;

myBob = (tsBob*)malloc(sizeof(tsBob) + sizeof(unsigned short) * 1);

END

myBob needs an additional 3 bytes alloc'd to it to be aligned. I
don't want to use %

Finally the question,

Can I use a bitwise something to determine how many extra bytes I need
and malloc that with the
rest of the function call?

 

[Erik Wikström]

Unless you can use some kind of struct/type which is 4 bytes (and make
sure that the first element in the array is correctly aligned using
padding) instead of the unsigned shorts you will have to take care when
allocating the memory. Perhaps using a factory-function (which allocates
the correct number of bytes) would be a good idea.

struct tsBob
{
unsigned int fieldA;
unsigned int fieldB;

unsigned short varLen[0];

I am not sure this is OK, you might consider giving it at least 1 element.

It's not OK, the standard requires a number greater than zero, but some
compilers (gcc for one) accepts this. We use this a lot at work when
building messages to be sent over the network, and I find it helpful to
have the zero-sized array as a way of describing the message layout. It
usually looks something like this:

struct Msg
{
unsigned nrFoo;
unsigned nrBar;
#if 0
Foo foos[0];
Bar bars[0];
#endif
};

 

[mojumbo]

Thanks for the above comments but I will clarify:

I am using an overloaded operator new in the structure which takes the
number of elements just as you suggested, so the actual spec for the
struct does look like this.

struct tsBob
{
unsigned int fieldA;
unsigned int fieldB;

unsigned short varLen[0];

void* operator new(size_t, int aNum)
{ return malloc(sizeof(tsBob) + sizeof(unsigned short)*num; }
};

This is exactly what I'm trying to solve - aNum can come in as any
number (which is why I multiplied by one in my first explanation)

"bitwise something" meaning -
I also have another structure with float aFlts[0].
My new looks like this: void* operator new(size_t, int aNum) { return
sizeof(tsFltStr) + sizeof(float)* aNum + (aNum & 0x01) ? 1:0); }

It's nice with 32-bits, just add another.
Also, I thought zero length arrays were part of the C99 standard?

 

[James Kanze]

Problem:
I have a structure which needs to store its data in
contiguous memory by there is a dynamic element which can't
be defined at compile time. It needs to be aligned along a
4 byte boundary. This is what my structure looks like:
START
struct tsBob
{
unsigned int fieldA;
unsigned int fieldB;
unsigned short varLen[0];

I am not sure this is OK, you might consider giving it at
least 1 element.

It's not legal C++. Nor legal C, but in C, the last element may
have an incomplete array type, e.g.:
unsigned short varLen[] ;
You can, of course, give the final element a length of 1 in
either language, but then any array access with an index greater
than 0 is undefined behavior.

This is better accomplished by the correctly implemented
'operator new' in the class itself, and then you just do

tsBob* myBob = new (true_varLen_count) myBob;

If alignment isn't an issue (and I don't think it can be in his
exact case), then this should be accompanied with:

struct tsBob
{
unsigned short* varLen()
{
return reinterpret_cast< unsigned short* >( this + 1 ) ;
}
} ;

or
struct tsBob
{
unsigned short& operator[]( std::size_t index ) ;
{
// bounds checking...
return reinterpret_cast< unsigned short* >(
this + 1)[ index ] ;
}
} ;

for accessing the additional elements. I'd also do something to
ensure that the actual length was correctly memorized somewhere.

If the size in bytes is divisible by 4, it's going to be
aligned on the 4 byte boundary. That's the specification of
'malloc', IIRC.

No. All that malloc (or operator new()) guarantee is that the
returned pointer is sufficiently aligned for any type. In his
case, he's safe because it's not conceivable that unsigned short
require more alignment than the unsigned int in his struct. In
general, however, you do have to worry about alignment; the
implementation of std::basic_string in g++ uses a similar trick,
and core dumps if you try to use std::basic_string< double > (or
probably std::basic_string< long long >). (Hmmm. I wonder if
there are any platforms where wchar_t is equivalent to a long
long. If so, g++ has a real problem.)

 

[Hrvoje Prge¹a]

No. All that malloc (or operator new()) guarantee is that the
returned pointer is sufficiently aligned for any type.

I knew about malloc, but are you sure this is true for operator new? It
should know the requirement of the needed type in advance so the largest
aligment seems like a overspecification?

-- Hrvoje Prge¹a

 

[Maxim Yegorushkin]

[..]
Also, I thought zero length arrays were part of the C99 standard?

They may have been, but what relevance does it have here?  We *are*
talking C++ here, aren't we?  Not C99.  Or are you unaware that they are
two different languages?

Was it not the original intent of C++ to be compatible with C?

 

[Hrvoje Prge¹a]

Was it not the original intent of C++ to be compatible with C?

It still is, but the c++ standard is always catching up with the c
standard. Unspecified/flexible length arrays are from C99, the latest
c++ standard is C++03 (which is an updated version of c++98). It'll
probably get in the new standard, sometime in the future.

For more about incompatibilities see:
http://david.tribble.com/text/cdiffs.htm#C99-fam
http://en.wikipedia.org/wiki/Compatibility_of_C_and_C++

 

[Ian Collins]

I knew about malloc, but are you sure this is true for operator new? It
should know the requirement of the needed type in advance so the largest
aligment seems like a overspecification?

No, operator new does not know which type is being allocated, it is only
passed the size.

 

[James Kanze]

Maxim Yegorushkin wrote:

Up to a certain point. The original C++ required function
prototypes, for example, which didn't even exist in C at the
time.

It still is, but the c++ standard is always catching up with
the c standard. Unspecified/flexible length arrays are from
C99, the latest c++ standard is C++03 (which is an updated
version of c++98). It'll probably get in the new standard,
sometime in the future.

I don't think so. It would require at least a minimum of work
(a formal proposal, etc.), and no one thought it worth the
effort.

 

[James Kanze]

James Kanze wrote:

I knew about malloc, but are you sure this is true for
operator new? It should know the requirement of the needed
type in advance so the largest aligment seems like a
overspecification?

The operator new function doesn't know what type it is being
called for, and must return a pointer suitably aligned for any
function. (See §3.7.4.1.)

 

[James Kanze]

No, operator new does not know which type is being allocated,
it is only passed the size.

There's a possibility of ambiguity with the expression "operator
new": it can be interpreted to mean a "new expression" or the
"new operator", or to mean the operator new function. In the
first case, it does know the type.

I specifically wrote "operator new()", with the parenthesis, to
make it clear that I was talking about the function.
Apparently, it wasn't clear enough.

 

[James Kanze]

<a479a519-6609-4a9b-976b-7ccd23ad8...@f63g2000hsf.googlegroups.com>, James

Couldn't it do the same that new char [] and new unsigned char
[] (operator, not function) does (C++03 section 5.3.4
paragraph 10)? There, the memory is suitably aligned for any
object of that size or less, since one can't (legally) store
an object of greater size.

The standard says (concerning the return value of an "allocation
functionr", §3.7.3.1/2): "The pointer returned shall be suitably
aligned so that it can be converted to a pointer of any complete
object type and then used to access the object or array in the
storage allocated (until the storage is explicitly deallocated
by a call to a corresponding deallocation function)." The first
part of the sentence is clear: it must be suitably aligned for
any type. The second part makes it a little less clear; it's
obvious that you can't use the pointer to access an object
larger than the ammount of memory allocated.