Alignment in Struct

[janus]

Hello All,

I really see where programmers strive to achievement alignment manually. For example,

struct {
char yo;
int slen;
char state;
}

I would see something of this nature,

struct {
int slen;
char yo;
char state;
}

On 32 bit machine, at times I see even dummies here and there to make sure that the whole thing is properly aligned.

I would need pointers on where to read more of this issue, and I also would like to hear from you on this.

Regards, Janus

 

[Eric Sosman]

Hello All,

I really see where programmers strive to achievement alignment manually. For example,

struct {
char yo;
int slen;
char state;
}

I would see something of this nature,

struct {
int slen;
char yo;
char state;
}

On a system where `int' requires four-byte alignment, the first
struct is likely to take twelve bytes:

char yo; // one byte
// three-byte gap
int slen; // four bytes
char state; // one byte
// three-byte gap

.... while the second would probably need only eight:

int slen; // four bytes
char yo; // one byte
char state; // one byte
// two-byte gap

If the program creates many instances of this struct, using the
second instead of the first could cut memory use by one-third.
(Alternatively: Using the first instead of the second could bloat
memory by fifty percent.) Since modern CPU's are much faster than
modern memories, getting the same amount of "payload" into fewer
bytes might make a significant difference in speed, too.

On 32 bit machine, at times I see even dummies here and there to make sure that the whole thing is properly aligned.

That seems odd. The compiler will insert whatever padding is
needed, so why would the programmer need to get involved? Maybe it
would make some sense if the code used a non-standard extension like
`#pragma packed" or `__attribute__ ((aligned(N)))' to say that the
programmer, rather than the compiler, is assuming responsibility for
alignment -- but if the programmer then has to turn around and create
dummies, what's the point?

My guess is that the dummies aren't there for the purpose of
aligning things, but in order to match some externally-defined
format: An IP packet header or an on-disk inode or some such. It
is tempting to use structs for such purposes -- tempting, but
hazardous, since different compilers may lay out "the same" struct
differently.

I would need pointers on where to read more of this issue, and I also would like to hear from you on this.

Question 2.12 on the comp.lang.c FAQ page <http://www.c-faq.com/>
is a good, if brief introduction. There are some useful links at the
bottom of the question page, too.

 

[Keith Thompson]

On 2/26/2014 9:06 AM, janus wrote: [...]

On 32 bit machine, at times I see even dummies here and there to make
sure that the whole thing is properly aligned.

That seems odd. The compiler will insert whatever padding is
needed, so why would the programmer need to get involved? Maybe it
would make some sense if the code used a non-standard extension like
`#pragma packed" or `__attribute__ ((aligned(N)))' to say that the
programmer, rather than the compiler, is assuming responsibility for
alignment -- but if the programmer then has to turn around and create
dummies, what's the point?

My guess is that the dummies aren't there for the purpose of
aligning things, but in order to match some externally-defined
format: An IP packet header or an on-disk inode or some such. It
is tempting to use structs for such purposes -- tempting, but
hazardous, since different compilers may lay out "the same" struct
differently.

I've worked on code that had to run on two different systems, 68k
and SPARC (same endianness, same type sizes, different alignment
requirements), sharing data via disk files. I inserted dummy
struct members to ensure that the non-dummy members were at the same
offset on both systems. There was no *externally imposed* layout
(no other software had to deal with the on-disk binary data); I only
cared about making the layout consistent between the two systems.

 

[BartC]

On 2/26/2014 9:06 AM, janus wrote:

That seems odd. The compiler will insert whatever padding is
needed, so why would the programmer need to get involved? Maybe it
would make some sense if the code used a non-standard extension like
`#pragma packed" or `__attribute__ ((aligned(N)))' to say that the
programmer, rather than the compiler, is assuming responsibility for
alignment -- but if the programmer then has to turn around and create
dummies, what's the point?

By using pragma packed(1) or whatever, and using dummy members, the
programmer has fuller control over the layout and overall size. Some might
see that as an advantage, for example with 100% control, it is possible to
make a particular set of fields, adjusting the size and ordering of each if
necessary, fit neatly into a power-of-two struct, instead of possibly being
a little over.

Or perhaps to avoid the compiler making an unnecessary alignment (for
example 8-byte item at 8-byte multiple offset, even though the machine is
32-bits, which might be a bit faster, but the programmer knows that access
to that field will be infrequent and that overall size is more important).

Also to make it more likely the same struct size and layout will occur with
a different compiler, different target, different compile options etc.

 

[David Brown]

On 2/26/2014 9:06 AM, janus wrote: [...]

On 32 bit machine, at times I see even dummies here and there to make
sure that the whole thing is properly aligned.

That seems odd. The compiler will insert whatever padding is
needed, so why would the programmer need to get involved? Maybe it
would make some sense if the code used a non-standard extension like
`#pragma packed" or `__attribute__ ((aligned(N)))' to say that the
programmer, rather than the compiler, is assuming responsibility for
alignment -- but if the programmer then has to turn around and create
dummies, what's the point?

My guess is that the dummies aren't there for the purpose of
aligning things, but in order to match some externally-defined
format: An IP packet header or an on-disk inode or some such. It
is tempting to use structs for such purposes -- tempting, but
hazardous, since different compilers may lay out "the same" struct
differently.

I've worked on code that had to run on two different systems, 68k
and SPARC (same endianness, same type sizes, different alignment
requirements), sharing data via disk files. I inserted dummy
struct members to ensure that the non-dummy members were at the same
offset on both systems. There was no *externally imposed* layout
(no other software had to deal with the on-disk binary data); I only
cared about making the layout consistent between the two systems.

I also have used explicit dummies for similar reasons - to match structs
exactly between more than one system. I use them occasionally within a
single system - for some of the processors I use, particular structure
sizes may be more efficient for arrays (for example, an 8-bit cpu would
be happy with 1-byte alignment, but if it doesn't have a multiply
instruction then an 8-byte struct size could be faster than a 7-byte
struct).

At the risk of irritating some people who think individual compilers are
off-topic, I sometimes use the "-Wpadding" flag in gcc to check that the
explicit dummies are all there. Static assertions on the size of the
struct are also useful.

 

[Keith Thompson]

By using pragma packed(1) or whatever, and using dummy members, the
programmer has fuller control over the layout and overall size. Some might
see that as an advantage, for example with 100% control, it is possible to
make a particular set of fields, adjusting the size and ordering of each if
necessary, fit neatly into a power-of-two struct, instead of possibly being
a little over.

Of course "pragma packed" (more commonly "#pragma pack") is
non-standard, and I think different compilers may assign different
meanings to the argument.

Apart from that, at least gcc's implementation can introduce undefined
behavior. If you have, say, an int member of a structure whose
alignment is forced to 1 byte, accesses to that member by name should
cause the compiler to generate whatever code is necessary to access it
correctly -- but if you take its address as an int* and later try to
dereference that pointer, you can have undefined behavior. This may be
part of the reason this feature is not in the C standard.

gcc-specific details at http://stackoverflow.com/q/8568432/827263

 

[BartC]

Of course "pragma packed" (more commonly "#pragma pack") is
non-standard, and I think different compilers may assign different
meanings to the argument.

Apart from that, at least gcc's implementation can introduce undefined
behavior. If you have, say, an int member of a structure whose
alignment is forced to 1 byte, accesses to that member by name should
cause the compiler to generate whatever code is necessary to access it
correctly -- but if you take its address as an int* and later try to
dereference that pointer, you can have undefined behavior. This may be
part of the reason this feature is not in the C standard.

gcc-specific details at http://stackoverflow.com/q/8568432/827263

That seems to be about items that are misaligned for the hardware, and where
the compiler doesn't generate the byte-at-a-time safe accesses that are
needed.

I rarely use the feature to deliberately misalign fields (the opposite in
fact; that's why dummy fields are sometimes added). Just to ensure a
particular arrangement.

One example from some old code (the original struct was in C, then I had to
replicate this in another language, and if I wanted now to write it in C
(the original is long lost), it would look like this):

#pragma pack(1) // etc

struct {
char padding[19966]
int16_t a
int16_t b
int16_t c
char* s
int32_t d
int32_t padding2
}

(There should be some semicolons sprinkled about too.) This was for
duplicating an extremely complex struct to do with JPG processing; only the
last few fields were of interest.

I'd measured that there was a 19966-byte offset to the first field I wanted,
which is a multiple of 2, but not 4. Who knows whether a compiler would
round up such an array to next multiple of 4? But the offset has to be
exact. The last member was to make sure the size of this struct matches
that of the original (in case that causes any problems: some function in a
library might clear it, and it has to be the right size).

This sort of thing is problematical anyway: the struct is used to
communicate with a pre-compiled library; even if I used the original struct
declarations, could my C compiler be trusted to have all the fields the
right size and at the right offsets? A library using specific field widths,
and
1-byte packing, could ensure that more reliably.

 

[Eric Sosman]

[...]
My guess is that the dummies aren't there for the purpose of
aligning things, but in order to match some externally-defined
format: An IP packet header or an on-disk inode or some such. It
is tempting to use structs for such purposes -- tempting, but
hazardous, since different compilers may lay out "the same" struct
differently.

I've worked on code that had to run on two different systems, 68k
and SPARC (same endianness, same type sizes, different alignment
requirements), sharing data via disk files. I inserted dummy
struct members to ensure that the non-dummy members were at the same
offset on both systems. There was no *externally imposed* layout
(no other software had to deal with the on-disk binary data); I only
cared about making the layout consistent between the two systems.

Externally-*imposed* means (I guess) "dictated by some
authority over which I have little or no influence." By
externally-*defined* I meant only "governed by considerations
external to the running program." In your situation the layout
might not be "externally-imposed" because you could choose it
for yourself (perhaps in consultation with others), but I'd
still call it "externally-defined" because once you've chosen
your layout every program that uses the data must conform. The
layout becomes part of the program's specification, part of its
requirements document -- and those, in my view, are "external"
to the program itself.

... and, yeah: Sometimes you *can* use structs and a bit of
inside information about particular compilers to make things
work, for a few compilers at any rate. But if there's any chance
that the code might someday find itself on other those couple of
intimately-understood compilers... I'll stick with "hazardous."