Structure size directives

[jacob navia]

Since standard C doesn't provide any way for the programmer to direct
the compiler as to how to layout structures, most compilers provide some
way to do this, albeit in different forms.

Microsoft (and lcc-win) uses
#pragma pack(1)

Gcc uses
__attribute__ {(packed)}

Has anyone seen other directives in other compilers?

Specifically, Microsoft allows to pack structures with some integer
constant (pack(2) for instance), but gcc doesn't seem to allow this.

What other semantic differences could exist there?

I am writing this part of my tutorial, and I would like to make a table
about the constructs used by the various compilers. Note that under
windows you must know this kind of stuff since windows uses packed
structures extensively.

 

[Walter Roberson]

Since standard C doesn't provide any way for the programmer to direct
the compiler as to how to layout structures, most compilers provide some
way to do this, albeit in different forms.

Microsoft (and lcc-win) uses
#pragma pack(1)

Gcc uses
__attribute__ {(packed)}

Has anyone seen other directives in other compilers?

http://techpubs.sgi.com/library/tpl...loper/books/Pragmas/sgi_html/ch04.html#Z26959

SGI IRIX

#pragma pack [n]

where [n] indicates an optional number 1, 2, 4, 8, or 16. The []
are not part of the syntax.

This differs from the Microsoft format in not having () and in having
a space after pack.


SGI includes these notes:

o SGI strongly discourages the use of #pragma pack , because it
is a nonportable feature and the semantics of this directive
may change in future compiler releases.

o A structure declaration must be subjected to identical
instances of a #pragma pack directive in all files, or else
misaligned memory accesses and erroneous structure member
dereferencing may ensue.

o References to fields in packed structures may be less efficient
than references to fields in unpacked structures.

 

[jacob navia]

Since standard C doesn't provide any way for the programmer to direct
the compiler as to how to layout structures, most compilers provide some
way to do this, albeit in different forms.

Microsoft (and lcc-win) uses
#pragma pack(1)

Gcc uses
__attribute__ {(packed)}

Has anyone seen other directives in other compilers?

http://techpubs.sgi.com/library/tpl...loper/books/Pragmas/sgi_html/ch04.html#Z26959

SGI IRIX

#pragma pack [n]

where [n] indicates an optional number 1, 2, 4, 8, or 16. The []
are not part of the syntax.

This differs from the Microsoft format in not having () and in having
a space after pack.

Thanks Mr Roberson

SGI includes these notes:

o SGI strongly discourages the use of #pragma pack , because it
is a nonportable feature and the semantics of this directive
may change in future compiler releases.

o A structure declaration must be subjected to identical
instances of a #pragma pack directive in all files, or else
misaligned memory accesses and erroneous structure member
dereferencing may ensue.

This is obviously the same under windows. ALl files must see the same
directive.

o References to fields in packed structures may be less efficient
than references to fields in unpacked structures.

That is obvious too.

What would be nice is that standard C would allow this to be
defined in the language, with well defined semantics.

 

[CJ]

That is obvious too.

What would be nice is that standard C would allow this to be
defined in the language, with well defined semantics.

Rubbish. Packing for structs is an implementation detail, and any
program relying on such details will be inherently non-portable.

 

[Walter Roberson]

Walter Roberson wrote:

This is obviously the same under windows. ALl files must see the same
directive.

Right, but it isn't the first thing that one thinks of when one
starts using #pragma pack, so it's probably worth mentioning
explicitly in the tutorial.

 

[jacob navia]

Rubbish. Packing for structs is an implementation detail, and any
program relying on such details will be inherently non-portable.

Its surely not a "detail".

If I have
struct foo {
int32_t val;
char *m;
};

Using normal packing rules in 64 bit systems, this makes
16 bytes, using "packed" layout it will be only
12 bytes. This is fully 25 percent of memory usage!

If I have 100 million of those, the difference is a whooping
25 million!

The fact that MANY (if not all) compilers give the programmer
a way to specify this is a testimnoy of the universal need
behind this.

P.S. It would be nice to avoid polemic... "Rubbish" means
here just that you do not agree with me I suppose

 

[jacob navia]

Right, but it isn't the first thing that one thinks of when one
starts using #pragma pack, so it's probably worth mentioning
explicitly in the tutorial.

Yes!

Actually I will include text that says in essence exactly what SGI
says.

 

[Walter Roberson]

Since standard C doesn't provide any way for the programmer to direct
the compiler as to how to layout structures, most compilers provide some
way to do this, albeit in different forms.

Microsoft (and lcc-win) uses
#pragma pack(1)

Gcc uses
__attribute__ {(packed)}

Has anyone seen other directives in other compilers?

http://docs.sun.com/app/docs/doc/819-5265/bjach?l=en&a=view&q=pragma+pack

The Sun Studio 12 C User's Guide
show a syntax largely the same as Microsoft's
but requires #pragma pack() to end the scope.

Sun also includes two important usage notes that might not be the
same on other systems:

Note that when you use #pragma pack, the alignment of the
packed structure or union itself is the same as its more
strictly aligned member. Therefore any declaration of that
struct or union will be at the pack alignment. For example, a
struct with only chars has no alignment restrictions, whereas a
struct containing a double would be aligned on an 8-byte
boundary.


Note --

If you use #pragma pack to align struct or union members on
boundaries other than their natural boundaries, accessing these
fields usually leads to a bus error on SPARC. In order to avoid
such an error, be sure to also specify the -xmemalign option.
See B.2.108 -xmemalign=ab , for the optimal way to compile such
programs.


(e.g., the implication from the SGI documentation is that the SGI
compilers would do the packing and unpacking as needed to handle
memory alignment issues.)

 

[CJ]

Its surely not a "detail".

If I have
struct foo {
int32_t val;
char *m;
};

Using normal packing rules in 64 bit systems, this makes
16 bytes, using "packed" layout it will be only
12 bytes. This is fully 25 percent of memory usage!

If I have 100 million of those, the difference is a whooping
25 million!

The fact that MANY (if not all) compilers give the programmer
a way to specify this is a testimnoy of the universal need
behind this.

The best way for the compiler to give the programmer a way to specifiy
this is surely through a command-line option (for example, as part of an
"optimize for size" option) and not by adding pointless new keywords to
the language. It would be very bad if programmers started making
non-portable assumptions that a struct was packed, rather than using
offsetof and the other proper mechanisms provided by C.

P.S. It would be nice to avoid polemic... "Rubbish" means
here just that you do not agree with me I suppose

It means that the statement you made was obviously nonsense, and badly
thought-out.

 

[Kenneth Brody]

Since standard C doesn't provide any way for the programmer to direct
the compiler as to how to layout structures, most compilers provide some
way to do this, albeit in different forms.

Microsoft (and lcc-win) uses
#pragma pack(1)

Gcc uses
__attribute__ {(packed)}

Has anyone seen other directives in other compilers?

#pragma pack(push)
and
#pragma pack(pop)

This pushes and pops the current pack value, allowing you to surround
your packs with:

#pragma pack(push)
#pragma pack(4)
... your 4-byte packing structs here ...
#pragma pack(pop)

This will not affect any packing outside of your own definitions.

[...]

I am writing this part of my tutorial, and I would like to make a table
about the constructs used by the various compilers. Note that under
windows you must know this kind of stuff since windows uses packed
structures extensively.

Under Windows, I have seen (admittedly non-standard) header files
which wrap these for you, and they are available in Windows and
MS-DOS compilers from several vendors.

pshpack1.h -- pack to 1 byte
pshpack2.h -- etc.
pshpack4.h
pshpack8.h
poppack.h -- restore packing size

So, you can use this across multiple vendors' Windows compilers
such as:

#include <pshpack4.h>
... your 4-byte packing structs here ...
#include <poppack.h>

--
+-------------------------+--------------------+-----------------------+
| Kenneth J. Brody | www.hvcomputer.com | #include |
| kenbrody/at\spamcop.net | www.fptech.com | <std_disclaimer.h> |
+-------------------------+--------------------+-----------------------+
Don't e-mail me at: <mailto:[email protected]>

 

[jacob navia]

Note --

If you use #pragma pack to align struct or union members on
boundaries other than their natural boundaries, accessing these
fields usually leads to a bus error on SPARC. In order to avoid
such an error, be sure to also specify the -xmemalign option.
See B.2.108 -xmemalign=ab , for the optimal way to compile such
programs.


(e.g., the implication from the SGI documentation is that the SGI
compilers would do the packing and unpacking as needed to handle
memory alignment issues.)

Very interesting. Obviously you must be EXTRA careful when a machine has
hardware alignment requirements. The x86 has none, it will just slow
down, but SPARC will trap... Another consideration for the tutorial.

 

[jacob navia]

The best way for the compiler to give the programmer a way to specifiy
this is surely through a command-line option (for example, as part of an
"optimize for size" option) and not by adding pointless new keywords to
the language.

There is no keyword added:

#pragma is used for precisely that: unportable compiler directives.

gcc uses __attribute__ anyway in many other situations.

It would be very bad if programmers started making
non-portable assumptions that a struct was packed, rather than using
offsetof and the other proper mechanisms provided by C.

Then, excuse me but WHAT you propose when you do NOT want that
a structure is aligned? How would you do it?

Specifically in the case above, where you have a structure
several million times in RAM?

It means that the statement you made was obviously nonsense, and badly
thought-out.

OK. You have thought it out better than gcc/microsoft/sun/sgi/.

 

[Tor Rustad]

[Better choice for subject, would be "struct padding ..."]

Since standard C doesn't provide any way for the programmer to direct
the compiler as to how to layout structures, most compilers provide some
way to do this, albeit in different forms.

Most C compilers? Out of how many?


Unaligned accesses to objects, should be banned. It has nothing to do
with portable C code, it might trigger a core dump on non-x86 CPU.

FYI, C++:

http://www.open-std.org/jtc1/sc22/wg14/www/docs/n1262.pdf

 

[Al Balmer]

The best way for the compiler to give the programmer a way to specifiy
this is surely through a command-line option (for example, as part of an
"optimize for size" option) and not by adding pointless new keywords to
the language.

Actually, I prefer a pragma, since it can be applied to structures
where it's needed, while not messing up the rest of the program. To be
done right, there needs to be a compile-time test which tells you
whether the implementation supports it.

As far as I'm concerned, memory savings is not usually an issue, but
forcing a structure layout to meet some external constraint may be.

 

[CJ]

There is no keyword added:

#pragma is used for precisely that: unportable compiler directives.

gcc uses __attribute__ anyway in many other situations.

Yes, and relying on this sort of thing is a bad idea if you're trying to
write portable code.

Then, excuse me but WHAT you propose when you do NOT want that
a structure is aligned? How would you do it?

Specifically in the case above, where you have a structure
several million times in RAM?

I think a struct will always be aligned at least as strictly as the
alignment required by its first member.

OK. You have thought it out better than gcc/microsoft/sun/sgi/.

Probably gcc/etc. want to allow the programmer to decide *for each
individual struct* whether it should be packed or not. This approach
will create terrible maintenance headaches - the pack/don't pack
directive will need to be present each time the compiler sees a
declaration of the struct.

 

[Walter Roberson]

Since standard C doesn't provide any way for the programmer to direct
the compiler as to how to layout structures, most compilers provide some
way to do this, albeit in different forms.

Microsoft (and lcc-win) uses
#pragma pack(1)

Gcc uses
__attribute__ {(packed)}

Has anyone seen other directives in other compilers?

http://docs.hp.com/en/B3901-90012/ch02s02.html

HP also uses

#pragma PACK n

where n is 1, 2, 4, 8, or 16. The reference there doesn't say how
to terminate the scope of a #pragma PACK . I have barely used HP
so I don't know if #pragma pack (lowercase) would be accepted.

The HP documentation gives an explicit table of natural vs requested
alignments; the documentation on some of the other systems is not
forthcoming on what happens if you request a stricter alignment
than would naturally occur.

There is an interesting usage note in the HP documentation:

The alignment of bit-fields is different than either of the two
bit-field alignments for the HP_ALIGN modes. Zero-length
bit-fields will still force the next bit-field to start at the
next boundary for that type. However, PACK bit-fields can cross
natural boundaries.

Note that last bit about what happens with bit-fields; the
other documentation I referenced doesn't mention behaviour upon
bit-fields.


The documentation directly about the HP #pragma PACK
documentation, that of #pragma HP_ALIGN, is sort of interesting.
HP's compilers appears to deal with a range of computers (some of
which I worked on 25 years ago!) and have developed a pragma
to support portability amongst those systems. The next chapter
in the HP manual,
http://docs.hp.com/en/B3901-90012/ch02s03.html
shows some good examples of just how messy that can get, and thus
why having built-in compiler support for it is nice to have.
But on the other hand, the messiness also gives an indication as
to why it might not be as easy as expected to standardize a single
#pragma pack

 

[jacob navia]

Yes, and relying on this sort of thing is a bad idea if you're trying to
write portable code.

Has it occurred to you that there are environments/situations where
you are more interested in getting the program to RUN is much
more important than writing it portably?

I have worked in DSPs where there was only 80K RAM, and of that 70K
was used by the modem software.

I had to run in 10K.

Or, you have to layout your structure according to EXTERNAL
requirements like passing it character-wise through a
serial interface, and at the other side you can't assume any
"alignment" wasted space, etc etc!

I think a struct will always be aligned at least as strictly as the
alignment required by its first member.

You are mistaken.

struct foo {
char a;
double b;
char c;
};

The double will be aligned (in many machines) into an 8 byte
boundary

Probably gcc/etc. want to allow the programmer to decide *for each
individual struct* whether it should be packed or not. This approach
will create terrible maintenance headaches - the pack/don't pack
directive will need to be present each time the compiler sees a
declaration of the struct.

Obviously the compiler has to see the declaration of the structure, so
it is imperative to put everything related to it in the declaration.
This poses no problem at all if you write your structure declarations
in header files...

 

[Walter Roberson]

The documentation directly about the HP #pragma PACK
documentation, that of #pragma HP_ALIGN, is sort of interesting.

Forgot to reference an interesting portion of it:

http://docs.hp.com/en/B3901-90012/ch02s02.html

The alignment pragma will affect typedef, struct, and union
types. Therefore, in the following declaration:

#pragma HP_ALIGN HPUX_WORD
typedef int int32;

int32 is not equivalent to int.

Sample code to illustrate the issue is given there.


This is not strictly a #pragma pack issue, but it is closely related,
an example of the complexities of dealing with packing and alignment
to external constraints.

 

[jacob navia]

Walter Roberson wrote:
[snip]

http://docs.hp.com/en/B3901-90012/ch02s03.html

This documebtation introcues the concept of "most restrictive
member", and I think it is the member with the most strict
alignment requirements. Lcc-win doesn't make any such
restriction and just aligns one member after the other...

Luckily the x86 is forgiving

 

[Al Balmer]

Probably gcc/etc. want to allow the programmer to decide *for each
individual struct* whether it should be packed or not. This approach
will create terrible maintenance headaches - the pack/don't pack
directive will need to be present each time the compiler sees a
declaration of the struct.

No, only on the definition.