Structure size directives

[santosh]

Why is this non-portable? The Ada standard (and, thus, compilers)
have done this portably for years.

It not portable in C. Ada is totally different.

 

[Spoon]

OK. Thanks. Maybe there is some document describing this in detail?

The manual perhaps?

http://gcc.gnu.org/onlinedocs/gcc-4.2.2/gcc/Variable-Attributes.html

 

[REH]

It not portable in C.

You miss my point. The post I replied too suggested there was not a
way to do it portable. Obviously, it is currently not portable in C.
That does not imply there is not a way to make it portable in the
standard (though whether it should be it a different argument, and not
one I care about).

Ada is totally different.

Thank you, I am aware of that fact.

REH

 

[Richard Heathfield]

santosh said:

It not portable in C. Ada is totally different.

I don't know Ada, but I guess I can guess, right? It is quite possible that
Ada makes structure packing portable by picking one strategy and insisting
on it. If so, well, yes, that would work. C, on the other hand, does not
do this (and in a different way, that works too, but it can be a trap for
the unwary).

Depending on your programming needs and preferences, you might call this an
obstacle to portability, or you might call it a sensible unwillingness to
place arbitrary restrictions on implementations.

In practice, the various struct packing strategies are not a serious
obstacle to portability. My approach is to ignore them completely except
where data interchange formats (eg file formats) are at stake, at which
point I recognise that I need to put a layer between the struct and the
point where data exchange occurs (whether it's a socket or a file or
whatever), so that I can sidestep the issue altogether.

 

[REH]

I don't know Ada, but I guess I can guess, right? It is quite possible that
Ada makes structure packing portable by picking one strategy and insisting
on it. If so, well, yes, that would work. C, on the other hand, does not
do this (and in a different way, that works too, but it can be a trap for
the unwary).

Depending on your programming needs and preferences, you might call this an
obstacle to portability, or you might call it a sensible unwillingness to
place arbitrary restrictions on implementations.

In practice, the various struct packing strategies are not a serious
obstacle to portability. My approach is to ignore them completely except
where data interchange formats (eg file formats) are at stake, at which
point I recognise that I need to put a layer between the struct and the
point where data exchange occurs (whether it's a socket or a file or
whatever), so that I can sidestep the issue altogether.

OT:
Ada has what are called representation specifications (rep. specs.).
You can define, down to the bit, where fields are laid out and how big
they are. I believe the latest standard (which I am only mildly
familiar with) also allows for defining endian. There can, of course,
be alignment issues. We have one compiler that allows anything, and
will create "fix up" code to workaround alignment issues, and another
that will not. So, it is not a perfect solution. But it is great
when you have to explicitly define an interface to another language,
hardware, etc.

REH

 

[Keith Thompson]

Probably gcc/etc. want to allow the programmer to decide *for each
individual struct* whether it should be packed or not.

Of course. You don't want that capability?

If packing is controlled by a command-line option, as you suggest, it
will also affect structs declared in system headers. That would be
incredibly messy.

This approach
will create terrible maintenance headaches - the pack/don't pack
directive will need to be present each time the compiler sees a
declaration of the struct.

Having more than one definition for the same struct already creates a
terrible maintenance headache. The solution, with or without packing,
is to define a struct exactly once (in a header file if it needs to be
visible in more than one translation unit).

If something like #pragma pack became standard, it would in effect be
part of the struct definition, to be specified exactly once. As long
as the compiler generates correct code to access the struct and its
members, I don't see a problem. Like inline or register, it could
even be ignored with no change to the semantics of the program (unless
the program makes assumptions about the layout).

 

[Keith Thompson]

Probably gcc/etc. want to allow the programmer to decide *for each
individual struct* whether it should be packed or not.

Of course. You don't want that capability?

If packing is controlled by a command-line option, as you suggest, it
will also affect structs declared in system headers. That would be
incredibly messy.

This approach
will create terrible maintenance headaches - the pack/don't pack
directive will need to be present each time the compiler sees a
declaration of the struct.

Having more than one definition for the same struct already creates a
terrible maintenance headache. The solution, with or without packing,
is to define a struct exactly once (in a header file if it needs to be
visible in more than one translation unit).

If something like #pragma pack became standard, it would in effect be
part of the struct definition, to be specified exactly once. As long
as the compiler generates correct code to access the struct and its
members, I don't see a problem. Like inline or register, it could
even be ignored with no change to the semantics of the program (unless
the program makes assumptions about the layout).

[I accidentally posted this through rr.com; I'm re-posting through
aioe.org. Sorry if you see this twice.]

 

[santosh]

Keith Thompson wrote:

<snip>

It seems that the UDP on rr.com has been lifted, as I can read this post
on Motzarella.org.

 

[jacob navia]

The manual perhaps?

http://gcc.gnu.org/onlinedocs/gcc-4.2.2/gcc/Variable-Attributes.html

Yes, I saw that already, but in that "manual" there is NO MENTION of
any packed <n>

<quote>
packed
The packed attribute specifies that a variable or structure field
should have the smallest possible alignment—one byte for a variable, and
one bit for a field, unless you specify a larger value with the aligned
attribute.

Here is a structure in which the field x is packed, so that it
immediately follows a:

struct foo
{
char a;
int x[2] __attribute__ ((packed));
};
<end quote>

This means (if I read correctly) that the packed attribute is
only allowed alone, and if you want a value different than 1 you should
use the "aligned" attribute.

 

[Keith Thompson]

Since standard C doesn't provide any way for the programmer to direct
the compiler as to how to layout structures, most compilers provide some
way to do this, albeit in different forms.

Microsoft (and lcc-win) uses
#pragma pack(1)

Gcc uses
__attribute__ {(packed)}

Has anyone seen other directives in other compilers?

http://docs.sun.com/app/docs/doc/819-5265/bjach?l=en&a=view&q=pragma+pack

The Sun Studio 12 C User's Guide
show a syntax largely the same as Microsoft's
but requires #pragma pack() to end the scope.

Sun also includes two important usage notes that might not be the
same on other systems:
[...]

Note --

If you use #pragma pack to align struct or union members on
boundaries other than their natural boundaries, accessing these
fields usually leads to a bus error on SPARC. In order to avoid
such an error, be sure to also specify the -xmemalign option.
See B.2.108 -xmemalign=ab , for the optimal way to compile such
programs.

[...]

I'd consider that to be a bug in the SPARC compiler. If you declare a
structure to be packed, it's the compiler's job to generate the right
code to access the members of that structure, however they happen to
be aligned. The fact that there's a workaround for this bug doesn't
make it any less a bug.

 

[Walter Roberson]

(e-mail address removed)-cnrc.gc.ca (Walter Roberson) writes:

I'd consider that to be a bug in the SPARC compiler. If you declare a
structure to be packed, it's the compiler's job to generate the right
code to access the members of that structure, however they happen to
be aligned. The fact that there's a workaround for this bug doesn't
make it any less a bug.

Surely a QOI rather than a bug? Since C doesn't define a #pragma pack
the meaning of the directive is unconstrained and it could introduce
UB into code that would be otherwise clean.

 

[Keith Thompson]

santosh said:

I don't know Ada, but I guess I can guess, right? It is quite possible that
Ada makes structure packing portable by picking one strategy and insisting
on it. If so, well, yes, that would work. C, on the other hand, does not
do this (and in a different way, that works too, but it can be a trap for
the unwary).

I do know Ada, and that's not quite how it works.

Ada provides two ways to affect the layout of a record (equivalent to
a C struct), pragma Pack and record representation clauses.

If you specify

pragma Pack(Some_Type);

then the *recommended* behavior of the compiler is:

If a type is packed, then the implementation should try to
minimize storage allocated to objects of the type, possibly at the
expense of speed of accessing components, subject to reasonable
complexity in addressing calculations.

(There's a bit more to it than that.)

In C, a standardized "#pragma PACK" or "#pragma STDC PACK" could do
exactly the same thing. It would not impose any specific strategy,
and code that uses it could be just as portable as code that doesn't
use it, as long as it doesn't depend on the specific layout of the
structure.

It would mean that there would be two unspecified strategies for
structure layout, one to be used in the absence of the pragma, and one
to be used in its presence. It would merely be a suggestion to the
compiler, not unlike "register" or "inline". The purpose would be to
save space, not to match a particular externally imposed layout.

(An Ada record representation clause can specify the *exact* layout of
each member of a record. I don't think anyone has suggested such a
thing for C.)

Note: I'm not necessarily advocating standardizing a pack pragma for
C, just commenting on how it should be defined if it is standardized.

 

[Keith Thompson]

Keith Thompson wrote:

<snip>

It seems that the UDP on rr.com has been lifted, as I can read this post
on Motzarella.org.

I don't see it on aioe.org, though.

 

[Ben Bacarisse]

Yes, I saw that already, but in that "manual"

Why put "manual" in quotes?

there is NO MENTION of any packed <n>

Why should there be? gcc does not use that syntax.

 

[Al Balmer]

(An Ada record representation clause can specify the *exact* layout of
each member of a record. I don't think anyone has suggested such a
thing for C.)

For my purposes, that would be the only variety of any use. I
generally wouldn't care as much about saving space as forcing a
particular layout.

 

[Flash Gordon]

jacob navia wrote, On 03/12/07 19:45:

Since standard C doesn't provide any way for the programmer to direct
the compiler as to how to layout structures, most compilers provide some
way to do this, albeit in different forms.

I am writing this part of my tutorial, and I would like to make a table
about the constructs used by the various compilers. Note that under
windows you must know this kind of stuff since windows uses packed
structures extensively.

If the library headers are written properly then you should not need to
know about the packing because just including win.h (or whatever) should
sort it all out for you.

Of course, on one occasion I did debug a problem for someone at work
which was caused by a header doing this type of stuff not working
properly in Borland C++ Builder. I have vague memories that it was a bug
in the version of the header provided specifically for Borland, but it
is years ago so I could be wrong.

Anyway, I would recommend leaving this type of stuff for an "advanced"
chapter clearly labelled as covering implementation specific extensions
which should only be used when actually required (and yes, I accpet that
sometimes you need to know about this sort of thing).

Oh, and I've used compilers where there was no concept of "packed"
because there was no penalty for any alignment.

 

[Keith Thompson]

Surely a QOI rather than a bug? Since C doesn't define a #pragma pack
the meaning of the directive is unconstrained and it could introduce
UB into code that would be otherwise clean.

It's not a violation of the C standard. I still consider it to be a
bug.

 

[Chris Torek]

(As someone else noted, this should use double parentheses, not
parentheses within braces.)

http://docs.hp.com/en/B3901-90012/ch02s02.html

HP also uses

#pragma PACK n

where n is 1, 2, 4, 8, or 16. ...

As this shows, there is another problem here. What are the
constraints on the "n" in PACK n or pack(n)? Is the "n" required
to be a power of two, or is it a number that specifies a power of
two, or is it something else entirely?

There are also "align" directives, which are different from "pack"
directives; and some compilers may have "endian" directives,
especially on CPUs that are endian-ambidextrous, as it were. (The
SPARC in particular can specify an endianness with each load or
store. This endianness is XORed with the current CPU endianness
and the MMU per-page endianness; all three are separately specified.)

... on the other hand, the messiness also gives an indication as
to why it might not be as easy as expected to standardize a single
#pragma pack

Indeed.

In any case, "pack" and "align" directives are the wrong solution,
or at least, an insufficient solution. If you really want to solve
the problem in general, you should provide a complete "representation"
specification. You must decide, in the process of defining your
representation specification, whether to allow for non-contiguous
spans, how many bits can be in a value, and how many bits can be
spanned. You should also decide whether to allow for "forbidden"
combinations. For instance, if the actual bit layout is:

bits 0, 4, and 143: specify an unsigned value between 0 and 7

bit 1: unused

bits 2, 3, 5, 6, 7: specify an unsigned value between 0 and 31,
but values between 16 and 23 are invalid

bits 8 through 35: unused
bits 36 through 71: specify a signed 36-bit value (this is a
36-bit CPU)
bits 72 through 142: specify an unsigned 71-bit value

will your compiler support this, and if so, how?

 

[jameskuyper]

James Kuyper wrote: ....

I don't think I said "couldn't be done", but I may have said "a
pain" and "inefficient". Yes, there are cases when it has to be
done. But the idea is to avoid them.

You said:

Since, having packed, you can no longer index arrays of these
beasties, please explain how you are going to address 100e6 of
them? ...

You didn't say it would be a pain or inefficient for an implementation
to support indexing into an array of packed structures. You said, flat
out, that it couldn't be done.

Jacob is working with a machine that is especially forgiving of
misalignment, the X86. Also, failing to keep an eye on the code
generated for simple operations leads to C++ bloat.

Packed structures are certainly more expensive on machines that are
less forgiving of misalignment. Anyone choosing to use packed
structures should understand that they are, in effect, telling the
compiler that memory space is significantly more important to them
then execution speed; if they don't actually feel that way, that's
their mistake. I've seen compilers for several other architectures
that had this feature; I doubt that they did so purely as an
advertising gimmick. I've never tested that feature on any of those
compilers, so I can't be sure, but I presume that any such compiler
would generate code with at least minimally acceptable efficiency even
when processing data in packed arrays; otherwise they wouldn't offer
the option.

 

[CBFalconer]

Why is this non-portable? The Ada standard (and, thus, compilers)
have done this portably for years.

The last time I looked there were considerable differences between
the C standard and the Ada standard. I see no reference to packing
in the C standard.

This also illustrates the foolishness of discussing non-standard
compiler features in this newsgroup.