Structure size directives

[CJ]

Has it occurred to you that there are environments/situations where
you are more interested in getting the program to RUN is much
more important than writing it portably?

I have worked in DSPs where there was only 80K RAM, and of that 70K
was used by the modem software.

I had to run in 10K.

Or, you have to layout your structure according to EXTERNAL
requirements like passing it character-wise through a
serial interface, and at the other side you can't assume any
"alignment" wasted space, etc etc!

One of the main advantages of writing in C is that it is widely
portable. If you don't care about portability, then sure, write in
C+extensions and drop down to assembly if you want. The fact that people
sometimes need to write unportable code doesn't justify polluting
standard C with obscure implementational details.

You are mistaken.

struct foo {
char a;
double b;
char c;
};

The double will be aligned (in many machines) into an 8 byte
boundary

This example does not contradict in any way what I wrote.

Obviously the compiler has to see the declaration of the structure, so
it is imperative to put everything related to it in the declaration.
This poses no problem at all if you write your structure declarations
in header files...

And if you happen not to follow this best practice, having a function
expecting to receive an unpacked struct and actually getting a packed
struct could lead to all sorts of subtle and hard-to-find bugs. Yuck!

 

[Al Balmer]

http://docs.hp.com/en/B3901-90012/ch02s02.html

HP also uses

#pragma PACK n

where n is 1, 2, 4, 8, or 16. The reference there doesn't say how
to terminate the scope of a #pragma PACK . I have barely used HP
so I don't know if #pragma pack (lowercase) would be accepted.

Lowercase is OK. The scope is terminated by omitting the 'n',
reverting to the default alignment.

 

[Richard Tobin]

Unaligned accesses to objects, should be banned. It has nothing to do
with portable C code, it might trigger a core dump on non-x86 CPU.

Obviously on such systems, for packing to be useful, the compiler must
do the work necessary to make it work, probably by using byte acesses
or special instructions.

-- Richard

 

[jameskuyper]

CJ wrote:
....

Probably gcc/etc. want to allow the programmer to decide *for each
individual struct* whether it should be packed or not. This approach
will create terrible maintenance headaches - the pack/don't pack
directive will need to be present each time the compiler sees a
declaration of the struct.

I don't see that as a significant problem. There should be exactly one
file containing a declaration of any given struct type, which is
directly or indirectly #included in every translation unit that needs
it. Therefore only one pair of pack/unpack directives is needed for
each struct type for which this is desired.

 

[jacob navia]

CJ wrote:
...

I don't see that as a significant problem. There should be exactly one
file containing a declaration of any given struct type, which is
directly or indirectly #included in every translation unit that needs
it. Therefore only one pair of pack/unpack directives is needed for
each struct type for which this is desired.

Microsoft scopes the packing directive with nesting

#pragma pack(push)
#pragma pack(4)
....
#pragma pack(pop)

gcc needs a declaration of packing for each structure.

Of course your remark is valid, this should be in the structure type
definition.

When you want several structures packed identically the Microsoft
syntax is slightly better.

Still I do not know if you can say
__attribute__ {(packed 4)} or not...

 

[Keith Thompson]

The best way for the compiler to give the programmer a way to specifiy
this is surely through a command-line option (for example, as part of an
"optimize for size" option) and not by adding pointless new keywords to
the language. It would be very bad if programmers started making
non-portable assumptions that a struct was packed, rather than using
offsetof and the other proper mechanisms provided by C.

[...]

I don't think a command-line option is appropriate. It means that (a)
it applies to an entire translation unit, not to a single struct
declaration, and (b) it becomes *very* easy to compile one file with
the option and one without it, leading to different parts of the same
program having different ideas of how a structure is laid out. It
makes more sense for information about a type's representation to be
associated *in the source* with that type's declaration.

One possibility might be to provide a standardized pack pragma
*without* specifying exactly what it does, merely that it suggests
favoring space over performance when choosing the layout. Different
implementations, especially for different platforms, could handle it
differently, but it would be consistent within an application.
There's precedent for this in other languages (Pascal, Ada), and as an
extension in some C implementations.

Alternatively, the pack pragma (or perhaps another form of it) might
specify that there shall be *no* padding between members; member
access would have to use the equivalent of memcpy() in some cases.

 

[CBFalconer]

Its surely not a "detail".

Surely you mean "it's"?

If I have
struct foo {
int32_t val;
char *m;
};

Using normal packing rules in 64 bit systems, this makes
16 bytes, using "packed" layout it will be only
12 bytes. This is fully 25 percent of memory usage!

If I have 100 million of those, the difference is a whooping
25 million!

Since, having packed, you can no longer index arrays of these
beasties, please explain how you are going to address 100e6 of
them? Maybe you will allocate storage one at a time, and make an
array of those pointers? This will use up at least as much storage
as you have 'saved'. Of course the malloced pointers will each
waste at least as much storage as you saved in the individual
object. Net result, an extra 25% (or more) of memory usage.

BTW, whooping cranes whoop. Differences can whop.

 

[CBFalconer]

The best way for the compiler to give the programmer a way to
specifiy this is surely through a command-line option (for
example, as part of an "optimize for size" option) and not by
adding pointless new keywords to the language. It would be very
bad if programmers started making non-portable assumptions that
a struct was packed, rather than using offsetof and the other
proper mechanisms provided by C.


It means that the statement you made was obviously nonsense,
and badly thought-out.

I have come to the conclusion that Jacob looks only to the positive
aspects of a thought, and never considers the negative results or
consequences until far too late. I often do the same thing, but at
least I am occasionally willing to listen when the negatives are
pointed out.

 

[CBFalconer]

.... snip ...

Then, excuse me but WHAT you propose when you do NOT want that
a structure is aligned? How would you do it?

Specifically in the case above, where you have a structure
several million times in RAM?


OK. You have thought it out better than gcc/microsoft/sun/sgi/.

Yes. Here, at last, is a statement we can agree with.

 

[CBFalconer]

.... snip ...

Has it occurred to you that there are environments/situations
where you are more interested in getting the program to RUN is
much more important than writing it portably?

I have worked in DSPs where there was only 80K RAM, and of that
70K was used by the modem software.

I had to run in 10K.

What did you do with all that space? I have often built systems
than ran in a total of 16k or less (see the 8008) (or see some 6502
systems) (or see small CP/M 8080 systems).

I suspect you should have reworked the modem software. Should have
been easy to cut it all down to a single 64k memory space.

 

[CBFalconer]

.... snip ...

And if you happen not to follow this best practice, having a
function expecting to receive an unpacked struct and actually
getting a packed struct could lead to all sorts of subtle and
hard-to-find bugs. Yuck!

Not yuck. I am getting all sorts of useful ideas for creating
unmodifiable C programs that are totally portable. Very useful.
Once I supply a few of these to the appropriate firms I suspect I
will have my health insurance etc. taken care of for the rest of my
life.

 

[jacob navia]

Surely you mean "it's"?


Since, having packed, you can no longer index arrays of these
beasties, please explain how you are going to address 100e6 of
them?

Why I couldn't make an array of 12 byte structures???

This supposition is wrong, and the conclusions then, are
also wrong.

 

[CBFalconer]

Why I couldn't make an array of 12 byte structures???

Because the normal indexing instructions multiply by sizes, which
are expected to be a restricted set (say 1, 2, 4, 8, 16).
Otherwise each address requires loading the index and size,
multiplying, adding the base address, then performing some loading
that also depends on the size. The X86 is especially likely to use
this, and I would have thought you to be fairly conversant with its
assembly language.

This supposition is wrong, and the conclusions then, are
also wrong.

No, it may be possible to overcome, but the effort is not
worthwhile.

 

[jacob navia]

Because the normal indexing instructions multiply by sizes, which
are expected to be a restricted set (say 1, 2, 4, 8, 16).

This is wrong. You can do arrays from structures of ANY size (at least
in lcc-win)!

struct f { char data[12];};

This is a perfectly valid structure and can be stored in an array
if the user decide to do so. And yes, I will generate the
corresponding assembly.

Otherwise each address requires loading the index and size,
multiplying, adding the base address, then performing some loading
that also depends on the size. The X86 is especially likely to use
this, and I would have thought you to be fairly conversant with its
assembly language.

???
So what?

I will load the index, multiply by 12, and add the base . And it will
work.

No, it may be possible to overcome, but the effort is not
worthwhile.

So, how would you do an array of the following struct?

struct f { char data[49];};

Structures can be bigger than 16...

 

[James Kuyper]

jacob navia wrote: ....

Because the normal indexing instructions multiply by sizes, which
are expected to be a restricted set (say 1, 2, 4, 8, 16).
Otherwise each address requires loading the index and size,
multiplying, adding the base address, then performing some loading
that also depends on the size. The X86 is especially likely to use
this, and I would have thought you to be fairly conversant with its
assembly language.

I'm curious - you claim it can't be done, then describe precisely how it
is actually done. How do you reconcile having these two thoughts at the
same time?
It is doable, and has actually been done, and it is done pretty much the
way you describe. It's inherently somewhat less time-efficient than
structures whose members are aligned for time efficiency rather than
space efficiency, but memory space/execution speed trade-offs are
nothing new in the computer field.

 

[Ben Bacarisse]

Still I do not know if you can say
__attribute__ {(packed 4)} or not...

Well off-topic now, but you keep giving the wrong syntax. It is:

__attribute__ ((packed))

The double brackets allow attributes to be #defined away even when
using a pre-processor that does have variable arg macros. To get fine
control over the details of the packing you use the 'aligned' or
'aligned(n)' attribute either on the whole structure or on individial
members within it.

 

[CBFalconer]

I'm curious - you claim it can't be done, then describe precisely
how it is actually done. How do you reconcile having these two
thoughts at the same time?

It is doable, and has actually been done, and it is done pretty much
the way you describe. It's inherently somewhat less time-efficient
than structures whose members are aligned for time efficiency rather
than space efficiency, but memory space/execution speed trade-offs
are nothing new in the computer field.

I don't think I said "couldn't be done", but I may have said "a
pain" and "inefficient". Yes, there are cases when it has to be
done. But the idea is to avoid them.

Jacob is working with a machine that is especially forgiving of
misalignment, the X86. Also, failing to keep an eye on the code
generated for simple operations leads to C++ bloat.

 

[jacob navia]

Well off-topic now, but you keep giving the wrong syntax. It is:

__attribute__ ((packed))

The double brackets allow attributes to be #defined away even when
using a pre-processor that does have variable arg macros. To get fine
control over the details of the packing you use the 'aligned' or
'aligned(n)' attribute either on the whole structure or on individial
members within it.

OK. Thanks. Maybe there is some document describing this in detail?

 

[Richard]

Yes, and relying on this sort of thing is a bad idea if you're trying to
write portable code.

"if".

Clearly, to use your phrase, you are not thinking this through. The type
of packing we are talking about is invariably NOT used in a portable
scenario anyway. In addition your comment about command line is fairly
silly - only certain structs need this type of packing and not the
entire module.

 

[REH]

Rubbish. Packing for structs is an implementation detail, and any
program relying on such details will be inherently non-portable.

Why is this non-portable? The Ada standard (and, thus, compilers)
have done this portably for years.

REH