Why is initializing a FAM invalid?

[Johannes Schaub (litb)]

The C99 Standard at 6.7.2.1/18 says the following is invalid

struct s { int x; int d[]; } e = { 0, { 1, 2 } };

"because struct s is treated as if it did not contain member d.". I
understand that this is just non-normative example text, so I was looking
for normative text about it. 6.7.8 about initialization does not state that
FAMs are ignored. Where is such a thing stated? 6.7.2.1/16 says "In most
situations, the flexible array member is ignored.", which doesn't forbid the
initialization either.

 

[Marcin Grzegorczyk]

The C99 Standard at 6.7.2.1/18 says the following is invalid

struct s { int x; int d[]; } e = { 0, { 1, 2 } };

"because struct s is treated as if it did not contain member d.". I
understand that this is just non-normative example text, so I was looking
for normative text about it. 6.7.8 about initialization does not state that
FAMs are ignored. Where is such a thing stated? 6.7.2.1/16 says "In most
situations, the flexible array member is ignored.", which doesn't forbid the
initialization either.

Or it does, depending on how you look at it...

6.7.2.1p16 is, IMHO, rather poorly worded. I believe the intention of
the Committee was "Unless explicitly specified otherwise, we pretend
that the flexible array member does not exist"(*). Initialization is
not covered by 6.7.2.1p16, and FAMs are not mentioned anywhere else,
hence an attempt to initialize the FAM is a violation of 6.7.8p2. Yes,
it does suck. :-(

(*) Obviously, this is not an attempt to provide an improved wording for
C1X. ;-) And on a more serious note, this interpretation does leave at
least one nasty hole in the specification (related to type
compatibility), but that's a different subject.

 

[James Kuyper]

The C99 Standard at 6.7.2.1/18 says the following is invalid

struct s { int x; int d[]; } e = { 0, { 1, 2 } };

"because struct s is treated as if it did not contain member d.". I
understand that this is just non-normative example text, so I was looking
for normative text about it. 6.7.8 about initialization does not state that
FAMs are ignored. Where is such a thing stated? 6.7.2.1/16 says "In most
situations, the flexible array member is ignored.", which doesn't forbid the
initialization either.

No, but it then goes on to say something else that causes serious problems:
"the size of the structure is as if the flexible array member were
omitted except that it may have more trailing padding than the omission
would imply."

Later on, it says: "when a . (or ->) operator has a left operand that is
(a pointer to) a structure with a flexible array member and the right
operand names that member, it behaves as if that member were replaced
with the longest array (with the same element type) that would not make
the structure larger than the object being accessed; the offset of the
array shall remain that of the flexible array member, even if this would
differ from that of the replacement array. If this array would have no
elements, it behaves as if it had one element but the behavior is
undefined if any attempt is made to access that element or to generate a
pointer one past it."

The consequence of this is that, since the compiler only sets aside
sizeof(struct s) bytes for e, then unless that size includes a
sufficient amount of padding, the replacement array will have zero
elements, which would mean that there is no such object as e.d[0] or
e.d[1]. In that case, your initializer violates the constraint in
6.7.8p1: "No initializer shall attempt to provide a value for an object
not contained within the entity being initialized."

That's why flexible array members are normally dynamically allocated. In
that case, the relevant object is the entire block of memory that was
allocated, which can (with care) be arranged to be big enough for
everything to work right.

 

[Marcin Grzegorczyk]

[...] 6.7.2.1/16 says "In most
situations, the flexible array member is ignored.", which doesn't
forbid the initialization either.

No, but it then goes on to say something else that causes serious problems:
"the size of the structure is as if the flexible array member were
omitted except that it may have more trailing padding than the omission
would imply."
[snip 6.7.2.1p16 quote]

The consequence of this is that, since the compiler only sets aside
sizeof(struct s) bytes for e, then unless that size includes a
sufficient amount of padding, the replacement array will have zero
elements, which would mean that there is no such object as e.d[0] or
e.d[1].

Of course, the compiler could postpone structure allocation until it has
seen the whole initializer list -- much like it has to do when
initializing an array of unknown size. GCC, for example, allows FAM
initialization as an extension.

In that case, your initializer violates the constraint in
6.7.8p1: "No initializer shall attempt to provide a value for an object
not contained within the entity being initialized."

(that's 6.7.8p2, not p1)
The constraint is violated because 6.7.2.1p16 says "In most situations,
the flexible array member is ignored", and nothing in the Standard
suggests that initialization is one of the "other" situations. At
least, that seems to be the intent of the rather vague wording.

As an unfortunate consequence of all that, it is impossible to create
portably a static object that could be used as a structure with a
non-empty FAM.

 

[David R Tribble]

As an unfortunate consequence of all that, it is impossible to create
portably a static object that could be used as a structure with a
non-empty FAM.

Off the top of my head [so I'm not really sure if this is conforming
or not]...

struct VarStruct
{
int nelems;
float vals[];
};

struct FixStruct
{
int nelems;
float vals[3];
};

union InitStruct
{
struct FixStruct fs;
struct VarStruct vs;
};

static union InitStruct my_struct =
{ { 3, { 1.0f, 2.0f, 3.0f } } };

You then access the FAM member of struct VarStruct as:

... my_struct.vs.vals ...

#define my_init_struct my_struct.vs

... my_init_struct.f ...

-drt

 

[Marcin Grzegorczyk]

As an unfortunate consequence of all that, it is impossible to create
portably a static object that could be used as a structure with a
non-empty FAM.

Off the top of my head [so I'm not really sure if this is conforming
or not]...

struct VarStruct
{
int nelems;
float vals[];
};

struct FixStruct
{
int nelems;
float vals[3];
};

union InitStruct
{
struct FixStruct fs;
struct VarStruct vs;
};

static union InitStruct my_struct =
{ { 3, { 1.0f, 2.0f, 3.0f } } };

You then access the FAM member of struct VarStruct as:

... my_struct.vs.vals ...

Not strictly conforming, because of this little clause in 6.7.2.1p16:

# [...] the offset of the array shall remain that of the flexible array
# member, even if this would differ from that of the replacement array.

This implies you can't rely on offsetof(union InitStruct, fs.vals) being
equal to offsetof(union InitStruct, vs.vals). The special guarantee of
6.5.2.3p5 does not necessarily help: one can argue that the FAM is
ignored "in most situations", which presumably covers the common initial
sequence as well.

This brings up an important but so far unaddressed question of how the
wording of 6.7.2.1p16 relates to that of 6.2.7p1 and 6.5.2.3p5. (Thanks
to Nick Maclaren for pointing this out to me.) Given the following
declarations:

struct s1 { int nelems; float vals[]; };
struct s2 { int n; float v[]; };
union u1 { struct s1 m1; struct s2 m2; };

is the flexible array member a part of the common initial sequence? The
vague wording of 6.7.2.1p16 seems to imply it isn't (it's "ignored"),
even though common sense dictates it should be.

Now let's consider the following declarations:

//----- translation unit 1 -----
struct s { int nelems; float vals[]; };

//----- translation unit 2 -----
struct s { int nelems; float vals[3]; };

//----- translation unit 3 -----
struct s { int nelems; };

Is the `struct s` declared in the translation unit 1 compatible with any
of the other two?

If we take the rules of cross-translation-unit compatibility of
structure types spelled out in 6.2.7p1 at face value, and combine them
with the rules of compatibility for array types (6.7.5.2p6), the `struct
s` declared in TU 1 should be compatible with the `struct s` declared in
TU 2.

But then, 6.7.2.1p16 says that "in most situations, the flexible array
member is ignored", and type compatibility is not mentioned as an
exception, so perhaps the `struct s` declared in TU 1 should be
compatible with the `struct s` declared in TU 3.

But then, the rest of 6.7.2.1p16 pretty much invalidates both. If the
offset of the flexible array member need not equal the offset of the
ordinary array member in an otherwise identical structure, then the
structure type from TU 1 cannot really be compatible with the structure
type from TU 2; and if the former "may have more trailing padding than
the omission would imply", it cannot really be compatible with the
structure type from TU 3 (think, e.g., of the implications for function
parameters declared as `struct s`).

In other words, it appears that as far as 6.2.7p1 (and 6.5.2.3p5) is
concerned, flexible array members are *not* ignored, and they are not
compatible with ordinary array members, 6.7.5.2p6 notwithstanding.

I believe this to be a defect in C99.

 

[Wojtek Lerch]

Of course, the compiler could postpone structure allocation until it has
seen the whole initializer list -- much like it has to do when
initializing an array of unknown size. GCC, for example, allows FAM
initialization as an extension.

What does sizeof return when applied to the name of such object? Will se0,
se1 and se2 all have the same value in this example:

extern struct s { int x; int d[]; } e;
size_t se0 = sizeof(struct s);
size_t se1 = sizeof e;
struct s e = { 0, { 1, 2 } };
size_t se2 = sizeof e;

 

[Wojtek Lerch]

What does sizeof return when applied to the name of such object?

To answer my own question, sizeof does not include the FAM. The analogy
with arrays of unknown size is bogus.

 

[Seebs]

To answer my own question, sizeof does not include the FAM. The analogy
with arrays of unknown size is bogus.

There was some quirk to do with sizeof(foo) vs. offsetof(struct foo, fam)
where something changed in one of the TCs.

My amazing memory for details is clearly at its finest today.

-s

 

[Marcin Grzegorczyk]

To answer my own question, sizeof does not include the FAM. The analogy
with arrays of unknown size is bogus.

The analogy was supposed to apply to how a compiler determines the size
of an object, and my point was that there are situations where the size
is known only after the initialization has been parsed. This does not
have to do anything with sizeof.

Of course, as far as C semantics are concerned, the analogy is indeed
bogus. A flexible array member is almost, but not quite, completely
unlike an array of unknown size.

 

[Marcin Grzegorczyk]

There was some quirk to do with sizeof(foo) vs. offsetof(struct foo, fam)
where something changed in one of the TCs.

You're thinking of DR #282 (fixed in TC2), I guess.

 

[Wojtek Lerch]

The analogy was supposed to apply to how a compiler determines the size of
an object, and my point was that there are situations where the size is
known only after the initialization has been parsed.

But even that analogy is not very good: in the case of normal arrays, the
initializer determines the *type* of the object, and then the size follows
from the type in the usual way. In the case of a structure with a FAM, the
type of the object is the struct, and the size of that type does not include
the array (or at least not necessarily all of it). Since the size of the
object, as reported by sizeof, doesn't include the array either, one could
argue that the array doesn't really belong to the declared object, but to a
larger unnamed object that contains both the struct and the array. (That's,
BTW, more or less how this extension is explained by GCC documentation.)

Presumably the assignment operator agress with sizeof about how many bytes
need to be copied?

This does not
have to do anything with sizeof.

Other than the fact that sizeof is supposed to report the size of the
object.

 

[Johannes Schaub (litb)]

Of course, the compiler could postpone structure allocation until it has
seen the whole initializer list -- much like it has to do when
initializing an array of unknown size. GCC, for example, allows FAM
initialization as an extension.

What does sizeof return when applied to the name of such object? Will
se0, se1 and se2 all have the same value in this example:

extern struct s { int x; int d[]; } e;
size_t se0 = sizeof(struct s);
size_t se1 = sizeof e;
struct s e = { 0, { 1, 2 } };
size_t se2 = sizeof e;

From the C99 wording it follows that the size is as if the FAM is omitted,
except that it may have more trailing padding than the omission would imply.

Easy matter dude

 

[Johannes Schaub (litb)]

But even that analogy is not very good: in the case of normal arrays, the
initializer determines the *type* of the object, and then the size follows
from the type in the usual way. In the case of a structure with a FAM,
the type of the object is the struct, and the size of that type does not
include
the array (or at least not necessarily all of it). Since the size of the
object, as reported by sizeof, doesn't include the array either, one could
argue that the array doesn't really belong to the declared object, but to
a
larger unnamed object that contains both the struct and the array.
(That's, BTW, more or less how this extension is explained by GCC
documentation.)

I think the C99 Standard makes no difference between non-subobjects and
subobjects. It just says

"Each brace-enclosed initializer list has an associated current object. When
no designations are present, subobjects of the current object are
initialized in order according to the type of the current object [...]"

"If an array of unknown size is initialized, its size is determined by the
largest indexed element with an explicit initializer. At the end of its
initializer list, the array no longer has incomplete type."

I think this states that the FAM's size is correctly determined by that
procedure.

Other than the fact that sizeof is supposed to report the size of the
object.

It can't report the size of objects, because that is only known at runtime.
It therefor only reports "from the type of the operand.".

 

[Johannes Schaub (litb)]

But even that analogy is not very good: in the case of normal arrays, the
initializer determines the *type* of the object, and then the size follows
from the type in the usual way. In the case of a structure with a FAM,
the type of the object is the struct, and the size of that type does not
include
the array (or at least not necessarily all of it). Since the size of the
object, as reported by sizeof, doesn't include the array either, one could
argue that the array doesn't really belong to the declared object, but to
a
larger unnamed object that contains both the struct and the array.
(That's, BTW, more or less how this extension is explained by GCC
documentation.)

I think the C99 Standard makes no difference between non-subobjects and
subobjects. It just says

"Each brace-enclosed initializer list has an associated current object. When
no designations are present, subobjects of the current object are
initialized in order according to the type of the current object [...]"

"If an array of unknown size is initialized, its size is determined by the
largest indexed element with an explicit initializer. At the end of its
initializer list, the array no longer has incomplete type."

I think this states that the FAM's size is correctly determined by that
procedure.

Other than the fact that sizeof is supposed to report the size of the
object.

It can't report the size of objects, because that is only known at runtime.
It therefor only reports "from the type of the operand.".

 

[Johannes Schaub (litb)]

Of course, the compiler could postpone structure allocation until it has
seen the whole initializer list -- much like it has to do when
initializing an array of unknown size. GCC, for example, allows FAM
initialization as an extension.

What does sizeof return when applied to the name of such object? Will
se0, se1 and se2 all have the same value in this example:

extern struct s { int x; int d[]; } e;
size_t se0 = sizeof(struct s);
size_t se1 = sizeof e;
struct s e = { 0, { 1, 2 } };
size_t se2 = sizeof e;

From the C99 wording it follows that the size is as if the FAM is omitted,
except that it may have more trailing padding than the omission would imply.

Easy matter dude

 

[Wojtek Lerch]

Wojtek Lerch wrote:
[...] GCC, for example, allows FAM
initialization as an extension. [...]

What does sizeof return when applied to the name of such object?

To answer my own question, sizeof does not include the FAM. The analogy
with arrays of unknown size is bogus.

The analogy was supposed to apply to how a compiler determines the size
of an object, and my point was that there are situations where the size
is known only after the initialization has been parsed.

But even that analogy is not very good: in the case of normal arrays, the
initializer determines the *type* of the object, and then the size follows
from the type in the usual way. In the case of a structure with a FAM,
the type of the object is the struct, and the size of that type does not
include
the array (or at least not necessarily all of it). Since the size of the
object, as reported by sizeof, doesn't include the array either, one could
argue that the array doesn't really belong to the declared object, but to
a
larger unnamed object that contains both the struct and the array.
(That's, BTW, more or less how this extension is explained by GCC
documentation.)

I think the C99 Standard [...]

You do realize that the discussion was not about a standard feature, but
a GCC extension, right?

[...] makes no difference between non-subobjects and
subobjects. It just says

"Each brace-enclosed initializer list has an associated current object. When
no designations are present, subobjects of the current object are
initialized in order according to the type of the current object [...]"

Um... It says "subobjects" right there, and doesn't mention
non-subobjects. Is that not a difference?

"If an array of unknown size is initialized, its size is determined by the
largest indexed element with an explicit initializer. At the end of its
initializer list, the array no longer has incomplete type."

I think this states that the FAM's size is correctly determined by that
procedure.

Does it? What is the type of the FAM before and after the initializer?

It can't report the size of objects, because that is only known at runtime.
It therefor only reports "from the type of the operand.".

Right. But does that mean that an object declared using the GCC
extension is bigger than its type, or does it mean that the declared
object is the size of its type but is a subobject of a bigger, unnamed
object that the compiler allocates to accommodate for the initialized
FAM? The documentation is not completely clear on that, and of course
C99 is silent on the matter.

 

[Wojtek Lerch]

[...] GCC, for example, allows FAM
initialization as an extension.

What does sizeof return when applied to the name of such object? Will
se0, se1 and se2 all have the same value in this example:

extern struct s { int x; int d[]; } e;
size_t se0 = sizeof(struct s);
size_t se1 = sizeof e;
struct s e = { 0, { 1, 2 } };
size_t se2 = sizeof e;

From the C99 wording it follows that the size is as if the FAM is omitted,
except that it may have more trailing padding than the omission would imply.

I thought that from the C99 wording it follows that the above is a
constraint violation.

 

[Marcin Grzegorczyk]

But even that analogy is not very good: in the case of normal arrays,
the initializer determines the *type* of the object, and then the size
follows from the type in the usual way. In the case of a structure with
a FAM, the type of the object is the struct, and the size of that type
does not include the array (or at least not necessarily all of it).
Since the size of the object, as reported by sizeof, doesn't include the
array either, one could argue that the array doesn't really belong to
the declared object, but to a larger unnamed object that contains both
the struct and the array.

Yeah, that's more or less what 6.7.2.1p16 seems to imply.

When I said "the size of an object" above, I meant the size allocated by
the compiler (because that was what the post to which I was originally
replying mentioned), not the size defined by the semantics of C99. They
need not be the same.

Presumably the assignment operator agress with sizeof about how many
bytes need to be copied?

This seems to be implied by the "in most situations, the flexible array
member is ignored" wording, too. Note that the assignment operator is
allowed to copy structures member-by-member instead of byte-by-byte
(though most compilers choose the latter).

 

[Michael Foukarakis]

The C99 Standard at 6.7.2.1/18 says the following is invalid

    struct s { int x; int d[]; } e = { 0, { 1, 2 } };

"because struct s is treated as if it did not contain member d.". I
understand that this is just non-normative example text, so I was looking
for normative text about it. 6.7.8 about initialization does not state that
FAMs are ignored. Where is such a thing stated? 6.7.2.1/16 says "In most
situations, the flexible array member is ignored.", which doesn't forbid the
initialization either.

One purpose FAMs can fulfill is mapping structures over data, for
examples network packet headers over network data, intended for
analysis. In those cases, pre-FAM, one would declare a pointer inside
the structure and made sure it pointed to the right spot. I suppose
I'm thinking of this use case of FAMs like a convenience for
addressing over memory that is allocated by "someone else". The
problem here, as I see it, is that {1, 2} has no memory allocated for
it, and is therefore impossible to initialize the struct with it.

My quibble is why

struct s { int x; char d[]; } e = { 0, "foo" };

produces the same error, as there is at least one way to resolve it
without conflicting with the wording of the standard.

2c