Qry : Behaviour of fgets -- ?

[Keith Thompson]

"Richard Bos" <[email protected]> a écrit dans le message de
(e-mail address removed)4all.nl... [...]

On strcpy(), yes, but on fgets()? A char * and a FILE * can only overlap
if a. you're invoking UB anyway, by scribbling wildly into the FILE
object through a mispointed char *, or b. you're invoking UB anyway, by
scribbling neatly into the FILE object using undefined and unportable
assumptions about the layout of the FILE.

FILE may even be an incomplete type, there is no guarantee that a FILE *
properly cast to a char * can be safely dereferenced at all.

No, FILE cannot be an incomplete type. C99 7.19.1p2:

FILE

which is an object type capable of recording all the information
needed to control a stream [...]

I don't think think this requirement is necessary (i.e., the standard
could just as easily have allowed, or even required, FILE to be an
incomplete type without breaking any reasonable code), but there it
is.

 

[André Gillibert]

I don't think think this requirement is necessary (i.e., the standard
could just as easily have allowed, or even required, FILE to be an
incomplete type without breaking any reasonable code), but there it
is.

I think the rationale was that there exists code that does:

FILE x;
int main(void) {
/* now &x is guaranteed to be unequal to any FILE* returned by fopen,
and thus can be used as a "special value" indicating some sort of
condition */
return 0;
}

 

[Casper H.S. Dik]

No, FILE cannot be an incomplete type. C99 7.19.1p2:

FILE

which is an object type capable of recording all the information
needed to control a stream [...]

I don't think think this requirement is necessary (i.e., the standard
could just as easily have allowed, or even required, FILE to be an
incomplete type without breaking any reasonable code), but there it
is.

Yep, when the 64 bit ABI for Solaris was defined our hopes to make
FILE completely opaque were quickly dashed by some standards
tests. (Now it's an opaque blob of memory of fixed size)

The hoops we have to jump through in 32 bit space to add additional
data are not pretty.

Casper

 

[Charlie Gordon]

No, FILE cannot be an incomplete type. C99 7.19.1p2:

FILE

which is an object type capable of recording all the information
needed to control a stream [...]

I don't think think this requirement is necessary (i.e., the standard
could just as easily have allowed, or even required, FILE to be an
incomplete type without breaking any reasonable code), but there it
is.

Yep, when the 64 bit ABI for Solaris was defined our hopes to make
FILE completely opaque were quickly dashed by some standards
tests. (Now it's an opaque blob of memory of fixed size)

The hoops we have to jump through in 32 bit space to add additional
data are not pretty.

So it cannot be an incomplete type, but 7.19.3p6 clarifies that the contents
of the FILE object can be meaningless ("The address of the FILE object used
to control a stream may be significant; a copy of a
FILE object need not serve in place of the original.") It should be even
more restrictive and invoke undefined behaviour if the contents of the FILE
object are saved to a copy and then restored to the original object.

Programs with actual FILE objects are asking for trouble.
Bending the Standard to accomodate them is debatable, I would argue against
it.

 

[Charlie Gordon]

I think the rationale was that there exists code that does:

FILE x;
int main(void) {
/* now &x is guaranteed to be unequal to any FILE* returned by fopen,
and thus can be used as a "special value" indicating some sort of
condition */
return 0;
}

I am curious how much code does this in real life. As a code reviewer, I
would forbid using actual FILE structures for this or any purpose.

Note that the same effet can be achieved without an actual FILE structure,
using a pointer instead of the address as the "special value":

static char dummy_file_object[1] = "";
FILE *dummy_file_pointer = (FILE *)&dummy_file_object;
dummy_file_pointer is guaranteed to be unequal to any FILE * returned by
fopen.

 

[CBFalconer]

.... snip ...

FILE may even be an incomplete type, there is no guarantee that a
FILE * properly cast to a char * can be safely dereferenced at all.

No, FILE cannot be an incomplete type. C99 7.19.1p2:

FILE

which is an object type capable of recording all the information
needed to control a stream [...]

I don't think think this requirement is necessary (i.e., the
standard could just as easily have allowed, or even required, FILE
to be an incomplete type without breaking any reasonable code),
but there it is.

However to be able to supply getc and putc as macros, with a FILE*
argument, it is necessary that FILE be a complete type. However it
is the height of foolishness to use this description, which would
be available in stdio.h.

 

[Richard]

My compilers exercise a random generator on each ++ (or --) call.
If two of them on the same object occur between sequence points it
uses the last random value to select a function to call. This can
be any function in any process, live or not (and requires system
specific code). This produces an amazing density of nasal demons.
Someday it may well launch WWIII. It is standard conforming.

No they don't. They either leave it as is or increments/decrements.

 

[André Gillibert]

"André Gillibert" <[email protected]> a écrit dans
le
message de news: op.tyhdvueq7pu1mk@andre...

I am curious how much code does this in real life. As a code reviewer, I
would forbid using actual FILE structures for this or any purpose.

I think very little code.

Note that the same effet can be achieved without an actual FILE
structure,
using a pointer instead of the address as the "special value":

static char dummy_file_object[1] = "";
FILE *dummy_file_pointer = (FILE *)&dummy_file_object;
dummy_file_pointer is guaranteed to be unequal to any FILE * returned by
fopen.

This seems much worse than the very ugly code I posted.
There may be alignment requirements issue. That looks like UB to me.

6.3.2.3p7 contains:
| A pointer to an object or incomplete type may be converted
| to a pointer to a different object or incomplete type.
| If the resulting pointer is not correctly aligned 57) for the
| pointed-to type, the behavior is undefined.

struct {int x;} dummy_file_object;

Has more chance to work, since 6.2.5p26 contains:
| All pointers to structure types shall have the same
| representation and alignment requirements as each other.

 

[Wojtek Lerch]

Note that the same effet can be achieved without an actual FILE structure,
using a pointer instead of the address as the "special value":

static char dummy_file_object[1] = "";
FILE *dummy_file_pointer = (FILE *)&dummy_file_object;
dummy_file_pointer is guaranteed to be unequal to any FILE * returned by
fopen.

That doesn't work if FILE has non-trivial alignment requirements.

 

[Wojtek Lerch]

However to be able to supply getc and putc as macros, with a FILE*
argument, it is necessary that FILE be a complete type.

Couldn't the macros cast their FILE* argument to a pointer to another,
complete type?

 

[Richard Tobin]

However to be able to supply getc and putc as macros, with a FILE*
argument, it is necessary that FILE be a complete type. However it
is the height of foolishness to use this description, which would
be available in stdio.h.

[/QUOTE]

Not necessarily. getc() and so on can use unsigned char * hackery in a
way which is legal and reasonable for the implementation, but not for
the user-programmer.

What "unsigned char *" hackery is needed? If FILE is an incomplete
type, getc() can just cast its argument to (say) _FILE *.

-- Richard

 

[Richard Bos]

FILE may even be an incomplete type, there is no guarantee that a
FILE * properly cast to a char * can be safely dereferenced at all.

No, FILE cannot be an incomplete type. C99 7.19.1p2:

FILE

which is an object type capable of recording all the information
needed to control a stream [...]

I don't think think this requirement is necessary (i.e., the
standard could just as easily have allowed, or even required, FILE
to be an incomplete type without breaking any reasonable code),
but there it is.

However to be able to supply getc and putc as macros, with a FILE*
argument, it is necessary that FILE be a complete type. However it
is the height of foolishness to use this description, which would
be available in stdio.h.

Not necessarily. getc() and so on can use unsigned char * hackery in a
way which is legal and reasonable for the implementation, but not for
the user-programmer.

Fix yer sig.

Richard

 

[Richard Bos]

Not necessarily. getc() and so on can use unsigned char * hackery in a
way which is legal and reasonable for the implementation, but not for
the user-programmer.

What "unsigned char *" hackery is needed? [/QUOTE]

Not necessary, but sufficient.

If FILE is an incomplete type, getc() can just cast its argument to
(say) _FILE *.

True, but in that case you might as well make FILE a complete type.

Richard

 

[Casper H.S. Dik]

So it cannot be an incomplete type, but 7.19.3p6 clarifies that the contents
of the FILE object can be meaningless ("The address of the FILE object used
to control a stream may be significant; a copy of a
FILE object need not serve in place of the original.") It should be even
more restrictive and invoke undefined behaviour if the contents of the FILE
object are saved to a copy and then restored to the original object.

Yes. But that is all well and good but we have to deal with
existing application binaries which need to continue to work
while we also try to lift some limits for new applications.

A large part of the actual trouble for exissting binaries stems
from *get* macros, the fileno() macro, etc.

Macros are really evil if you want to have future binary compatibility
and lack sufficiently effective crystal balls.

Casper

 

[Charlie Gordon]

Note that the same effet can be achieved without an actual FILE
structure, using a pointer instead of the address as the "special value":

static char dummy_file_object[1] = "";
FILE *dummy_file_pointer = (FILE *)&dummy_file_object;
dummy_file_pointer is guaranteed to be unequal to any FILE * returned by
fopen.

That doesn't work if FILE has non-trivial alignment requirements.

My mistake.

Casting any object address to (char *) is OK, but the opposite is not.
The constraint is actually much stronger than just an alignment issue.
In order the construct a legal address for a FILE object, we need to call
fopen or directly instantiate a FILE object. Anything else fails on the
DS9K, but your solution likely works everywhere else.

 

[Charlie Gordon]

Yes. But that is all well and good but we have to deal with
existing application binaries which need to continue to work
while we also try to lift some limits for new applications.

A large part of the actual trouble for exissting binaries stems
from *get* macros, the fileno() macro, etc.

Macros are really evil if you want to have future binary compatibility
and lack sufficiently effective crystal balls.

Binary compatibility is a separate issue.

Maintaining binary compatibility was the main cause for inventing shadow
structures to extend FILE objects without changing their size, structure or
location, all fixed because of getc/putc and stdin/stdout... macros.

macros or inline functions for getc/putc used to be key for program
efficiency. This is less true today, because processors are so much faster
and cpu time waste so ubiquitous.

Making FILE an incomplete type only impacts source code compatibility in
very remote cases where FILE objects are instantiated explicitly. An oddity
found only in C library test suites and not worth preserving IMHO.

Making FILE an incomplete type and not resorting to casting FILE* to some
other type to achieve inline expansion would effectively support future
binary compatibility, but the current wording of the Standard prevents that.

 

[Peter J. Holzer]

jacob navia wrote: [...]
In the case of fgets this implies: [...]
o Testing that the non-null stream points to a legal C object

Only the char * pointer would be hard to verify, the stream pointer should
be easy to check against open streams handled by the library, as any
conforming library needs to account for at least those open for write and
update:

7.19.5.2p3 If stream is a null pointer, the fflush function performs this
flushing action on all
streams for which the behavior is defined above.

C libraries used to allocate FILE structures from a static array: checking
for valid FILE pointers was pretty easy then. I would be surprised if
current implementations could no longer do this efficiently.

On many modern system there is no fixed limit on the number of open
files, so a static array isn't feasible. And once you start allocating
them individually, you either have to check all of them (which is not
efficient) or use a more complicated structure (like a binary tree) only
this check.

hp

 

[David Spencer]

Keith Thompson wrote:

No, FILE cannot be an incomplete type. C99 7.19.1p2:

FILE

which is an object type capable of recording all the information
needed to control a stream [...]

I don't think think this requirement is necessary (i.e., the
standard could just as easily have allowed, or even required, FILE
to be an incomplete type without breaking any reasonable code),
but there it is.

However to be able to supply getc and putc as macros, with a FILE*
argument, it is necessary that FILE be a complete type.

If they are macros, if they are in C and if they access the structure
members directly. Although those are traditional (and, if they are C
macros, it's hard to see the point if they do not access the structure
directly), none of them is required.

The implementor can use knowledge of the structure, while hiding it
from the user.

 

[André Gillibert]

Charlie Gordon wrote:

Making FILE an incomplete type and not resorting to casting FILE* to some
other type to achieve inline expansion would effectively support future
binary compatibility, but the current wording of the Standard prevents
that.

But the standard permits FILE to be completely opaque and safe to binary
changes.

typedef struct FILE {char dummy;} FILE;

Is permitted by the standard.

Now, as an implementor, you can have your own opaque _FILE structure, and
internally typecast FILE* to _FILE*.

It's even possible in macros.

Even though the requirement of the standard may sound stupid, I see no
good reason for violating the standard at this place.

 

[Douglas A. Gwyn]

No they don't. They either leave it as is or increments/decrements.

While CBFalconer was exaggerating, it is also not the case that
any given C compiler will "either leave it as is or increment/
decrement". Because the behavior is undefined, the compiler
can generate code that is efficient in the well-defined case
but which happens to "misbehave" in the undefined case. That
misbehavior often is deterministic, but might not be either of
the behaviors you stated.