Making FILE opaque

[Seebs]

It was recently pointed out that the theoretical construct:
struct __file;
typedef struct __file FILE;
is not a conforming implementation (assuming no definition of struct __file),
because FILE is specified to be an object type.

However!

7.19.3 Files
6. The address of the FILE object used to control a stream may be
significant; a copy of a FILE object need not serve in place of the
original.

So.

typedef unsigned char FILE;

If the address is significant (it must be the address of the beginning of
some object of undisclosed nature), this allows me to refer to an arbitrarily
complex item through a "FILE *", assures me that I can represent the address
of any such item in a "FILE *", and hides the implementation details.

Am I missing anything? (I mean, apart from the implicit performance hit from
not being able to do some things as macros...)

-s

 

[Keith Thompson]

It was recently pointed out that the theoretical construct:
struct __file;
typedef struct __file FILE;
is not a conforming implementation (assuming no definition of struct __file),
because FILE is specified to be an object type.

However!

7.19.3 Files
6. The address of the FILE object used to control a stream may be
significant; a copy of a FILE object need not serve in place of the
original.

So.

typedef unsigned char FILE;

If the address is significant (it must be the address of the beginning of
some object of undisclosed nature), this allows me to refer to an arbitrarily
complex item through a "FILE *", assures me that I can represent the address
of any such item in a "FILE *", and hides the implementation details.

Am I missing anything? (I mean, apart from the implicit performance hit from
not being able to do some things as macros...)

Not that I can see. And a macro could conceivably cast a FILE* to
__INTERNAL_FILE*. The __INTERNAL_FILE type would have to be visible,
which kills some of the purpose of making FILE relatively opaque, but
at least something like stdin->_fileno wouldn't compile.

Or a macro could invoke some compiler-specific magic that's not
available to ordinary users.

 

[Kaz Kylheku]

It was recently pointed out that the theoretical construct:
struct __file;
typedef struct __file FILE;
is not a conforming implementation (assuming no definition of struct __file),
because FILE is specified to be an object type.

However!

7.19.3 Files
6. The address of the FILE object used to control a stream may be
significant; a copy of a FILE object need not serve in place of the
original.

This doesn't buy you anything. What matters is that the construction
and destruction of FILE is abstracted at the API level. A program cannot
manufacture an instance of a FILE which can be passed to the library;
all of the construction methods for a FILE return a pointer to an
allocated object (or pre-allocated, in the case of stdin, etc).

It's better for an implementation to keep FILE opaque as an incomplete
type; this catches programs which do something stupid, like define
variables of type FILE:

FILE x;
freopen(..., ..., &x); /* WTF is this doing? */

Can you think of a reason to implement FILE as an object type other
than to pass a conformance test suite which contains a ``FILE x;'' test
case?

So.

typedef unsigned char FILE;

Rather than a character type, how about using a complete struct type to
represent it. This can be done all in one definition;

typedef struct __file {
#ifdef __COMPILING_C_LIBRARY
/* real contents */
int __fd;
unsigned char *__buf;
/* ... */
#else
/* const? */ int __dummy;
#endif
} FILE;

So now there is a different struct type, depending on whetehr you are
in the translation units ithin the C library, or within the program.

These pointers should be compatible (and in any case, maximal
portability is not necessarily a concern for a library implementation).

Pointers to different kinds of structs should be compatible, and in any
case this is in a different translation unit.

I've used this trick in the past to enforce opaqueness, providing
a compiling mode in which the otherwise visible members are ifdef'd out.

The trick can be used as a compile time device strictly; i.e. the normal
mode of building the program reveals the members and provides
macros/inline functions to access them. The opaque debugging mode
catches code which bypasses the macros and inlines. (In this mode,
the macros and inlines are disabled in favor of real external functions,
so they don't blow up in the same way).

 

[Seebs]

This doesn't buy you anything. What matters is that the construction
and destruction of FILE is abstracted at the API level. A program cannot
manufacture an instance of a FILE which can be passed to the library;
all of the construction methods for a FILE return a pointer to an
allocated object (or pre-allocated, in the case of stdin, etc).

What it buys me is that someone who has headers, but not library source,
can't write any plain-C code which does anything with the internals of a
FILE.

Can you think of a reason to implement FILE as an object type other
than to pass a conformance test suite which contains a ``FILE x;'' test
case?

Well, standards conformance. It is declared to be an object type.

These pointers should be compatible (and in any case, maximal
portability is not necessarily a concern for a library implementation).

Good point.

Anyway, the issue I'm looking at is stuff like perl's idiotic stdio-poking
(along with the insulting "your stdio isn't very std" which is delivered
if you don't have compatibility with some ages-old implementation choice),
and how nice it would be to prevent this.

-s

 

[jacob navia]

Seebs a écrit :

It was recently pointed out that the theoretical construct:
struct __file;
typedef struct __file FILE;
is not a conforming implementation (assuming no definition of struct __file),
because FILE is specified to be an object type.

lcc-win64 makes FILE opaque.

 

[Keith Thompson]

Seebs a écrit :
lcc-win64 makes FILE opaque.

Opaque in what sense? C99 7.19.1p2 specifically requires FILE to be
an object type.

This:

#include <stdio.h>
int main(void)
{
FILE f;
return 0;
}

is a strictly conforming program that produces no output. How does
lcc-win64 handle it?

If you've made FILE opaque in a way that's consistent with the
standard's requirements, that's great. If you're done so in a way
that's not consistent with the standard's requirements, that's ok
with me, as long as you don't claim it's conforming. (I just
want to make it clear that this is a question, not an attack.)

 

[Richard Tobin]

It was recently

and not-so-recently

pointed out that the theoretical construct:
struct __file;
typedef struct __file FILE;
is not a conforming implementation (assuming no definition of struct __file),
because FILE is specified to be an object type.

However!

[description of bogus FILE object elided]

What are you trying to achieve? Obviously making FILE * opaque has
a certain elegance to it, but do you have a practical goal? Do you
plan to conceal your source code so that no-one can work out what's
really in the structure? After all, you have to be fairly determined

Am I missing anything? (I mean, apart from the implicit performance hit from
not being able to do some things as macros...)

And of course as inline functions.

If you were to do such a thing, it would be courteous to provide
functions for the (currently unportable) things for which knowledge of
the FILE structure is occasionally needed, notably finding out whether
any input characters are buffered and creating FILEs with user-supplied
low-level i/o functions.

-- Richard

 

[Seebs]

What are you trying to achieve? Obviously making FILE * opaque has
a certain elegance to it, but do you have a practical goal? Do you
plan to conceal your source code so that no-one can work out what's
really in the structure? After all, you have to be fairly determined
to do anything with it even if it's openly defined in <stdio.h>.

Avoiding people messing around with internal data that may not have the
semantics they have in mind for it.

If you were to do such a thing, it would be courteous to provide
functions for the (currently unportable) things for which knowledge of
the FILE structure is occasionally needed, notably finding out whether
any input characters are buffered and creating FILEs with user-supplied
low-level i/o functions.

glibc has the latter hooks. I think that could make sense. The thing is,
I would rather people know they can't safely "find out whether input
characters are buffered" than have them think they can and trip over
non-obvious semantics which had no reason to be documented.

I actually rather like the notion of a standard API for querying and/or
interacting with the buffer past just setvbuf.

-s

 

[Alan Curry]

|>> It was recently pointed out that the theoretical construct:
|>> struct __file;
|>> typedef struct __file FILE;
|>> is not a conforming implementation (assuming no definition of
|>> struct __file),
|>> because FILE is specified to be an object type.
|>>
|> lcc-win64 makes FILE opaque.
|
|Opaque in what sense? C99 7.19.1p2 specifically requires FILE to be
|an object type.

What's the point? File this requirement under "design by committee" and
ignore it.

If you're looking inside the FILE type because you actually want to do
something with the information in there, beyond what the standard
interfaces offer, that's different.

 

[Alan Curry]

|> What are you trying to achieve? Obviously making FILE * opaque has
|> a certain elegance to it, but do you have a practical goal? Do you
|> plan to conceal your source code so that no-one can work out what's
|> really in the structure? After all, you have to be fairly determined
|> to do anything with it even if it's openly defined in <stdio.h>.
|
|Avoiding people messing around with internal data that may not have the
|semantics they have in mind for it.

You wanna start an arms race? Here's your future:

/* SEEBS_LIBC and SEEBS_LIBC_VERSION defined by configure script, by
strings'ing the library if you're being stubborn about that too */
#if SEEBS_LIBC
#if SEEBS_LIBC_VERSION <= 1003001
#define STDIO_BUFSTARTP(f) (*(unsigned char **)((char *)(f)+16))
#define STDIO_BUFENDP(f) (*(unsigned char **)((char *)(f)+20))
#elif SEEBS_LIBC_VERSION <= 1004002
#define STDIO_BUFSTARTP(f) (*(unsigned char **)((char *)(f)+28))
#define STDIO_BUFENDP(f) (*(unsigned char **)((char *)(f)+32))
#else
#error "please download an updated package from seebs-libc-workarounds.org"
#endif
#define STDIO_BUFFER_EMPTY(f) (STDIO_BUFSTARTP(f) == STDIO_BUFENDP(f))
#define STDIO_PEEKCHAR(f) (STDIO_BUFFER_EMPTY(f) ? EOF : *STDIO_BUFSTARTP(f))
#endif

 

[Seebs]

You wanna start an arms race? Here's your future:

No, just discourage people from casually assuming that they know what
the interface really is.

Basically, if you expose the interface, people assume they're allowed to
use it freely, so then you're stuck with that interface -- existing code
tends to win against proper design. But if you don't expose the interface,
people are likely to assume that the unexposed part is *actually* subject
to change, and not use it.

-s

 

[jacob navia]

Alan Curry a écrit :

|>> It was recently pointed out that the theoretical construct:
|>> struct __file;
|>> typedef struct __file FILE;
|>> is not a conforming implementation (assuming no definition of
|>> struct __file),
|>> because FILE is specified to be an object type.
|>>
|> lcc-win64 makes FILE opaque.
|
|Opaque in what sense? C99 7.19.1p2 specifically requires FILE to be
|an object type.

What's the point? File this requirement under "design by committee" and
ignore it.

If you're looking inside the FILE type because you actually want to do
something with the information in there, beyond what the standard
interfaces offer, that's different.

I need to make it opaque because the fields, size, and many other things
aren't fixed.

I want to make things like

FILE *fp = fopen("http://www.q-software-solutions.de","r");

That would open any file in the network. I think users will be
happier with those features than with a strictly conforming
stuff.

 

[Seebs]

I need to make it opaque because the fields, size, and many other things
aren't fixed.

I want to make things like

FILE *fp = fopen("http://www.q-software-solutions.de","r");

That would open any file in the network. I think users will be
happier with those features than with a strictly conforming
stuff.

Well, if you want to make it totally conforming AND do that, I've now
revealed how! You are totally entitled to do things like convert the
pointer between different structure types when you get to it; the standard
is quite clear that a copy of a FILE isn't the same as the original, so
people can't rely on being able to copy those fields or otherwise mess
with them.

You could have:
typedef struct {
unsigned char u;
} FILE;

then have an array of FILE:
FILE file_table[64] = { 0 };

Then implement stdio functions with something to the effect of:

int
fprintf(FILE *ext_fp, blah blah) {
real_file_t *fp = real_file_table[ext_fp - file_table];
...
}

All that's required is that it is AN object type, not that it is THE
object type.

-s

 

[Keith Thompson]

Alan Curry a écrit :

I need to make it opaque because the fields, size, and many other things
aren't fixed.

I want to make things like

FILE *fp = fopen("http://www.q-software-solutions.de","r");

That would open any file in the network. I think users will be
happier with those features than with a strictly conforming
stuff.

You didn't directly answer my question, but I suppose that last bit
implies that your implementation isn't conforming. Not the choice I
would have made, but it's between you and your users.

But I'm still curious: How do you actually define FILE, and how does
making it something other than an object type help?

 

[Richard Tobin]

If you were to do such a thing, it would be courteous to provide
functions for the (currently unportable) things for which knowledge of
the FILE structure is occasionally needed, notably finding out whether
any input characters are buffered and creating FILEs with user-supplied
low-level i/o functions.

[/QUOTE]

glibc has the latter hooks. I think that could make sense. The thing is,
I would rather people know they can't safely "find out whether input
characters are buffered" than have them think they can and trip over
non-obvious semantics which had no reason to be documented.

A common case for checking whether characters are buffered is to
implement a stdio-level version of select() on unix-like systems (so
that you can read from multiple input streams). You can get the
underlying file descriptors with fileno(), and you can use the real
select() to wait for input available on those descriptors, but if
there are characters already buffered by stdio you may block when a
getc() would succeed.

-- Richard