Making FILE opaque

Discussion in 'C Programming' started by Seebs, Feb 12, 2010.

  1. Seebs

    Seebs Guest

    It was recently pointed out that the theoretical construct:
    struct __file;
    typedef struct __file FILE;
    is not a conforming implementation (assuming no definition of struct __file),
    because FILE is specified to be an object type.

    However!

    7.19.3 Files
    6. The address of the FILE object used to control a stream may be
    significant; a copy of a FILE object need not serve in place of the
    original.

    So.

    typedef unsigned char FILE;

    If the address is significant (it must be the address of the beginning of
    some object of undisclosed nature), this allows me to refer to an arbitrarily
    complex item through a "FILE *", assures me that I can represent the address
    of any such item in a "FILE *", and hides the implementation details.

    Am I missing anything? (I mean, apart from the implicit performance hit from
    not being able to do some things as macros...)

    -s
    --
    Copyright 2010, all wrongs reversed. Peter Seebach /
    http://www.seebs.net/log/ <-- lawsuits, religion, and funny pictures
    http://en.wikipedia.org/wiki/Fair_Game_(Scientology) <-- get educated!
     
    Seebs, Feb 12, 2010
    #1
    1. Advertising

  2. Seebs <> writes:
    > It was recently pointed out that the theoretical construct:
    > struct __file;
    > typedef struct __file FILE;
    > is not a conforming implementation (assuming no definition of struct __file),
    > because FILE is specified to be an object type.
    >
    > However!
    >
    > 7.19.3 Files
    > 6. The address of the FILE object used to control a stream may be
    > significant; a copy of a FILE object need not serve in place of the
    > original.
    >
    > So.
    >
    > typedef unsigned char FILE;
    >
    > If the address is significant (it must be the address of the beginning of
    > some object of undisclosed nature), this allows me to refer to an arbitrarily
    > complex item through a "FILE *", assures me that I can represent the address
    > of any such item in a "FILE *", and hides the implementation details.
    >
    > Am I missing anything? (I mean, apart from the implicit performance hit from
    > not being able to do some things as macros...)


    Not that I can see. And a macro could conceivably cast a FILE* to
    __INTERNAL_FILE*. The __INTERNAL_FILE type would have to be visible,
    which kills some of the purpose of making FILE relatively opaque, but
    at least something like stdin->_fileno wouldn't compile.

    Or a macro could invoke some compiler-specific magic that's not
    available to ordinary users.

    --
    Keith Thompson (The_Other_Keith) <http://www.ghoti.net/~kst>
    Nokia
    "We must do something. This is something. Therefore, we must do this."
    -- Antony Jay and Jonathan Lynn, "Yes Minister"
     
    Keith Thompson, Feb 12, 2010
    #2
    1. Advertising

  3. Seebs

    Kaz Kylheku Guest

    On 2010-02-12, Seebs <> wrote:
    > It was recently pointed out that the theoretical construct:
    > struct __file;
    > typedef struct __file FILE;
    > is not a conforming implementation (assuming no definition of struct __file),
    > because FILE is specified to be an object type.
    >
    > However!
    >
    > 7.19.3 Files
    > 6. The address of the FILE object used to control a stream may be
    > significant; a copy of a FILE object need not serve in place of the
    > original.


    This doesn't buy you anything. What matters is that the construction
    and destruction of FILE is abstracted at the API level. A program cannot
    manufacture an instance of a FILE which can be passed to the library;
    all of the construction methods for a FILE return a pointer to an
    allocated object (or pre-allocated, in the case of stdin, etc).

    It's better for an implementation to keep FILE opaque as an incomplete
    type; this catches programs which do something stupid, like define
    variables of type FILE:

    FILE x;
    freopen(..., ..., &x); /* WTF is this doing? */

    Can you think of a reason to implement FILE as an object type other
    than to pass a conformance test suite which contains a ``FILE x;'' test
    case?

    > So.
    >
    > typedef unsigned char FILE;


    Rather than a character type, how about using a complete struct type to
    represent it. This can be done all in one definition;

    typedef struct __file {
    #ifdef __COMPILING_C_LIBRARY
    /* real contents */
    int __fd;
    unsigned char *__buf;
    /* ... */
    #else
    /* const? */ int __dummy;
    #endif
    } FILE;

    So now there is a different struct type, depending on whetehr you are
    in the translation units ithin the C library, or within the program.

    These pointers should be compatible (and in any case, maximal
    portability is not necessarily a concern for a library implementation).

    Pointers to different kinds of structs should be compatible, and in any
    case this is in a different translation unit.

    I've used this trick in the past to enforce opaqueness, providing
    a compiling mode in which the otherwise visible members are ifdef'd out.

    The trick can be used as a compile time device strictly; i.e. the normal
    mode of building the program reveals the members and provides
    macros/inline functions to access them. The opaque debugging mode
    catches code which bypasses the macros and inlines. (In this mode,
    the macros and inlines are disabled in favor of real external functions,
    so they don't blow up in the same way).
     
    Kaz Kylheku, Feb 12, 2010
    #3
  4. Seebs

    Seebs Guest

    On 2010-02-12, Kaz Kylheku <> wrote:
    > This doesn't buy you anything. What matters is that the construction
    > and destruction of FILE is abstracted at the API level. A program cannot
    > manufacture an instance of a FILE which can be passed to the library;
    > all of the construction methods for a FILE return a pointer to an
    > allocated object (or pre-allocated, in the case of stdin, etc).


    What it buys me is that someone who has headers, but not library source,
    can't write any plain-C code which does anything with the internals of a
    FILE.

    > Can you think of a reason to implement FILE as an object type other
    > than to pass a conformance test suite which contains a ``FILE x;'' test
    > case?


    Well, standards conformance. It is declared to be an object type.

    > These pointers should be compatible (and in any case, maximal
    > portability is not necessarily a concern for a library implementation).


    Good point.

    Anyway, the issue I'm looking at is stuff like perl's idiotic stdio-poking
    (along with the insulting "your stdio isn't very std" which is delivered
    if you don't have compatibility with some ages-old implementation choice),
    and how nice it would be to prevent this.

    -s
    --
    Copyright 2010, all wrongs reversed. Peter Seebach /
    http://www.seebs.net/log/ <-- lawsuits, religion, and funny pictures
    http://en.wikipedia.org/wiki/Fair_Game_(Scientology) <-- get educated!
     
    Seebs, Feb 12, 2010
    #4
  5. Seebs

    jacob navia Guest

    Seebs a écrit :
    > It was recently pointed out that the theoretical construct:
    > struct __file;
    > typedef struct __file FILE;
    > is not a conforming implementation (assuming no definition of struct __file),
    > because FILE is specified to be an object type.
    >

    lcc-win64 makes FILE opaque.
     
    jacob navia, Feb 12, 2010
    #5
  6. jacob navia <> writes:
    > Seebs a écrit :
    >> It was recently pointed out that the theoretical construct:
    >> struct __file;
    >> typedef struct __file FILE;
    >> is not a conforming implementation (assuming no definition of
    >> struct __file),
    >> because FILE is specified to be an object type.
    >>

    > lcc-win64 makes FILE opaque.


    Opaque in what sense? C99 7.19.1p2 specifically requires FILE to be
    an object type.

    This:

    #include <stdio.h>
    int main(void)
    {
    FILE f;
    return 0;
    }

    is a strictly conforming program that produces no output. How does
    lcc-win64 handle it?

    If you've made FILE opaque in a way that's consistent with the
    standard's requirements, that's great. If you're done so in a way
    that's not consistent with the standard's requirements, that's ok
    with me, as long as you don't claim it's conforming. (I just
    want to make it clear that this is a question, not an attack.)

    --
    Keith Thompson (The_Other_Keith) <http://www.ghoti.net/~kst>
    Nokia
    "We must do something. This is something. Therefore, we must do this."
    -- Antony Jay and Jonathan Lynn, "Yes Minister"
     
    Keith Thompson, Feb 13, 2010
    #6
  7. In article <>,
    Seebs <> wrote:
    >It was recently


    and not-so-recently

    >pointed out that the theoretical construct:
    > struct __file;
    > typedef struct __file FILE;
    >is not a conforming implementation (assuming no definition of struct __file),
    >because FILE is specified to be an object type.
    >
    >However!


    [description of bogus FILE object elided]

    What are you trying to achieve? Obviously making FILE * opaque has
    a certain elegance to it, but do you have a practical goal? Do you
    plan to conceal your source code so that no-one can work out what's
    really in the structure? After all, you have to be fairly determined
    to do anything with it even if it's openly defined in <stdio.h>.

    >Am I missing anything? (I mean, apart from the implicit performance hit from
    >not being able to do some things as macros...)


    And of course as inline functions.

    If you were to do such a thing, it would be courteous to provide
    functions for the (currently unportable) things for which knowledge of
    the FILE structure is occasionally needed, notably finding out whether
    any input characters are buffered and creating FILEs with user-supplied
    low-level i/o functions.

    -- Richard
    --
    Please remember to mention me / in tapes you leave behind.
     
    Richard Tobin, Feb 13, 2010
    #7
  8. Seebs

    Seebs Guest

    On 2010-02-13, Richard Tobin <> wrote:
    > What are you trying to achieve? Obviously making FILE * opaque has
    > a certain elegance to it, but do you have a practical goal? Do you
    > plan to conceal your source code so that no-one can work out what's
    > really in the structure? After all, you have to be fairly determined
    > to do anything with it even if it's openly defined in <stdio.h>.


    Avoiding people messing around with internal data that may not have the
    semantics they have in mind for it.

    > If you were to do such a thing, it would be courteous to provide
    > functions for the (currently unportable) things for which knowledge of
    > the FILE structure is occasionally needed, notably finding out whether
    > any input characters are buffered and creating FILEs with user-supplied
    > low-level i/o functions.


    glibc has the latter hooks. I think that could make sense. The thing is,
    I would rather people know they can't safely "find out whether input
    characters are buffered" than have them think they can and trip over
    non-obvious semantics which had no reason to be documented.

    I actually rather like the notion of a standard API for querying and/or
    interacting with the buffer past just setvbuf.

    -s
    --
    Copyright 2010, all wrongs reversed. Peter Seebach /
    http://www.seebs.net/log/ <-- lawsuits, religion, and funny pictures
    http://en.wikipedia.org/wiki/Fair_Game_(Scientology) <-- get educated!
     
    Seebs, Feb 13, 2010
    #8
  9. Seebs

    Alan Curry Guest

    In article <>,
    Keith Thompson <> wrote:
    |jacob navia <> writes:
    |> Seebs a écrit :
    |>> It was recently pointed out that the theoretical construct:
    |>> struct __file;
    |>> typedef struct __file FILE;
    |>> is not a conforming implementation (assuming no definition of
    |>> struct __file),
    |>> because FILE is specified to be an object type.
    |>>
    |> lcc-win64 makes FILE opaque.
    |
    |Opaque in what sense? C99 7.19.1p2 specifically requires FILE to be
    |an object type.

    What's the point? File this requirement under "design by committee" and
    ignore it.

    If you're looking inside the FILE type because you actually want to do
    something with the information in there, beyond what the standard
    interfaces offer, that's different.

    --
    Alan Curry
     
    Alan Curry, Feb 13, 2010
    #9
  10. Seebs

    Alan Curry Guest

    In article <>,
    Seebs <> wrote:
    |On 2010-02-13, Richard Tobin <> wrote:
    |> What are you trying to achieve? Obviously making FILE * opaque has
    |> a certain elegance to it, but do you have a practical goal? Do you
    |> plan to conceal your source code so that no-one can work out what's
    |> really in the structure? After all, you have to be fairly determined
    |> to do anything with it even if it's openly defined in <stdio.h>.
    |
    |Avoiding people messing around with internal data that may not have the
    |semantics they have in mind for it.

    You wanna start an arms race? Here's your future:

    /* SEEBS_LIBC and SEEBS_LIBC_VERSION defined by configure script, by
    strings'ing the library if you're being stubborn about that too */
    #if SEEBS_LIBC
    #if SEEBS_LIBC_VERSION <= 1003001
    #define STDIO_BUFSTARTP(f) (*(unsigned char **)((char *)(f)+16))
    #define STDIO_BUFENDP(f) (*(unsigned char **)((char *)(f)+20))
    #elif SEEBS_LIBC_VERSION <= 1004002
    #define STDIO_BUFSTARTP(f) (*(unsigned char **)((char *)(f)+28))
    #define STDIO_BUFENDP(f) (*(unsigned char **)((char *)(f)+32))
    #else
    #error "please download an updated package from seebs-libc-workarounds.org"
    #endif
    #define STDIO_BUFFER_EMPTY(f) (STDIO_BUFSTARTP(f) == STDIO_BUFENDP(f))
    #define STDIO_PEEKCHAR(f) (STDIO_BUFFER_EMPTY(f) ? EOF : *STDIO_BUFSTARTP(f))
    #endif

    --
    Alan Curry
     
    Alan Curry, Feb 13, 2010
    #10
  11. Seebs

    Seebs Guest

    On 2010-02-13, Alan Curry <> wrote:
    > You wanna start an arms race? Here's your future:


    No, just discourage people from casually assuming that they know what
    the interface really is.

    Basically, if you expose the interface, people assume they're allowed to
    use it freely, so then you're stuck with that interface -- existing code
    tends to win against proper design. But if you don't expose the interface,
    people are likely to assume that the unexposed part is *actually* subject
    to change, and not use it.

    -s
    --
    Copyright 2010, all wrongs reversed. Peter Seebach /
    http://www.seebs.net/log/ <-- lawsuits, religion, and funny pictures
    http://en.wikipedia.org/wiki/Fair_Game_(Scientology) <-- get educated!
     
    Seebs, Feb 13, 2010
    #11
  12. Seebs

    jacob navia Guest

    Alan Curry a écrit :
    > In article <>,
    > Keith Thompson <> wrote:
    > |jacob navia <> writes:
    > |> Seebs a écrit :
    > |>> It was recently pointed out that the theoretical construct:
    > |>> struct __file;
    > |>> typedef struct __file FILE;
    > |>> is not a conforming implementation (assuming no definition of
    > |>> struct __file),
    > |>> because FILE is specified to be an object type.
    > |>>
    > |> lcc-win64 makes FILE opaque.
    > |
    > |Opaque in what sense? C99 7.19.1p2 specifically requires FILE to be
    > |an object type.
    >
    > What's the point? File this requirement under "design by committee" and
    > ignore it.
    >
    > If you're looking inside the FILE type because you actually want to do
    > something with the information in there, beyond what the standard
    > interfaces offer, that's different.
    >


    I need to make it opaque because the fields, size, and many other things
    aren't fixed.

    I want to make things like

    FILE *fp = fopen("http://www.q-software-solutions.de","r");

    That would open any file in the network. I think users will be
    happier with those features than with a strictly conforming
    stuff.
     
    jacob navia, Feb 13, 2010
    #12
  13. Seebs

    Seebs Guest

    On 2010-02-13, jacob navia <> wrote:
    > I need to make it opaque because the fields, size, and many other things
    > aren't fixed.
    >
    > I want to make things like
    >
    > FILE *fp = fopen("http://www.q-software-solutions.de","r");
    >
    > That would open any file in the network. I think users will be
    > happier with those features than with a strictly conforming
    > stuff.


    Well, if you want to make it totally conforming AND do that, I've now
    revealed how! You are totally entitled to do things like convert the
    pointer between different structure types when you get to it; the standard
    is quite clear that a copy of a FILE isn't the same as the original, so
    people can't rely on being able to copy those fields or otherwise mess
    with them.

    You could have:
    typedef struct {
    unsigned char u;
    } FILE;

    then have an array of FILE:
    FILE file_table[64] = { 0 };

    Then implement stdio functions with something to the effect of:

    int
    fprintf(FILE *ext_fp, blah blah) {
    real_file_t *fp = real_file_table[ext_fp - file_table];
    ...
    }

    All that's required is that it is AN object type, not that it is THE
    object type.

    -s
    --
    Copyright 2010, all wrongs reversed. Peter Seebach /
    http://www.seebs.net/log/ <-- lawsuits, religion, and funny pictures
    http://en.wikipedia.org/wiki/Fair_Game_(Scientology) <-- get educated!
     
    Seebs, Feb 13, 2010
    #13
  14. jacob navia <> writes:
    > Alan Curry a écrit :
    >> In article <>,
    >> Keith Thompson <> wrote:
    >> |jacob navia <> writes:
    >> |> Seebs a écrit :
    >> |>> It was recently pointed out that the theoretical construct:
    >> |>> struct __file;
    >> |>> typedef struct __file FILE;
    >> |>> is not a conforming implementation (assuming no definition of
    >> |>> struct __file),
    >> |>> because FILE is specified to be an object type.
    >> |>>
    >> |> lcc-win64 makes FILE opaque.
    >> |
    >> |Opaque in what sense? C99 7.19.1p2 specifically requires FILE to be
    >> |an object type.
    >>
    >> What's the point? File this requirement under "design by committee" and
    >> ignore it.
    >>
    >> If you're looking inside the FILE type because you actually want to do
    >> something with the information in there, beyond what the standard
    >> interfaces offer, that's different.
    >>

    >
    > I need to make it opaque because the fields, size, and many other things
    > aren't fixed.
    >
    > I want to make things like
    >
    > FILE *fp = fopen("http://www.q-software-solutions.de","r");
    >
    > That would open any file in the network. I think users will be
    > happier with those features than with a strictly conforming
    > stuff.


    You didn't directly answer my question, but I suppose that last bit
    implies that your implementation isn't conforming. Not the choice I
    would have made, but it's between you and your users.

    But I'm still curious: How do you actually define FILE, and how does
    making it something other than an object type help?

    --
    Keith Thompson (The_Other_Keith) <http://www.ghoti.net/~kst>
    Nokia
    "We must do something. This is something. Therefore, we must do this."
    -- Antony Jay and Jonathan Lynn, "Yes Minister"
     
    Keith Thompson, Feb 13, 2010
    #14
  15. In article <>,
    Seebs <> wrote:

    >> If you were to do such a thing, it would be courteous to provide
    >> functions for the (currently unportable) things for which knowledge of
    >> the FILE structure is occasionally needed, notably finding out whether
    >> any input characters are buffered and creating FILEs with user-supplied
    >> low-level i/o functions.


    >glibc has the latter hooks. I think that could make sense. The thing is,
    >I would rather people know they can't safely "find out whether input
    >characters are buffered" than have them think they can and trip over
    >non-obvious semantics which had no reason to be documented.


    A common case for checking whether characters are buffered is to
    implement a stdio-level version of select() on unix-like systems (so
    that you can read from multiple input streams). You can get the
    underlying file descriptors with fileno(), and you can use the real
    select() to wait for input available on those descriptors, but if
    there are characters already buffered by stdio you may block when a
    getc() would succeed.

    -- Richard
    --
    Please remember to mention me / in tapes you leave behind.
     
    Richard Tobin, Feb 13, 2010
    #15
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. dna
    Replies:
    0
    Views:
    1,215
  2. Martin Meier

    opaque - Bedeutung ?

    Martin Meier, Sep 10, 2004, in forum: Java
    Replies:
    3
    Views:
    1,487
    Eric Sosman
    Sep 10, 2004
  3. Brian Tozer

    Semi-opaque colored background

    Brian Tozer, Feb 8, 2004, in forum: HTML
    Replies:
    11
    Views:
    4,801
    Steve Pugh
    Feb 9, 2004
  4. Beauregard T. Shagnasty

    IE hides inline images with opaque background

    Beauregard T. Shagnasty, Dec 31, 2004, in forum: HTML
    Replies:
    13
    Views:
    919
  5. Ellarco
    Replies:
    12
    Views:
    604
    Jerry Coffin
    Oct 3, 2003
Loading...

Share This Page