Building extensibility into an API

Discussion in 'C Programming' started by Francis Johnson, Oct 27, 2007.

  1. I've been having a discussion with some folks here about the wisdom or
    otherwise of "building extensibility into an API".

    Consider your favorite function in one of your libraries, perhaps
    void foo(int bar, char *baz);

    Now maybe you think of some cool improvement to foo that you could make,
    but this relies on being able to pass an additional parameter to foo.
    You now have some unattractive choices:
    1) change the signature of foo and break API compatability between
    versions of the library
    2) have a new function
    foo_v2(int bar, char *baz, long double newparam);
    and make foo a wrapper for foo_v2.
    3) don't make the improvement.

    Now, this dilemma would never have arisen if you'd "built extensibility
    into the API" in the following way.

    Each function you write takes an unused void * parameter. So foo starts
    out as
    void foo(int bar, char *baz, void *p);
    and the last parameter should be NULL.

    Now, consider extending the parameters of foo. You now use a
    foo_v2_extra_params structure. Its first field encodes the additional
    parameters, and the remaining fields are these parameters. Then version
    2 of foo can do this:

    void foo(int bar, char *baz, void *p)
    {
    /* define additional params as auto variables */
    if(p)
    /* process *p, set up extra args */
    else
    /* set additional params to defaults */
    /* do stuff */
    }

    If foo improves again, the struct just gets enlarged, and the function
    casts p appropriately before proceeding.

    Have people here implemented something like this in the past? What do
    you think of it as a solution to the "API extension" problem?

    --
    Red danger: http://www.mindszenty.org
    Francis Johnson, Oct 27, 2007
    #1
    1. Advertising

  2. Francis Johnson

    Eric Sosman Guest

    Francis Johnson wrote:
    > I've been having a discussion with some folks here about the wisdom or
    > otherwise of "building extensibility into an API".
    >
    > Consider your favorite function in one of your libraries, perhaps
    > void foo(int bar, char *baz);
    >
    > Now maybe you think of some cool improvement to foo that you could make,
    > but this relies on being able to pass an additional parameter to foo.
    > You now have some unattractive choices:
    > 1) change the signature of foo and break API compatability between
    > versions of the library
    > 2) have a new function
    > foo_v2(int bar, char *baz, long double newparam);
    > and make foo a wrapper for foo_v2.
    > 3) don't make the improvement.


    Option 2 seems most sensible for C. Nor does it seem
    especially "unattractive:" the new, improved, all-singing,
    all-dancing foo() must be doing something the old foo()
    did not. The "function" of the function is different, so
    why shouldn't the name be different, too?

    > Now, this dilemma would never have arisen if you'd "built extensibility
    > into the API" in the following way.
    >
    > Each function you write takes an unused void * parameter. So foo starts
    > out as
    > void foo(int bar, char *baz, void *p);
    > and the last parameter should be NULL.
    >
    > Now, consider extending the parameters of foo. You now use a
    > foo_v2_extra_params structure. Its first field encodes the additional
    > parameters, and the remaining fields are these parameters. Then version
    > 2 of foo can do this:
    >
    > void foo(int bar, char *baz, void *p)
    > {
    > /* define additional params as auto variables */
    > if(p)
    > /* process *p, set up extra args */
    > else
    > /* set additional params to defaults */
    > /* do stuff */
    > }
    >
    > If foo improves again, the struct just gets enlarged, and the function
    > casts p appropriately before proceeding.


    The struct "just gets enlarged" isn't quite enough: all
    the callers must now fill the additional fields with something.
    Even if your documentation carefully warned everybody to set
    up a fooParams like

    struct fooParams foop = { 0 };
    foop.x = 42;
    foop.y = -99;
    /* the newly-added foop.z remains zero */

    and not like

    struct fooParams foop;
    foop.x = 42;
    foop.y = -99;
    /* foop.z has indeterminate content */

    or like

    struct fooParams *pfoop = malloc(sizeof *pfoop);
    /* assume success */
    pfoop->x = 42;
    pfoop->y = -99;
    /* pfoop->z has indeterminate content */

    you can bet your bottom dollar that somebody, somewhere will
    have done it anyhow. Then you get to the unwinnable argument
    about whether the user is at fault for disregarding the
    documentation, or you are at fault for designing an error-
    inviting interface. Better to avoid the argument altogether
    than to try to win it.

    > Have people here implemented something like this in the past? What do
    > you think of it as a solution to the "API extension" problem?


    Personally, I don't like it. You begin by requiring all
    the users of foo() to supply an extra, unused, "must be NULL"
    argument, just on the off-chance that someday you'll think of
    a use for it -- that sucks. Second, by using `void*' you defeat
    any attempt at type-checking when and if you eventually invent
    a struct fooParams -- that sucks, too. And thirdly, as noted
    above, it creates its own set of problems.

    Perhaps a more compelling case could be made if foo() and
    its ilk were left behind and a concrete, real-life example
    were presented. Tell us about your most recent encounter with
    "the API extension problem:" what was the original function,
    what was the nature of the improvement, what did the additional
    parameter(s) convey?

    --
    Eric Sosman
    lid
    Eric Sosman, Oct 27, 2007
    #2
    1. Advertising

  3. Francis Johnson

    cr88192 Guest

    "Francis Johnson" <> wrote in message
    news:...
    > I've been having a discussion with some folks here about the wisdom or
    > otherwise of "building extensibility into an API".
    >
    > Consider your favorite function in one of your libraries, perhaps
    > void foo(int bar, char *baz);
    >
    > Now maybe you think of some cool improvement to foo that you could make,
    > but this relies on being able to pass an additional parameter to foo.
    > You now have some unattractive choices:
    > 1) change the signature of foo and break API compatability between
    > versions of the library
    > 2) have a new function
    > foo_v2(int bar, char *baz, long double newparam);
    > and make foo a wrapper for foo_v2.
    > 3) don't make the improvement.
    >


    2.
    VirtualAllocEx anyone?...

    you know, maybe in the future, this too will be extended:
    VirtualAllocXXX
    and:
    VirtualAllocHardCore
    VirtualAllocExtreme
    VirtualAllocTooTheMax

    or maybe:
    VirtualAllocReloaded
    VirtualAllocRessurection
    ....

    now, they have to merge some of these together into a more powerful
    function:
    VirtualAllocHardCoreExtremeTooTheMax

    or something...


    > Now, this dilemma would never have arisen if you'd "built extensibility
    > into the API" in the following way.
    >
    > Each function you write takes an unused void * parameter. So foo starts
    > out as
    > void foo(int bar, char *baz, void *p);
    > and the last parameter should be NULL.
    >
    > Now, consider extending the parameters of foo. You now use a
    > foo_v2_extra_params structure. Its first field encodes the additional
    > parameters, and the remaining fields are these parameters. Then version
    > 2 of foo can do this:
    >

    <snip>
    >
    > If foo improves again, the struct just gets enlarged, and the function
    > casts p appropriately before proceeding.
    >
    > Have people here implemented something like this in the past? What do
    > you think of it as a solution to the "API extension" problem?
    >


    syscall, ioctl, ...

    4. or, how about, we encode all our arguments and data as a big string?...
    then, on call, the API parses the string and figures out what is going on.

    5. or, even more spiffy, how about we make each argument its own function
    call.
    so, we call a function telling the API we are sending something, and a call
    for each arg, and then a call to indicate when we are done.

    ....

    hmm, was trying to come up with ideas which were bizarre, but oddly enough,
    these are actually useful (and I have made good use of them in the past).

    4. works good when the other end of the API is some elaborate piece of
    machinery (such as a compiler or interpreter).
    5. works well when you are building something, such as in a physics library
    or GUI framework.


    > --
    > Red danger: http://www.mindszenty.org
    cr88192, Oct 27, 2007
    #3
  4. Francis Johnson

    Tor Rustad Guest

    Francis Johnson wrote:
    > I've been having a discussion with some folks here about the wisdom or
    > otherwise of "building extensibility into an API".
    >
    > Consider your favorite function in one of your libraries, perhaps
    > void foo(int bar, char *baz);
    >
    > Now maybe you think of some cool improvement to foo that you could make,
    > but this relies on being able to pass an additional parameter to foo.
    > You now have some unattractive choices:
    > 1) change the signature of foo and break API compatability between
    > versions of the library


    Many times, backward compability is important, but in some cases the API
    is simply broken, and there is no way to avoid fixing existing code,
    unless you change the API. gets() is a prime example here.

    > 2) have a new function
    > foo_v2(int bar, char *baz, long double newparam);
    > and make foo a wrapper for foo_v2.


    If the existing code isn't broken, adding a new API, is the bets way
    IMO, but instead of using a wrapper, the usual case involve duplicating
    some code and providing a new independent function to ensure you don't
    break existing code.

    > 3) don't make the improvement.


    On a complex system, it might not be an improvement... :)


    --
    Tor < | tr i-za-h a-z>

    My C page: http://www.pg2.moo.no/C/index.html
    -include Win32 build of splint 3.1.2
    Tor Rustad, Oct 28, 2007
    #4
  5. Francis Johnson

    Guest

    On Oct 27, 3:27 pm, Francis Johnson <> wrote:
    > I've been having a discussion with some folks here about the wisdom or
    > otherwise of "building extensibility into an API".
    >
    > Consider your favorite function in one of your libraries, perhaps
    > void foo(int bar, char *baz);
    >
    > Now maybe you think of some cool improvement to foo that you could make,
    > but this relies on being able to pass an additional parameter to foo.
    > You now have some unattractive choices:
    > 1) change the signature of foo and break API compatability between
    > versions of the library
    > 2) have a new function
    > foo_v2(int bar, char *baz, long double newparam);
    > and make foo a wrapper for foo_v2.
    > 3) don't make the improvement.
    >
    > Now, this dilemma would never have arisen if you'd "built extensibility
    > into the API" in the following way.
    >
    > Each function you write takes an unused void * parameter. So foo starts
    > out as
    > void foo(int bar, char *baz, void *p);
    > and the last parameter should be NULL.
    >
    > Now, consider extending the parameters of foo. You now use a
    > foo_v2_extra_params structure. Its first field encodes the additional
    > parameters, and the remaining fields are these parameters. Then version
    > 2 of foo can do this:
    >
    > void foo(int bar, char *baz, void *p)
    > {
    > /* define additional params as auto variables */
    > if(p)
    > /* process *p, set up extra args */
    > else
    > /* set additional params to defaults */
    > /* do stuff */
    >
    > }
    >
    > If foo improves again, the struct just gets enlarged, and the function
    > casts p appropriately before proceeding.
    >
    > Have people here implemented something like this in the past? What do
    > you think of it as a solution to the "API extension" problem?


    Microsoft likes to put the size of a structure argument into itself
    as a version indicator. This has an advantage (for poor programmer)
    of not having to have/use magic constants.

    SOMESTUFF stuff;
    ZeroMemory (&stuff, sizeof stuff);
    stuff.cbSize = sizeof stuff;
    /* ... */
    CallFunctionExSoWhatIfItsExAlready (&stuff);

    Because Microsoft cares.

    Best regards,
    Yevgen
    , Oct 28, 2007
    #5
  6. Francis Johnson

    Eric Sosman Guest

    wrote:
    >
    > Microsoft likes to put the size of a structure argument into itself
    > as a version indicator. This has an advantage (for poor programmer)
    > of not having to have/use magic constants.


    DEC used to do that in OpenVMS (so I imagine H-P still does).
    It didn't mesh well with C, though, and I once had to swat a bug
    that the practice caused.

    As I recall, the structure in question was an RMS ("file
    system") control block of some kind. C didn't and doesn't have
    a good construct for initializing such things in a way that's
    "opaque" to the programmer, so VMS' library provided `const'
    versions of the various control blocks, initialized with assorted
    defaults. Your program set up its own control block by copying
    the library's version and then filling in extra fields:

    struct RMSthing cb = preInitializedRMSthing;
    cb.this = 42;
    cb.that = "foobar.txt";

    The control block contained its own self-describing size field,
    which got copied from the library to the program's version -- and
    since both of them were `struct RMSthing' instances, all was well,
    right?

    Wrong. DEC introduced a new feature (I think it was in
    support of what was then called the "High Sierra" CD-ROM format)
    that involved expanding the RMSthing by a few bytes -- let's say
    it went from 40 bytes to 48, although I don't recall the actual
    numbers.

    Alas, preInitializedRMSthing lived in a shared library (like
    a "DLL" or ".so" in other systems). Our code, already compiled
    and running in the field, had allocated 40 bytes for `cb' above,
    and copied only 40 bytes into it from preInitializedRMSthing.
    So far, so good -- but among those 40 bytes was the size field,
    which advertised the control block as being 48 bytes long, not 40.
    So when we used the block to tell VMS to do something, VMS looked
    at the size field, said "Oh, goodie: it's a new-style RMSthing!"
    and started messing around with the eight bytes past the end of
    our control block ...

    --
    Eric Sosman
    lid
    Eric Sosman, Oct 28, 2007
    #6
  7. <> wrote in message
    > On Oct 27, 3:27 pm, Francis Johnson <> wrote:
    >> I've been having a discussion with some folks here about the wisdom or
    >> otherwise of "building extensibility into an API".
    >>

    > Microsoft likes to put the size of a structure argument into itself
    > as a version indicator. This has an advantage (for poor programmer)
    > of not having to have/use magic constants.
    >
    > SOMESTUFF stuff;
    > ZeroMemory (&stuff, sizeof stuff);
    > stuff.cbSize = sizeof stuff;
    > /* ... */
    > CallFunctionExSoWhatIfItsExAlready (&stuff);
    >
    > Because Microsoft cares.
    >

    That's the way to do it. If you need make a function extensible, wrap the
    parameters into a structure and pass a size in so you can extend it. The
    fact that MS do it means the paradigm will be familiar to a large base of
    programmers.

    However it's not a good way of writing software. Filling out the struct will
    be a grief to your caller. It's the best that can be done with C, and in
    fact with most languages.

    --
    Free games and programming goodies.
    http://www.personal.leeds.ac.uk/~bgy1mm
    Malcolm McLean, Oct 28, 2007
    #7
  8. "Malcolm McLean" <> a écrit dans le message de news:
    ...
    > <> wrote in message
    >> On Oct 27, 3:27 pm, Francis Johnson <> wrote:
    >>> I've been having a discussion with some folks here about the wisdom or
    >>> otherwise of "building extensibility into an API".
    >>>

    >> Microsoft likes to put the size of a structure argument into itself
    >> as a version indicator. This has an advantage (for poor programmer)
    >> of not having to have/use magic constants.
    >>
    >> SOMESTUFF stuff;
    >> ZeroMemory (&stuff, sizeof stuff);
    >> stuff.cbSize = sizeof stuff;
    >> /* ... */
    >> CallFunctionExSoWhatIfItsExAlready (&stuff);
    >>
    >> Because Microsoft cares.
    >>

    > That's the way to do it. If you need make a function extensible, wrap the
    > parameters into a structure and pass a size in so you can extend it. The
    > fact that MS do it means the paradigm will be familiar to a large base of
    > programmers.
    >
    > However it's not a good way of writing software. Filling out the struct
    > will be a grief to your caller. It's the best that can be done with C, and
    > in fact with most languages.


    Unix has even worse solutions for similar but related issues:

    int accept(int sockfd, struct sockaddr *addr, socklen_t *addrlen);

    --
    Chqrlie.
    Charlie Gordon, Oct 28, 2007
    #8
  9. On Sun, 28 Oct 2007 17:43:57 -0000, "Malcolm McLean" wrote:
    >That's the way to do it. If you need make a function extensible, wrap the
    >parameters into a structure and pass a size in so you can extend it. The
    >fact that MS do it means the paradigm will be familiar to a large base of
    >programmers.


    More often MS uses different names, e.g. CreateWindowEx vs.
    CreateWindow. BTW, from a C programmer's point of view the Windows API
    is a relatively good API.


    --
    Roland Pibinger
    "The best software is simple, elegant, and full of drama" - Grady Booch
    Roland Pibinger, Oct 28, 2007
    #9
  10. "Roland Pibinger" <> a écrit dans le message de news:
    ...
    > On Sun, 28 Oct 2007 17:43:57 -0000, "Malcolm McLean" wrote:
    >>That's the way to do it. If you need make a function extensible, wrap the
    >>parameters into a structure and pass a size in so you can extend it. The
    >>fact that MS do it means the paradigm will be familiar to a large base of
    >>programmers.

    >
    > More often MS uses different names, e.g. CreateWindowEx vs.
    > CreateWindow. BTW, from a C programmer's point of view the Windows API
    > is a relatively good API.


    That's a case of a glas half full or half empty. I as a C programmer
    consider the Windows API to a relatively bad API.

    --
    Chqrlie
    Charlie Gordon, Oct 28, 2007
    #10
  11. Charlie Gordon said:

    <snip>

    > I as a C programmer
    > consider the Windows API to a relatively bad API.


    Which just goes to show that it takes all sorts, since I think the Win32 C
    API is not only very good indeed, but in fact the only part of Windows I
    actually enjoy using.

    --
    Richard Heathfield <http://www.cpax.org.uk>
    Email: -http://www. +rjh@
    Google users: <http://www.cpax.org.uk/prg/writings/googly.php>
    "Usenet is a strange place" - dmr 29 July 1999
    Richard Heathfield, Oct 29, 2007
    #11
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. =?Utf-8?B?UmFncw==?=

    datagrid extensibility

    =?Utf-8?B?UmFncw==?=, Jan 23, 2004, in forum: ASP .Net
    Replies:
    0
    Views:
    378
    =?Utf-8?B?UmFncw==?=
    Jan 23, 2004
  2. Olga
    Replies:
    2
    Views:
    344
  3. Replies:
    0
    Views:
    413
  4. john
    Replies:
    1
    Views:
    440
  5. sandeep

    Extensibility

    sandeep, Jun 4, 2009, in forum: C++
    Replies:
    4
    Views:
    269
    cr88192
    Jun 6, 2009
Loading...

Share This Page