token pasting problem in K&R preprocessor

Discussion in 'C Programming' started by Henry Townsend, Jun 19, 2006.

  1. I hope this is on-topic in c.l.c - it's about the C preprocessor more
    than the language per se, more generally about the K&R behavior, and
    most specifically about the Sun cpp which is why I've cross posted there.

    The test case below is taken from an Imake setup (yes, old, I know).
    There's a Concat() macro which uses old-style /**/ for token pasting (we
    are not allowed to assume ANSI so ## isn't allowed). I'm trying to paste
    AND expand the two macros X and Y. I cannot figure out why it doesn't
    work in this scenario; the literal /**/ works but the macro doesn't.

    % cat /tmp/X.c
    #define X xxx
    #define Y yyy
    #define Concat(a,b)a/**/b
    X/**/Y
    Concat(X,Y)

    % /usr/ccs/lib/cpp /tmp/X.c
    # 1 "/tmp/X.c"
    [blank space elided]
    xxxyyy
    XY

    Thanks,
    HT
     
    Henry Townsend, Jun 19, 2006
    #1
    1. Advertising

  2. Henry Townsend wrote:
    [...]
    > % cat /tmp/X.c
    > #define X xxx
    > #define Y yyy
    > #define Concat(a,b)a/**/b
    > X/**/Y
    > Concat(X,Y)
    >
    > % /usr/ccs/lib/cpp /tmp/X.c
    > # 1 "/tmp/X.c"
    > [blank space elided]
    > xxxyyy
    > XY


    Using MSVC6 (cl /P usenet.c), I get:
    ==========
    xxxyyy
    xxx yyy
    ==========

    Using gcc version "egcs-2.91.66" (gcc -E usenet.c), I get:

    ==========
    # 1 "usenet.c"


    xxx yyy
    xxx yyy
    ==========

    I can't imagine how you get "XY" for the second line. Doesn't the
    preprocessor have to expand the X and Y into their #define'd values?

    Can you run the C compiler, asking it to only preprocess the file,
    rather than run cpp directly? What happens then?

    Now, as to why MSVC gives "xxxyyy" for the first line, and gcc gives
    "xxx yyy", I don't know. Can someone here tell me which is "right"
    and which is "wrong"? (Or are they both "right"?)

    --
    +-------------------------+--------------------+-----------------------+
    | Kenneth J. Brody | www.hvcomputer.com | #include |
    | kenbrody/at\spamcop.net | www.fptech.com | <std_disclaimer.h> |
    +-------------------------+--------------------+-----------------------+
    Don't e-mail me at: <mailto:>
     
    Kenneth Brody, Jun 19, 2006
    #2
    1. Advertising

  3. Henry Townsend

    Michael Mair Guest

    Henry Townsend schrieb:
    > I hope this is on-topic in c.l.c - it's about the C preprocessor more
    > than the language per se, more generally about the K&R behavior, and
    > most specifically about the Sun cpp which is why I've cross posted there.
    >
    > The test case below is taken from an Imake setup (yes, old, I know).
    > There's a Concat() macro which uses old-style /**/ for token pasting (we
    > are not allowed to assume ANSI so ## isn't allowed). I'm trying to paste
    > AND expand the two macros X and Y. I cannot figure out why it doesn't
    > work in this scenario; the literal /**/ works but the macro doesn't.
    >
    > % cat /tmp/X.c
    > #define X xxx
    > #define Y yyy
    > #define Concat(a,b)a/**/b
    > X/**/Y
    > Concat(X,Y)
    >
    > % /usr/ccs/lib/cpp /tmp/X.c
    > # 1 "/tmp/X.c"
    > [blank space elided]
    > xxxyyy
    > XY


    I don't have a K&R compiler, so I am only guessing.

    Note that you have the same problem for ##. There, you solve
    it by an additional "level of indirection", i.e. you force
    expansion of the macro arguments by wrapping "Concat" once:
    #define X xxx
    #define Y yyy
    #define con_cat(a,b) a##b
    #define CONCAT(a,b) con_cat(a,b)
    con_cat(X,Y)
    CONCAT(X,Y)
    leads to
    XY
    xxxyyy

    I'd try the same for K&R, too.

    HTH
    Michael
    --
    E-Mail: Mine is an /at/ gmx /dot/ de address.
     
    Michael Mair, Jun 19, 2006
    #3
  4. Henry Townsend

    Eric Sosman Guest

    Henry Townsend wrote On 06/19/06 14:52,:
    > [...]
    > There's a Concat() macro which uses old-style /**/ for token pasting (we
    > are not allowed to assume ANSI so ## isn't allowed). [...]


    ANSI C has been around for a little less than seventeen
    years, so perhaps your caution in adopting it is justified.
    Just as a point of idle interest, how many decades do you
    think must elapse before it is safe to adopt a new standard?
    How are you doing with the plans to move from FORTRAN II to
    FORTRAN IV?

    Challenge: Without reference to a newspaper, almanac,
    Wikipedia, or other such source, name three people who were
    heads of state of members of the UN Security Council at the
    time ANSI C was adopted.

    Challenge: Is an `int' large enough to count the number
    of seconds since the adoption of ANSI C? The number of hours?
    Of days?

    Challenge: In twenty-five words or fewer, compare and contrast
    your organization's rate of standards adoption with the rate of
    proton decay.

    (All right, all right -- I'm having some fun at your expense.
    But in all seriousness, I urge you to consider moving forward to
    within a decade of the leading edge. There were once good reasons
    to accommodate pre-ANSI implementations, but their goodness has
    diminished with the passage of time and is now approximately equal
    to that of a funerary meal from a Pharaoh's tomb. You really,
    really ought to take a hard look at your reasons for adhering to
    outdated technologies. Observe: The thread at hand demonstrates
    that this antiquarian romanticism is making trouble and hence
    costing you money ...)

    --
     
    Eric Sosman, Jun 19, 2006
    #4
  5. Henry Townsend

    CBFalconer Guest

    Henry Townsend wrote:
    >

    .... snip ...
    >
    > The test case below is taken from an Imake setup (yes, old, I
    > know). There's a Concat() macro which uses old-style /**/ for
    > token pasting (we are not allowed to assume ANSI so ## isn't
    > allowed). I'm trying to paste AND expand the two macros X and Y.
    > I cannot figure out why it doesn't work in this scenario; the
    > literal /**/ works but the macro doesn't.


    Then you just don't paste tokens. Even old K&R specified that any
    comment was replaced by at least one blank. Some compilers were
    too stupid to do this.

    --
    "I don't know where bin Laden is. I have no idea and really
    don't care. It's not that important." - G.W. Bush, 2002-03-13
    "No, we've had no evidence that Saddam Hussein was involved
    with September the 11th." - George Walker Bush 2003-09-17
     
    CBFalconer, Jun 19, 2006
    #5
  6. CBFalconer wrote:
    > Henry Townsend wrote:
    > ... snip ...
    >> The test case below is taken from an Imake setup (yes, old, I
    >> know). There's a Concat() macro which uses old-style /**/ for
    >> token pasting (we are not allowed to assume ANSI so ## isn't
    >> allowed). I'm trying to paste AND expand the two macros X and Y.
    >> I cannot figure out why it doesn't work in this scenario; the
    >> literal /**/ works but the macro doesn't.

    >
    > Then you just don't paste tokens. Even old K&R specified that any
    > comment was replaced by at least one blank. Some compilers were
    > too stupid to do this.


    A) The example was not typed in, it was pasted verbatim from a shell
    session. It's only five lines - feel free to try it yourself if you want
    to see that there's no blank in the /**/ case.

    B) This is an imake-based build system which has been in use for about
    18 years (yes, before ANSI C was ratified), and has been running on
    every commercially viable Unix platform during that time. Developers and
    users of this very-well-known product will be surprised to hear that it
    can't be built.

    C) The Concat() macro was take directly from the X11 imake system, where
    it survives to this day.

    So regardless of what K&R may have _specified_, empirical evidence is
    that no blank is inserted.
     
    Henry Townsend, Jun 20, 2006
    #6
  7. Henry Townsend

    Eric Sosman Guest

    Henry Townsend wrote:

    > CBFalconer wrote:
    >
    >> Henry Townsend wrote:
    >> ... snip ...
    >>
    >>> The test case below is taken from an Imake setup (yes, old, I
    >>> know). There's a Concat() macro which uses old-style /**/ for
    >>> token pasting (we are not allowed to assume ANSI so ## isn't
    >>> allowed). I'm trying to paste AND expand the two macros X and Y.
    >>> I cannot figure out why it doesn't work in this scenario; the
    >>> literal /**/ works but the macro doesn't.

    >>
    >>
    >> Then you just don't paste tokens. Even old K&R specified that any
    >> comment was replaced by at least one blank. Some compilers were
    >> too stupid to do this.

    >
    >
    > A) The example was not typed in, it was pasted verbatim from a shell
    > session. It's only five lines - feel free to try it yourself if you want
    > to see that there's no blank in the /**/ case.
    >
    > B) This is an imake-based build system which has been in use for about
    > 18 years (yes, before ANSI C was ratified), and has been running on
    > every commercially viable Unix platform during that time. Developers and
    > users of this very-well-known product will be surprised to hear that it
    > can't be built.
    >
    > C) The Concat() macro was take directly from the X11 imake system, where
    > it survives to this day.
    >
    > So regardless of what K&R may have _specified_, empirical evidence is
    > that no blank is inserted.


    The biggest problem tackled by the ANSI Standard was not
    inventing the token-pasting operator, or void, or prototypes,
    or any of the other "new features" in the language. Larger
    than all of these -- than all of these put together -- was
    the problem of reconciling the multiple divergent versions of
    C that had arisen.

    Does sprintf() return a count or a pointer?

    Does unsigned short promote to int or to unsigned int?

    Does an extern declaration inside a block have block
    scope or file scope?

    If a macro definition contains a string literal that in
    turn contains a substring identical to one of the macro's
    arguments, does substitution occur?

    ... and, of course: What happens when a macro definition
    expands two of its arguments with only a comment separating
    them?

    Actual, real implementations of pre-Standard C differed
    on all of these points (and more). The whole reason for the
    standardization effort in the first place was that there was
    no consensus on these matters. There was no single definition,
    either formal or de facto, about how to paste tokens.

    Now: You say you are "not allowed to assume ANSI C," a
    stance I've commented on elsethread. However, you mention
    that all this token-pasting is done through a Concat() macro.
    Can you not make a preprocessor test for __STDC__ and define
    Concat() accordingly? If you are worried about pre-Standard
    (or anti-Standard) compilers that define __STDC__ but don't
    do token-pasting, I think you're worried about trifles.

    --
    Eric Sosman
    lid
     
    Eric Sosman, Jun 20, 2006
    #7
  8. Eric Sosman wrote:
    > Now: You say you are "not allowed to assume ANSI C," a
    > stance I've commented on elsethread. However, you mention
    > that all this token-pasting is done through a Concat() macro.
    > Can you not make a preprocessor test for __STDC__ and define
    > Concat() accordingly? If you are worried about pre-Standard
    > (or anti-Standard) compilers that define __STDC__ but don't
    > do token-pasting, I think you're worried about trifles.


    No, if you go back to the original post I was trying to understand
    specifically how to make a certain kind of pasting work with a certain
    K&R cpp, with the idea that the answer would probably apply to the 6-8
    other old K&R preprocessors I'll be encountering soon on other
    platforms. But what I think I've learned here is that each cpp may
    behave differently. Mind you, though, this Concat() macro has worked for
    every platform X11 has been ported to over the last 20 years, which is
    quite a lot.

    As for whether to upgrade to ANSI, I personally am a big fan of getting
    with the times but it's not necessarily part of my charter to do so in
    this case. I may be able to get it on the table but at the moment the
    referenced problem must be dealt with in the context of K&R. Luckily
    it's not a huge difficulty; I can special-case this particular value. I
    was just hoping for a solution short of ANSI C.

    HT
     
    Henry Townsend, Jun 20, 2006
    #8
  9. Henry Townsend wrote:

    > Eric Sosman wrote:
    >
    >> Now: You say you are "not allowed to assume ANSI C," a
    >> stance I've commented on elsethread. However, you mention
    >> that all this token-pasting is done through a Concat() macro.
    >> Can you not make a preprocessor test for __STDC__ and define
    >> Concat() accordingly? If you are worried about pre-Standard
    >> (or anti-Standard) compilers that define __STDC__ but don't
    >> do token-pasting, I think you're worried about trifles.

    >
    >
    > No, if you go back to the original post I was trying to understand
    > specifically how to make a certain kind of pasting work with a certain
    > K&R cpp, with the idea that the answer would probably apply to the 6-8
    > other old K&R preprocessors I'll be encountering soon on other
    > platforms. But what I think I've learned here is that each cpp may
    > behave differently. Mind you, though, this Concat() macro has worked for
    > every platform X11 has been ported to over the last 20 years, which is
    > quite a lot.
    >
    > As for whether to upgrade to ANSI, I personally am a big fan of getting
    > with the times but it's not necessarily part of my charter to do so in
    > this case. I may be able to get it on the table but at the moment the
    > referenced problem must be dealt with in the context of K&R. Luckily
    > it's not a huge difficulty; I can special-case this particular value. I
    > was just hoping for a solution short of ANSI C.
    >
    > HT


    There are other advantages to coding in ANSI C, the nicest one is that
    the compilers no longer complain about the sleazy coding practices that
    were so prevalent in K&R. The other nice thing is that it tends to
    uncover bugs that may have been making you itch for years. Doing it is
    generally tedious but trivial.
     
    Richard B. Gilbert, Jun 20, 2006
    #9
  10. Henry Townsend

    Old Wolf Guest

    [OT] token pasting problem in K&R preprocessor

    Eric Sosman wrote:
    >
    > Challenge: Without reference to a newspaper, almanac,
    > Wikipedia, or other such source, name three people who were
    > heads of state of members of the UN Security Council at the
    > time ANSI C was adopted.


    I'll bite: Elizabeth II, Mitterrand, Deng Xiaoping, Gorbachev, Reagan

    When was the standard actually ratified? I've heard rumblings
    that even though the "89" tag is attached, it wasn't actually
    ratified by ANSI until sometime in 1990 (in which case I
    may have failed your challenge)

    > Challenge: Is an `int' large enough to count the number
    > of seconds since the adoption of ANSI C? The number of hours?
    > Of days?


    Well, a 32-bit int is going to get us from 1 Jan 1970 to sometime
    in 2039.
     
    Old Wolf, Jun 21, 2006
    #10
  11. Re: [OT] token pasting problem in K&R preprocessor

    "Old Wolf" <> writes:
    > Eric Sosman wrote:
    >>
    >> Challenge: Without reference to a newspaper, almanac,
    >> Wikipedia, or other such source, name three people who were
    >> heads of state of members of the UN Security Council at the
    >> time ANSI C was adopted.

    >
    > I'll bite: Elizabeth II, Mitterrand, Deng Xiaoping, Gorbachev, Reagan
    >
    > When was the standard actually ratified? I've heard rumblings
    > that even though the "89" tag is attached, it wasn't actually
    > ratified by ANSI until sometime in 1990 (in which case I
    > may have failed your challenge)


    The ANSI C standard was dated 1989. The ISO C standard, equivalent to
    the ANSI standard but with the sections renumbered and some other
    cosmetic changes, was dated 1990. ANSI then officially adopted the
    1990 ISO standard (I don't know just when that happened).

    >> Challenge: Is an `int' large enough to count the number
    >> of seconds since the adoption of ANSI C? The number of hours?
    >> Of days?

    >
    > Well, a 32-bit int is going to get us from 1 Jan 1970 to sometime
    > in 2039.


    Almost. Tue 2038-01-19 03:14:07 UTC.

    --
    Keith Thompson (The_Other_Keith) <http://www.ghoti.net/~kst>
    San Diego Supercomputer Center <*> <http://users.sdsc.edu/~kst>
    We must do something. This is something. Therefore, we must do this.
     
    Keith Thompson, Jun 21, 2006
    #11
  12. Henry Townsend

    Joe Wright Guest

    Re: [OT] token pasting problem in K&R preprocessor

    Keith Thompson wrote:
    > "Old Wolf" <> writes:
    >> Eric Sosman wrote:
    >>> Challenge: Without reference to a newspaper, almanac,
    >>> Wikipedia, or other such source, name three people who were
    >>> heads of state of members of the UN Security Council at the
    >>> time ANSI C was adopted.

    >> I'll bite: Elizabeth II, Mitterrand, Deng Xiaoping, Gorbachev, Reagan
    >>
    >> When was the standard actually ratified? I've heard rumblings
    >> that even though the "89" tag is attached, it wasn't actually
    >> ratified by ANSI until sometime in 1990 (in which case I
    >> may have failed your challenge)

    >
    > The ANSI C standard was dated 1989. The ISO C standard, equivalent to
    > the ANSI standard but with the sections renumbered and some other
    > cosmetic changes, was dated 1990. ANSI then officially adopted the
    > 1990 ISO standard (I don't know just when that happened).
    >
    >>> Challenge: Is an `int' large enough to count the number
    >>> of seconds since the adoption of ANSI C? The number of hours?
    >>> Of days?

    >> Well, a 32-bit int is going to get us from 1 Jan 1970 to sometime
    >> in 2039.

    >
    > Almost. Tue 2038-01-19 03:14:07 UTC.
    >

    Regard..

    With 'time' = 'INT_MAX' or 31 one bits..
    2147483647 Tue Jan 19 03:14:07 2038

    With 'time' = 'UINT_MAX' or 32 one bits..
    4294967295 Sun Feb 7 06:28:15 2106

    --
    Joe Wright
    "Everything should be made as simple as possible, but not simpler."
    --- Albert Einstein ---
     
    Joe Wright, Jun 21, 2006
    #12
  13. Henry Townsend

    Guest

    Re: [OT] token pasting problem in K&R preprocessor

    Old Wolf <> wrote:
    >
    > When was the standard actually ratified?


    According to the Foreword:

    This document was approved as an American National Standard by the
    American National Standards Institute (ANSI) on December 14, 1989.

    > I've heard rumblings
    > that even though the "89" tag is attached, it wasn't actually
    > ratified by ANSI until sometime in 1990 (in which case I
    > may have failed your challenge)


    The delay came before the tag was attached -- the committee voted the
    document out in 1988 but a procedural snafu at ANSI kept it from being
    approved until 1989.

    -Larry Jones

    These things just seem to happen. -- Calvin
     
    , Jun 21, 2006
    #13
  14. Henry Townsend

    Guest

    Eric Sosman wrote:
    >
    > The biggest problem tackled by the ANSI Standard was not
    > inventing the token-pasting operator, or void, or prototypes,
    > or any of the other "new features" in the language. Larger
    > than all of these -- than all of these put together -- was
    > the problem of reconciling the multiple divergent versions of
    > C that had arisen. ...
    >
    > Does unsigned short promote to int or to unsigned int?



    And they got that one wrong.
     
    , Jun 24, 2006
    #14
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Cronus
    Replies:
    1
    Views:
    681
    Paul Mensonides
    Jul 15, 2004
  2. Wessi
    Replies:
    3
    Views:
    867
    Lawrence Kirby
    Aug 11, 2005
  3. Mark Odell

    Token-pasting trouble

    Mark Odell, Apr 6, 2006, in forum: C Programming
    Replies:
    8
    Views:
    367
    Mark Odell
    Apr 6, 2006
  4. Mark Odell
    Replies:
    7
    Views:
    359
    Mark Odell
    May 1, 2006
  5. Token pasting problem

    , Oct 16, 2008, in forum: C Programming
    Replies:
    1
    Views:
    322
    Harald van Dijk
    Oct 16, 2008
Loading...

Share This Page