preprocessor implementation GURU question

Discussion in 'C++' started by Dan W., Dec 3, 2003.

  1. Dan W.

    Dan W. Guest

    I'm trying to resolve a disagreement between friends --Digital Mars
    and BOOST-- about what the precompiler should do in a given situation:

    The problem arose when I was trying to compile a boost example program
    with the DM compiler, and the name of a file which was put together by
    a set of macros, ended up as,

    ....\...\list10 .cpp
    vs.
    ....\...\list10.cpp

    (notice the space before the period)

    The macro where the problem occurs is in a file called
    "...boost\mpl\list.hpp", which goes:

    # define MPL_AUX_LIST_HEADER
    BOOST_PP_CAT(list,BLAH_LIST_SIZE).hpp \
    /**/

    which I was able to work-around by adding the ##, such as:
    BOOST_PP_CAT(list,BLAH_LIST_SIZE)##.hpp \
    /**/

    I reported the problem to boost mailing list, and they said that the
    compiler shouldn't be adding a space.

    I reported this to the Digital Mars people and they say that the only
    legal way to concatenate is with ##.

    I reported again to boost, and they say that the above does not
    involve concatenation, as it does not cause tokens to merge.

    I reported this to DM and they insist that that is the old kludge of
    'concatenation by juxtaposition'.

    I reported this to boost and they say the DM compiler is relying on
    the old kludge of pure textual processing, whereas tokenization should
    happen first.

    I think I agree with boost. Given,

    #defina a(x) x
    a(the_file).ext

    should result in

    the_file.ext

    But then I'm not sure what the compiler should do if I write,

    a(the_file)ext //no dot

    Could someone please shed light on this issue before they shoot the
    messenger? >:^0 Thanks in advance.
    Dan W., Dec 3, 2003
    #1
    1. Advertising

  2. Dan W. wrote:
    ....
    >
    > I think I agree with boost. Given,
    >
    > #defina a(x) x
    > a(the_file).ext
    >
    > should result in
    >
    > the_file.ext
    >
    > But then I'm not sure what the compiler should do if I write,
    >
    > a(the_file)ext //no dot
    >
    > Could someone please shed light on this issue before they shoot the
    > messenger? >:^0 Thanks in advance.
    >


    I don't know what the right answer is, however, you need to boost.

    a) Shoot boost because the usage of this kind of behaviour is so
    uncommon that you're bound to raise issues like this. If boost relies
    on bug prone features, it's use is going to be limited.

    b) I'm not sure, you might need to shoot DM. They should be considering
    the possibility they might be wrong. "##" has been the standard way of
    concatenating for a while, I suspect that this is probably more
    universally supported than what boost is relying on.
    Gianni Mariani, Dec 3, 2003
    #2
    1. Advertising

  3. Dan W. wrote in news::

    [snip]

    > I think I agree with boost. Given,
    >
    > #defina a(x) x
    > a(the_file).ext
    >
    > should result in
    >
    > the_file.ext
    >
    > But then I'm not sure what the compiler should do if I write,
    >
    > a(the_file)ext //no dot
    >


    It should produce the output "the_fileext" (not the quotes), however
    this is 2 preproesor tokens (though only one C++ identifier). Add:

    #define the_fileext Expansion
    #define B() a(the_file)

    B()

    should produce: "the_fileext" Not "Expansion".

    I.e. juxtaposition is fine for producing C++ tokens but not for
    preprocessor tokens (i.e. tokens that can themselves be expanded).

    The pre ## way was something like:

    #define cat(a, b ) a/**/b
    #define XY Expanded

    cat(X, Y)

    produced "Expanded".

    This is what (IMO) ## was designed to fix.

    Rob.
    --
    http://www.victim-prime.dsl.pipex.com/
    Rob Williscroft, Dec 3, 2003
    #3
  4. Dan W.

    Dan W. Guest

    On 03 Dec 2003 16:23:57 EST, Gianni Mariani <>
    wrote:

    >I don't know what the right answer is, however, you need to boost.
    >
    >a) Shoot boost because the usage of this kind of behaviour is so
    >uncommon that you're bound to raise issues like this. If boost relies
    >on bug prone features, it's use is going to be limited.
    >
    >b) I'm not sure, you might need to shoot DM. They should be considering
    >the possibility they might be wrong. "##" has been the standard way of
    >concatenating for a while, I suspect that this is probably more
    >universally supported than what boost is relying on.
    >


    Yeah, I'd shoot them both; I just don't want Armageddon to start over
    a blank space ;-)

    But the thing is, the boost guys may be right. What they say is that
    tokenization should come before macro substitution, and that the use
    of ## is prescribed only for situations where tokens, otherwise
    separated by white space, should be forcibly violated into a merger.
    Juxtaposing ".ext" does not violate the tokens, therefore does not
    justify the use of ##. So, making the macro like

    #define filename(name,num,ext) name##num##.##ext

    would not exactly be right either; maybe it shouldn't even compile.
    And yet I see DM's point: Allowing,

    #defina a(x) x
    a(b)c
    would produce
    bc
    and therefore allow concatenation by juxtaposition.

    But then again, this may not be the precomp's responsibility. The
    boost guy said, paraphrasing, "Show me one place in the entire
    Standard where it says that the compiler should add a space."

    And the thing is, I wouldn't know what the Standard looks like if I
    stepped on it. (I should get a copy, one of these days.)

    Cheers!
    Dan W., Dec 3, 2003
    #4
  5. Dan W.

    Dan W. Guest

    >The pre ## way was something like:
    >
    >#define cat(a, b ) a/**/b
    >#define XY Expanded
    >
    >cat(X, Y)
    >
    >produced "Expanded".
    >
    >This is what (IMO) ## was designed to fix.
    >
    >Rob.


    Wow! My head is spinning. Ok, you're saying that in the old days a
    macro could modify the instance of another macro before the latter was
    applied. But I thought that was the way it is currently, isn't it?
    Would your last example NOT do that now? And how does ## prevent...
    what exactly?
    Maybe if you could give me a 4-way example: What worked and didn't
    work / then and now. Thx.
    Dan W., Dec 3, 2003
    #5
  6. Dan W. wrote in news::

    >>The pre ## way was something like:
    >>
    >>#define cat(a, b ) a/**/b
    >>#define XY Expanded
    >>
    >>cat(X, Y)
    >>
    >>produced "Expanded".
    >>
    >>This is what (IMO) ## was designed to fix.
    >>
    >>Rob.

    >
    > Wow! My head is spinning. Ok, you're saying that in the old days a
    > macro could modify the instance of another macro before the latter was
    > applied. But I thought that was the way it is currently, isn't it?
    > Would your last example NOT do that now? And how does ## prevent...
    > what exactly?
    > Maybe if you could give me a 4-way example: What worked and didn't
    > work / then and now. Thx.


    The problem with this method is when the C standard came out it
    required comments to be replaced with a single space character.

    Presumably this was so you couldn't write print/* whatever */f( "\n" );
    and have the compiler see printf( "\n" );

    So code that relied on this behaviour needed another way of achiving
    the same thing hence the invention of ##.

    Rob.
    --
    http://www.victim-prime.dsl.pipex.com/
    Rob Williscroft, Dec 3, 2003
    #6
  7. Dan W.

    Dan W. Guest

    >The problem with this method is when the C standard came out it
    >required comments to be replaced with a single space character.
    >
    >Presumably this was so you couldn't write print/* whatever */f( "\n" );
    >and have the compiler see printf( "\n" );
    >
    >So code that relied on this behaviour needed another way of achiving
    >the same thing hence the invention of ##.


    Ahhh! I see now. Thanks!
    Dan W., Dec 3, 2003
    #7
  8. "Rob Williscroft" <> wrote in message
    news:Xns9446DE9B1A9D9ukcoREMOVEfreenetrtw@195.129.110.200...

    > It should produce the output "the_fileext" (not the quotes), however
    > this is 2 preproesor tokens (though only one C++ identifier). Add:


    No. It is two preprocessing tokens *and* two identifier tokens. Each
    preprocessing token is converted directly to a token--there is no
    retokenization.

    Regards,
    Paul Mensonides
    Paul Mensonides, Dec 4, 2003
    #8
  9. Paul Mensonides wrote in news::

    > "Rob Williscroft" <> wrote in message
    > news:Xns9446DE9B1A9D9ukcoREMOVEfreenetrtw@195.129.110.200...
    >
    >> It should produce the output "the_fileext" (not the quotes), however
    >> this is 2 preproesor tokens (though only one C++ identifier). Add:

    >
    > No. It is two preprocessing tokens *and* two identifier tokens. Each
    > preprocessing token is converted directly to a token--there is no
    > retokenization.
    >


    Right so the only way to write a preprocessor as a separate
    programme is to introduce a space as the DM one does.

    But with:

    #define A() <c
    #define B(X) X
    #define C() >
    #define INC(X) A()B(X)C()

    #include INC(stdio)

    since the expansion can always be internal it should include <cstdio>
    not <c stdio>.

    Rob.
    --
    http://www.victim-prime.dsl.pipex.com/
    Rob Williscroft, Dec 4, 2003
    #9
  10. "Rob Williscroft" <> wrote in message
    news:Xns94471759C424FukcoREMOVEfreenetrtw@195.129.110.130...
    > Paul Mensonides wrote in news::
    >
    > > "Rob Williscroft" <> wrote in message
    > > news:Xns9446DE9B1A9D9ukcoREMOVEfreenetrtw@195.129.110.200...
    > >
    > >> It should produce the output "the_fileext" (not the quotes), however
    > >> this is 2 preproesor tokens (though only one C++ identifier). Add:

    > >
    > > No. It is two preprocessing tokens *and* two identifier tokens. Each
    > > preprocessing token is converted directly to a token--there is no
    > > retokenization.
    > >

    >
    > Right so the only way to write a preprocessor as a separate
    > programme is to introduce a space as the DM one does.
    >
    > But with:
    >
    > #define A() <c
    > #define B(X) X
    > #define C() >
    > #define INC(X) A()B(X)C()
    >
    > #include INC(stdio)
    >
    > since the expansion can always be internal it should include <cstdio>
    > not <c stdio>.


    Yes, but the whitespace can only be inserted *after* the preprocessor does what
    it normally does. E.g. when the preprocessing tokens are output to a text
    stream.

    Regards,
    Paul Mensonides
    Paul Mensonides, Dec 4, 2003
    #10
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Jonathan Bromley

    LRM guru question

    Jonathan Bromley, Oct 30, 2003, in forum: VHDL
    Replies:
    3
    Views:
    673
    Deb P
    Oct 31, 2003
  2. Mythran

    IIS Setup Guru Wannabe Question

    Mythran, Nov 6, 2003, in forum: ASP .Net
    Replies:
    3
    Views:
    436
    Jeff Cochran
    Nov 7, 2003
  3. Cronus
    Replies:
    1
    Views:
    667
    Paul Mensonides
    Jul 15, 2004
  4. =?Utf-8?B?QXNob2s=?=

    Question for ASP.NET + HTML guru

    =?Utf-8?B?QXNob2s=?=, May 30, 2006, in forum: ASP .Net
    Replies:
    6
    Views:
    543
    Laurent Bugnion
    Jun 18, 2006
  5. Chris Thomasson
    Replies:
    0
    Views:
    328
    Chris Thomasson
    Jun 14, 2006
Loading...

Share This Page