K
Keith Thompson
CBFalconer said:Keith said:... snip ...[...]The preprocessor is often -- and incorrectly -- said to operate
on the text of the program's source code. In fact, it operates
on tokens (formally, "preprocessing tokens") that have been
derived from the text. These tokens simply come one after
another, pretty maids all in a row; they do not need to be
separated by spaces. Equally, though, they do not magically
combine with each other just because they happen to be adjacent.
Quite correct.
This sounds as if you are agreeing, which I trust is not so. The
point is that the tokens may be alphabetical (or numeric) strings,
and need to remain separated. For example:
#define foo((a), (b)) (a) (b)
....
foo(sizeof, char) ---> sizeof char
or ...> sizeofchar
which have much different meanings.
Yes, I agree with Eric. The preprocessor's output is a stream of
tokens; the standard says nothing about how those tokens are
represented. If the output is text to be interpreted as C source,
then certain tokens do need to be separated by whitespace so they'll
be interpreted as separate tokens. If the output is in some binary
form, or if the interface to the preprocess is a get_next_token()
function, then there is no such separation is necessary (and it may
not be meaningful).