ctype & cctype in transform

[Taras_96]

Hi all,

A poster at http://bytes.com/forum/thread60652.html implies that using
strtoupper in transform doesn't work because ctype.h may define
strtoupper as a macro:

"The problem is that most implementations of the standard C <ctype.h>
header define functions like toupper/tolower/etc as macros. To make it
work in STL algorithms, you have to include <cctype> header instead of
<ctype.h>. At least on my PC (Debian/gcc 3.3), <cctype> undefines all
tolower/etc macros and pulls ::tolower/::toupper/etc functions into
std namespace, so that your sample will work fine."

However, I'm quite sure the reason the call to transform fails is
because automatic type deduction fails. Is the comment made incorrect?

Also, are macro implementations allowed to be used in the C++
implementation?

Can you pass a macro in as a template parameter? eg
transform(i.begin(),i.end(),i.begin(), MY_MACRO) - maybe this would
create an anonymous function?

Thanks

Taras

 

[Michael DOUBEZ]

Taras_96 a écrit :

Hi all,

A poster at http://bytes.com/forum/thread60652.html implies that using
strtoupper in transform doesn't work because ctype.h may define
strtoupper as a macro.
Yes.


"The problem is that most implementations of the standard C <ctype.h>
header define functions like toupper/tolower/etc as macros. To make it
work in STL algorithms, you have to include <cctype> header instead of
<ctype.h>. At least on my PC (Debian/gcc 3.3), <cctype> undefines all
tolower/etc macros and pulls ::tolower/::toupper/etc functions into
std namespace, so that your sample will work fine."
Yes

However, I'm quite sure the reason the call to transform fails is
because automatic type deduction fails. Is the comment made incorrect?

You are wrong. The preprocessing phase happens before compilation so the
macro name is never a candidate for type deduction.

Also, are macro implementations allowed to be used in the C++
implementation?

Macro are allowed in C++ if it is what you mean.

Can you pass a macro in as a template parameter? eg
transform(i.begin(),i.end(),i.begin(), MY_MACRO) - maybe this would
create an anonymous function?

No. MY_MACRO would be expanded.

In the case of strtoupper:
#define strtoupper(x) ((x>'a' && x<'z')?x+('A'-'a'):x)

The line:
transform(i.begin(),i.end(),i.begin(), strtoupper);
would be generated by the pre-processor as:
transform(i.begin(),i.end(),i.begin(),
((x>'a' && x<'z')?x+('A'-'a'):x)
);

Unless you can make this a lambda expression, the compiler doesn't
understand that .

 

[Taras_96]

Taras_96 a écrit :



You are wrong. The preprocessing phase happens before compilation so the
macro name is never a candidate for type deduction.

Ignoring the fact that toupper might actually be a macro, what I was
suggesting that the call to transform would fail because automatic
type deduction would fail (as toupper is overloaded AND templated).
This is stated at http://lists.debian.org/debian-gcc/2002/04/msg00092.html.
If macro definitions of C standard functions *are* allowed in C++ (see
my comment below), it seems that you may have the *extra* problem of
toupper being a macro, so using toupper as an input to a function that
expects a function pointer may cause problems.

Does the C++ standard specify whether the standard functions are
allowed to be macro'ed in
- <name.h> type libraries?
- <cname> type libraries?

The poster suggests that macro definitions are allowed in <name.h>

Macro are allowed in C++ if it is what you mean.

Where I was coming from was this statement:

"I'm not sure a conforming C++ implementation can have macro versions
of the ctype.h headers. Most versions I have seen have #ifdef __cpp__
or similar, using inline functions for the C++ version and macros for
the C one." - http://bytes.com/forum/thread60652.html (of course this
statement is not necessarily correct). If it *was* correct then the
macro expansion of toupper should never occur in a conforming standard
implementation, and thus you wouldn't have to consider the problem of
toupper actually being a macro when passing it into functions as a
function pointer (for a standard conforming implementation)

Thanks

Taras

 

[Taras_96]

On 2008-04-24 10:15:36 -0400, Taras_96 <[email protected]> said:

If MY_MACRO is an object-like macro, that line would pass whatever
MY_MACRO expands to. In the case of toupper from ctype.h, if toupper is
a macro, it's a function-like macro, so it needs an argument. When
there are no arguments the macro isn't used, and the name "toupper"
refers to the underlying function.

What underlying function are you referring to here?

Taras

 

[Taras_96]

But it's not a template in <ctype.h>, and not in <cctype>. Of course,
each standard header in C++ implementations are allowed to pull in
names that aren't required to be defined in that header, but it would
take a rather wierd implementation to get the std::toupper from the
<locale> header when you include <cctype>.





The poster is correct. The <name.h> headers are defined by the C
standard, and, with only a few exceptions (the string functions that
take const char* and return char*), their contents are the same in C++.
C allows masking macros for most functions. The <cname> headers aren't
allowed to use masking macros, but there are a few C functions that are
defined as macros (assert, offsetof, setjmp, va_arg, va_end, va_start),
and those must be defined as macros.

Thanks Pete, this is the information I was after.

Even if it wasn't correct, a function-like macro won't be applied if
it's not followed by a left parenthesis. So having a masking macro for
toupper doesn't matter. The second issue, and it's a genuine problem in
the original code, is that toupper expects non-negative character
values. The only negative value that's allowed is EOF. If char is
signed, the original code runs the risk of passing an illegal value to
toupper. The third issue requires more code:

For the sake of this discussion I'm ignoring that the inputs into the
functions may be negative (ie: have their upper bit set) .

Taras

 

[Michael DOUBEZ]

Pete Becker a écrit :

On 2008-04-24 10:33:48 -0400, Michael DOUBEZ <[email protected]> said:

Well, yes, if the call is

transform(i.begin(), i.end(), i.begin(), toupper(x));

But

transform(i.begin(), i.end(), i.begin(), toupper);

is okay syntactically, because toupper (without any parentheses) doesn't
name the function-like macro, but names the underlying function. The
problem is that its runtime semantics are wrong if values other than EOF
in the input sequence might be negative.

Yes, silly of me.
But the problem would be the same. The preprocessing phase is before the
compilation phase so transform(,,,toupper), even when expanded/matched
with the template, wouldn't be able to use the macro.

If there is a function of the same name, then yes, it would match it but
this function would conflict with the #define in the header because the
declaration would be expanded with the macro (hence the #undef in
<cctype> I guess).

 

[James Kanze]

On 2008-04-24 11:54:10 -0400, Taras_96 <[email protected]> said:

But it's not a template in <ctype.h>, and not in <cctype>. Of
course, each standard header in C++ implementations are
allowed to pull in names that aren't required to be defined in
that header, but it would take a rather wierd implementation
to get the std::toupper from the <locale> header when you
include <cctype>.

It would surprise me too, but it seems to be allowed. It would
surprise me somewhat less if <algorithm> pulled in <locale>,
however, and if he's going to use std::transform, he'll have

The poster is correct. The <name.h> headers are defined by the
C standard, and, with only a few exceptions (the string
functions that take const char* and return char*), their
contents are the same in C++. C allows masking macros for
most functions. The <cname> headers aren't allowed to use
masking macros, but there are a few C functions that are
defined as macros (assert, offsetof, setjmp, va_arg, va_end,
va_start), and those must be defined as macros.

Where does it say that? In D.5 of the copy of the draft I have
on line, the contents of the <name.h> headers is defined (very
succinctly) in terms of the corresponding <cname> headers, the
only difference being that the names must be visible in ::.

With regards to the original problem, I think that:

#include <ctype.h>
#include <algorithm>

std::transform( ..., ::toupper ) ;

is guaranteed to compile (and do the right thing if plain char
is unsigned).

 

[James Kanze]

What underlying function are you referring to here?

The C standard allows toupper to be defined as a function style
macro, but it requires a toupper function, even if there is also
a macro. An implementation providing the macro must do
something like:
extern int toupper( int ) ;
#define toupper( ch ) ...
If you use the symbol toupper in a context where it is not
immediately followed by an opening parentheses, the macro is not
expanded, and you get the function, e.g.

int (*pf)( int ) = toupper ;
// or even
int ch = (toupper)( ch ) ;

(The implementation probably uses this trick itself, e.g.:

// toupper.c
#include <ctype.h>

int
(toupper)( int ch )
{
return toupper( ch ) ;
}

.)

 

[James Kanze]

On 2008-04-25 05:29:29 -0400, James Kanze <[email protected]> said:

17.4.1.2 [headers]/6: "Names that are de?ned as functions in C
shall be de?ned as functions in the C++ Standard Library.*"
The "*" points to a footnote: "This disallows the practice,
allowed in C, of providing a "masking macro" in addition to
the function prototype. The only way to achieve equivalent
"inline" behavior in C++ is to provide a de?nition as an
extern inline function. "

I don't think I was clear. That part I knew. I interpreted
your posting as saying that the <name.h> headers could define
the macros, even in C++, and I was wondering where you'd read
that. On rereading what you said, however, I don't think you
actually said that, so the question is irrelevant.

 

[James Kanze]

On 2008-04-25 05:34:54 -0400, James Kanze <[email protected]> said:

"Immediately followed by" is a bit too strong; you can have
whitespace between them. You get the function if the first
non-whitespace character after the name is not an opening
parenthesis.

Yes. What I meant by "immediately" were the tokens, not the
text. (Thinking too much in terms of compiler writing.)

I wonder: is this before or after other expansions. In other
words, given something like:

#define TOTO
// ...
toupper TOTO (...)

am I guaranteed to get the function or not?

 

[Taras_96]

On Apr 24, 6:52 pm, Pete Becker <[email protected]> wrote:

Where does it say that? In D.5 of the copy of the draft I have
on line, the contents of the <name.h> headers is defined (very
succinctly) in terms of the corresponding <cname> headers, the
only difference being that the names must be visible in ::.

With regards to the original problem, I think that:

#include <ctype.h>
#include <algorithm>

std::transform( ..., ::toupper ) ;

is guaranteed to compile (and do the right thing if plain char
is unsigned).

--

Sorry for the newb question, but what is the ::function_name syntax?

Taras

 

[James Kanze]

Sorry for the newb question, but what is the ::function_name syntax?

The :: operator is scope resolution operator. It is used to
tell the compiler where to look for the symbol (any symbol, not
just a function name); as a unary operator (as it is here), it
specifies that the symbol should be looked up in global scope
(and only in global scope). Thus, symbols in the std namespace
will not be found (and if you write std::toupper, symbols in the
global namespace will not be found).

If I understand the current draft correctly, symbols (except
macros) defined in <ctype.h> will be defined in the global
namespace (and maybe in std as well); symbols defined in
<cctype> will be defined in the std namespace, and maybe in ::
as well.

 

[James Kanze]

On 2008-04-25 11:51:55 -0400, James Kanze <[email protected]> said:

[...]

I think so. Formally, if the next preprocessing-token is not a
(, it's not a macro invocation. TOTO is an identifier, hence a
preprocessing-token, so I think you get the function.

That's what I think the standard requires, but I'm quite sure
about the implications of [cpp.rescan] if the text in question
is itself in a macro. If I interpret it correctly, the
replacement list is only rescanned once, after replacement of
the parameters (but not other macros), to I'd still be
guaranteed to get the function. But if the text is rescanned
after TOTO has been expanded, then suddenly toupper becomes a
macro again.

 

[Taras_96]

The :: operator is scope resolution operator. It is used to
tell the compiler where to look for the symbol (any symbol, not
just a function name); as a unary operator (as it is here), it
specifies that the symbol should be looked up in global scope
(and only in global scope). Thus, symbols in the std namespace
will not be found (and if you write std::toupper, symbols in the
global namespace will not be found).

Ok - makes sense