C11 Compiler Test Suite

[Chicken McNuggets]

Does anyone know if a C11 compiler test suite is available?

If not might I suggest that one be written? It would certainly be
something that I would be interested in helping with and I think it
might well have the advantage of pushing open source (and possibly
closed source) compilers to support the full standard quicker than if
there was no test suite to call them out on their support.

It could be a project that might be quite easy to coordinate on
something like Github.

For instance I know that Clang supports some of the features of C11
already but having inquired on the LLVM IRC channel there is no public
documentation stating what is available. Nor can I find any specific
information on the state of GCC's C11 support.

It is essential that the C community makes some noise about C11 support
as most compiler vendors seem to be concentrating on C++11.

 

[steve.b.hafner]

Does anyone know if a C11 compiler test suite is available?



If not might I suggest that one be written? It would certainly be

something that I would be interested in helping with and I think it

might well have the advantage of pushing open source (and possibly

closed source) compilers to support the full standard quicker than if

there was no test suite to call them out on their support.

Don't know, but efforts could be combined easily if you start and you find similar projects along the way.

Anyway, grab the C11 standard:

http://www.open-std.org/jtc1/sc22/wg14/www/docs/n1570.pdf

and C99 standard:

http://www.open-std.org/jtc1/sc22/wg14/www/docs/n1256.pdf

and have at it, if you like.

For instance I know that Clang supports some of the features of C11

already but having inquired on the LLVM IRC channel there is no public

documentation stating what is available. Nor can I find any specific

information on the state of GCC's C11 support.

See

http://clang.llvm.org/docs/LanguageExtensions.html#c11

and the release notes for GCC 4.7 series:

http://gcc.gnu.org/gcc-4.7/changes.html

where it states:

"There is support for some more features from the C11 revision of the ISO C standard. GCC now accepts the options -std=c11 and -std=gnu11, in addition to the previous -std=c1x and -std=gnu1x.

"Unicode strings (previously supported only with options such as -std=gnu11, now supported with -std=c11), and the predefined macros __STDC_UTF_16__ and __STDC_UTF_32__.

-- Nonreturning functions (_Noreturn and <stdnoreturn.h>).
-- Alignment support (_Alignas, _Alignof, max_align_t, <stdalign.h>).
-- A built-in function __builtin_complex is provided to support C library implementation of the CMPLX family of macros."

It is essential that the C community makes some noise about C11 support

The most effective noise is someone working at solving the problems. You're a person just like any other.

 

[Chicken McNuggets]

The most effective noise is someone working at solving the problems. You're a person just like any other.

True. I'll certainly have a look into it once I have some free time in
the coming weeks.

I was really just interested in seeing if there was much interest from
people in the project.

 

[Kaz Kylheku]

It is essential that the C community makes some noise about C11 support

No, it isn't. Not only is is not essential, it is counterproductive.
The "C community" should just go about its business and not mess with its
compilers or libraries, except in those areas where they are foudn to be
broken.

 

[Chicken McNuggets]

No, it isn't. Not only is is not essential, it is counterproductive.
The "C community" should just go about its business and not mess with its
compilers or libraries, except in those areas where they are foudn to be
broken.

If a compiler does not follow the standard (and that includes the latest
standard) then I would argue that the compiler is broken. Full standards
support is not optional. It should be demanded.

 

[Kaz Kylheku]

If a compiler does not follow the standard (and that includes the latest
standard) then I would argue that the compiler is broken.

You would argue, but you'd be wrong. Something is broken if it doesn't
implement its specification. The choice of specification is the implementor's
privilege. If someone writes a compiler that is documented as conforming to
ISO 9899:1990, and it does exactly that, then it's not broken in any way.

Full standards support is not optional. It should be demanded.

Such a thing can be demanded by making a monetary bid. Offer someone
money to make or patch you a C11 compiler.

 

[Chicken McNuggets]

You would argue, but you'd be wrong. Something is broken if it doesn't
implement its specification. The choice of specification is the implementor's
privilege. If someone writes a compiler that is documented as conforming to
ISO 9899:1990, and it does exactly that, then it's not broken in any way.

This is incorrect the C11 standard states explicitly:

"This third edition cancels and replaces the second edition, ISO/IEC
9899:1999, as
corrected by ISO/IEC 9899:1999/Cor 1:2001, ISO/IEC 9899:1999/Cor 2:2004, and
ISO/IEC 9899:1999/Cor 3:2007."

thus the only way to be standards compliant is by supporting the latest
standard. If you only support C90 or C99 then you are not standards
compliant as the C11 standard explicitly states it cancels and replaces
all previous standards. Likewise C99 stated that it canceled and
replaced C90.

 

[Rui Maciel]

This is incorrect the C11 standard states explicitly:

"This third edition cancels and replaces the second edition, ISO/IEC
9899:1999, as
corrected by ISO/IEC 9899:1999/Cor 1:2001, ISO/IEC 9899:1999/Cor 2:2004,
and ISO/IEC 9899:1999/Cor 3:2007."

thus the only way to be standards compliant is by supporting the latest
standard. If you only support C90 or C99 then you are not standards
compliant as the C11 standard explicitly states it cancels and replaces
all previous standards. Likewise C99 stated that it canceled and
replaced C90.

Your comment is detached from reality. No matter what is stated in that
paragraph, which should be noted that it isn't enforced by anyone or
anything, people still write code and develop compilers which target and
support specific versions of the C standard which precede ISO/IEC 9899:2011,
and neither of those ceasses to be C if a new revision is published.

In addition, if your comment made any sense then it would mean that once a
new revision of the C standard was released, suddenly no software in the
world would have been developed in C.

There is a reason why standards are referred not only by their name but also
by their revision number/publication date.


Rui Maciel

 

[Keith Thompson]

This is incorrect the C11 standard states explicitly:

"This third edition cancels and replaces the second edition, ISO/IEC
9899:1999, as corrected by ISO/IEC 9899:1999/Cor 1:2001, ISO/IEC
9899:1999/Cor 2:2004, and ISO/IEC 9899:1999/Cor 3:2007."

thus the only way to be standards compliant is by supporting the
latest standard. If you only support C90 or C99 then you are not
standards compliant as the C11 standard explicitly states it cancels
and replaces all previous standards. Likewise C99 stated that it
canceled and replaced C90.

That's true in a sense, but practically speaking it's not.

The ISO C standard (any version of it) does not have the force
of law. Some jurisdiction could pass a law requiring conformance
to it; I know of none that have done so. And a contract requiring
conformance could be enforced by law. But there's no fundamental
reason why such a contract, or even such a law, couldn't specify
a standard other than the current one.

If a vendor states that an implementation conforms to the C90 or C99
standard, we all know what that means, and it's a very useful piece
of information. The *only* thing missing in that situation is that
ISO doesn't recognize it as a conforming implementation. Anyone not
speaking officially on behalf of ISO is free to accept or reject
that implementation, on that or any other basis. And if a compiler
that's claimed to conform to C90 doesn't let me use "restrict" as an
identifier, or fails to diagnose mixed declarations and statements
within a block, then I have a legitimate basis for a bug report.

And even within the context of ISO, the current C++ standard is
ISO/IEC 14882:2011 (C++11). It includes ISO/IEC 9899:1999 (C99)
as a normative reference.

Old versions of standard do not cease to exist when ISO declares
them obsolete.

Having said all that, I personally would like to see much better
support for the current C standard, and I find, for example,
Microsoft's decision not to support either C99 or C11 annoying.
But Microsoft doesn't get enough money from me for my opinion to
be very relevant to them.

 

[Kaz Kylheku]

This is incorrect the C11 standard states explicitly:

I did not write anything incorrect ...

"This third edition cancels and replaces the second edition, ISO/IEC
9899:1999, as
corrected by ISO/IEC 9899:1999/Cor 1:2001, ISO/IEC 9899:1999/Cor 2:2004, and
ISO/IEC 9899:1999/Cor 3:2007."

thus the only way to be standards compliant is by supporting the latest
standard.

... because I was addressing myself to the term "broken", and did not
write a single word about the term "standards compliant".

If you only support C90 or C99 then you are not standards
compliant as the C11 standard explicitly states it cancels and replaces
all previous standards. Likewise C99 stated that it canceled and
replaced C90.

That does not make existing copies of C90 spontaneously combust.
People continue to have old documents in their libraries.

So there continues to be a C90 language, with a defining document. We can write
something which conforms to that document.

Whether the result is called "standard conforming" is about as important as
debating how many angels you can fit on on top of the head of a modern day
comp.lang.c regular.

 

[Alan Curry]

That does not make existing copies of C90 spontaneously combust.

Careful, Amazon might read that as a feature request for the next Kindle.

 

[Chicken McNuggets]

Having said all that, I personally would like to see much better
support for the current C standard, and I find, for example,
Microsoft's decision not to support either C99 or C11 annoying.
But Microsoft doesn't get enough money from me for my opinion to
be very relevant to them.

I take on board the possibility that I am being overly pedantic but the
point of the original post that lead to this somewhat off-topic
discussion was the intention to get more C compilers to support the full
C11 standard irrespective of whether they also continue to support
previous C standards (via the common -std=cxx compiler switch).

I fail to see how trying to get more compilers to fully support the C11
standard is a bad thing.

Specifically this response struck me as being ridiculous and somewhat
idiotic:

No, it isn't. Not only is is not essential, it is counterproductive.
The "C community" should just go about its business and not mess with
its compilers or libraries, except in those areas where they are
foudn to be broken.

(Apologies if the quoting above appears broken. Thunderbird isn't the
best NNTP client in the world)

 

[James Kuyper]

On 10/05/2012 02:33 PM, Chicken McNuggets wrote:
....

I take on board the possibility that I am being overly pedantic but the
point of the original post that lead to this somewhat off-topic
discussion was the intention to get more C compilers to support the full
C11 standard irrespective of whether they also continue to support
previous C standards (via the common -std=cxx compiler switch).

I'm in favor of it, but I think that labeling it as "essential" was
excessive. "Desirable" is about the strongest thing I can say about it.
C99 had lots of nice new features I wanted to try out; C2011 made some
of them optional, and the only new features it added were ones I'm not
especially interested in. The ability to write multi-threaded code in an
OS-independent fashion is probably of value to someone, but I've no
particular need for it.

I fail to see how trying to get more compilers to fully support the C11
standard is a bad thing.

Specifically this response struck me as being ridiculous and somewhat
idiotic:

I agree - a library doesn't need to be broken in order to benefit from
an upgrade. On the other hand, an upgrade isn't necessarily beneficial,
either. If he'd followed up with a claim that there was something
actively wrong with C2011, or at least that it added nothing of value,
his comments would have made more sense. A reasonable argument of that
kind could be made, based upon the fact that various useful features of
C99 have been made optional. I've got him killfiled, but I just checked
with groups.google.com, and he does not appear to have made any such
argument.

(Apologies if the quoting above appears broken. Thunderbird isn't the
best NNTP client in the world)

The quoting does not appear to be broken. Have you seen Thunderbird
generate quotation errors? If, of what kind?

 

[Keith Thompson]

I take on board the possibility that I am being overly pedantic but the
point of the original post that lead to this somewhat off-topic
discussion was the intention to get more C compilers to support the full
C11 standard irrespective of whether they also continue to support
previous C standards (via the common -std=cxx compiler switch).

On the other hand, given a compiler that supports C11, having it also
support C90 and C99 shouldn't be all that difficult; it would mostly be
a matter of disabling certain features. And of course the reason it's
common for compilers to support older standards is that they've evolved
from earlier compilers.

On the other other hand, for a newly developed C11 compiler it might not
be worth the effort to support the earlier standards as well.

I fail to see how trying to get more compilers to fully support the C11
standard is a bad thing.

Specifically this response struck me as being ridiculous and somewhat
idiotic:

On 26/09/2012 19:11, Kaz Kylheku wrote:

[snip]

I'm not going to comment on that; perhaps you should have posted your
followup to Kaz's article rather than mine. (He's in my killfile,
so I probably hadn't read it.)

 

[Kaz Kylheku]

I take on board the possibility that I am being overly pedantic but the
point of the original post that lead to this somewhat off-topic
discussion was the intention to get more C compilers to support the full
C11 standard irrespective of whether they also continue to support
previous C standards (via the common -std=cxx compiler switch).

I can't think of any computational problem that I cannot solve until
a C11 compiler falls out of the sky and lands in my lap.

Is there some secret bit manipulation in the CPU that I cannot unlock
otherwise?

I fail to see how trying to get more compilers to fully support the C11
standard is a bad thing.

Software development carries risks. Unnecessary software development,
therefore, carries unnecessary risks.

If a C11 upgrade to a compiler breaks some user's code which doesn't use
anything from C11, the net value of that upgrade is negative.

 

[Rui Maciel]

I can't think of any computational problem that I cannot solve until
a C11 compiler falls out of the sky and lands in my lap.

Is there some secret bit manipulation in the CPU that I cannot unlock
otherwise?

This is a non-argument. If solving computational problems was the only
metric then, as it's possible to solve them with programs coded purely in
assembly, every high level programming language would be as useless as you
are making C11 out to be. And this would be a silly thing to say.

Software development carries risks. Unnecessary software development,
therefore, carries unnecessary risks.

If a C11 upgrade to a compiler breaks some user's code which doesn't use
anything from C11, the net value of that upgrade is negative.

No. It would mean that you had upgraded to a broken compiler, and that you
were less than rigorous with your decision to rely on a brand new compiler
to use in production. This has absolutely nothing to do with C11.


Rui Maciel

 

[Chicken McNuggets]

I can't think of any computational problem that I cannot solve until
a C11 compiler falls out of the sky and lands in my lap.

Is there some secret bit manipulation in the CPU that I cannot unlock
otherwise?

The point is not whether you can perform some task with previous C
standards or not. It is whether you can perform those tasks in an easier
manner. For instance threading in C11 promises to be much easier as you
will have a threading library that is cross platform compatible built
into the language itself rather than having to rely on third party
implementations. Writing code that works on Windows and UNIX will be
much easier.

No one is claiming that C99, C90 or even K&R C are not Turing complete
the point is just that additional language features make certain
development tasks easier.

Software development carries risks. Unnecessary software development,
therefore, carries unnecessary risks.

If a C11 upgrade to a compiler breaks some user's code which doesn't use
anything from C11, the net value of that upgrade is negative.

I would be highly surprised if an upgrade to C11 in a compiler would
break existing code especially as most compilers (all that I know of)
allow you to specify the standard you wish to compile your code against
(with the -std=cxx command line switch).

If it was shown that a particular compiler broke existing code due to an
upgrade to C11 I would imagine that the compiler vendor would treat that
problem with the up-most priority.

Regardless refusing to implement modern standards because of the risk of
problems seems somewhat short sighted. There are always risks no matter
what you are doing. Life is all about making decisions based on the
possible risk and the eventual benefit. Personally I feel that
implementing C11 is worth the (small) risk.

Let us not forget that C compiler vendors are used to implementing new C
standards. They did it when they transitioned from K&R to ANSI (with the
possible C90 and C94 being in there depending on whether you count them
as actual standards or merely just modifications of existing standards)
then to C99 and now to C11.

 

[Rui Maciel]

I'm in favor of it, but I think that labeling it as "essential" was
excessive. "Desirable" is about the strongest thing I can say about it.
C99 had lots of nice new features I wanted to try out; C2011 made some
of them optional, and the only new features it added were ones I'm not
especially interested in. The ability to write multi-threaded code in an
OS-independent fashion is probably of value to someone, but I've no
particular need for it.

Truth be told, pthreads already provided a way to write multi-threaded code
in an OS-independent fashion for nearly a decade, now. The only thing that
blocked pthreads' adoption was Microsoft's unwillingness to support it.
Now, looking at Microsoft's notorious unwillingness to comply with the C
standard, I believe that C11 won't change that as well.

As a side note, it would be nice to know why a company insists in being
involved in the definition of a new standard, actively shaping how it
evolves and what is shoved into it, if the people behind it have absolutely
no intention to implement their own work.


Rui Maciel

 

[Kaz Kylheku]

This is a non-argument. If solving computational problems was the only
metric then, as it's possible to solve them with programs coded purely in
assembly, every high level programming language would be as useless as you
are making C11 out to be. And this would be a silly thing to say.

If expressivity and power are also metrics, then some piffling upgrades to high
level languages do look genuinely useless.

 

[Rui Maciel]

If expressivity and power are also metrics, then some piffling upgrades to
high level languages do look genuinely useless.

They can't be metrics because they are vague to the point they are
essentially meaningless. We are referring to a set of features which might
be useful to some people, anf if C11 isn't supported then we simply can't
rely on them to exist.

You can spend hours discussing your indifference towards the features
introduced in C11, but you need to understand that that's only your
subjective opinion on a subject. Meanwhile, depriving the world of those
features is something which isn't adequately justified with an uninformed
"meh".


Rui Maciel