miscompilation of volatiles?

J

John Regehr

I'm trying to figure out what -- if any -- ammunition the C standard
gives me for identifying bugs in the translation of C code containing
volatile variables. For example consider this fragment:

volatile int g_1;

void self_assign (void)
{
g_1 = g_1;
}

One of gcc's embedded ports translates this into the following asm
when invoked with -Os:

self_assign:
ret

On the other hand, when optimizations are disabled the same compiler
produces object code that properly loads from g_1 and then stores the
loaded value back into g_1.

At an informal level, the optimized code is obviously wrong in the
sense that any embedded C programmer would expect this function to
load from g_1 and then store back to it.

What I am trying to figure out is, is this output buggy from the
language lawyer point of view? On one hand the standard tells us that
"any expression referring to such an object shall be evaluated
strictly according to the rules of the abstract machine." This
appears to clearly call for a load and then a store. On the other
hand the standard also says "What constitutes an access to an object
that has volatile-qualified type is implementation-defined." This
seems to admit an implementation that specifies that volatile-
qualified types are never accessed, regardless of what the source code
looks like. But is it legal for the compiler to access the object at
some optimization levels and not at others?

Any help appreciated.
 
P

Peter Nilsson

John Regehr said:
I'm trying to figure out what -- if any -- ammunition the C
standard gives me for identifying bugs in the translation of
C code containing volatile variables.

The ammunition you have is the standard itself, and the
implementation documentation.
 For example consider this fragment:

volatile int g_1;

I think you're unlikely to find any compile that will treat
'access' to this variable any differently from any other
non-volatile global. Since it is the compiler that allocates
space, it can determine whether there are any inherent
access considerations involved. As I said, I think most
compilers will not treat this declaration as anything
special beyond the volatile qualification in and of itself.
void self_assign (void)
{
  g_1 = g_1;
}

One of gcc's embedded ports translates this into the
following asm when invoked with -Os:

self_assign:
        ret

On the other hand, when optimizations are disabled the
same compiler produces object code that properly loads
from g_1 and then stores the loaded value back into g_1.

Which simply implies that self assignment will only be
optimised at higher levels.

Instead, try...

void self_assign (volatile int *vp)
{
*vp = *vp;
}
At an informal level, the optimized code is obviously wrong
in the sense that any embedded C programmer would expect
this function to load from g_1 and then store back to it.

Would it? Even an embedded implementation would still be
managing control of the allocation. Note that many compilers
offer extensions to place variables in certain memory
addresses. Also note that access to memory mapped I/O (et al)
can (and is) also done via constructs like...

#define RESET (* (volatile unsigned short *) 0xFFFE)
What I am trying to figure out is, is this output buggy from
the language lawyer point of view?
No.

 On one hand the standard tells us that "any expression
referring to such an object shall be evaluated strictly
according to the rules of the abstract machine."  This
appears to clearly call for a load and then a store.

It's valid for the optimised version under the 'as if' rule.
Note that you cannot modify your program to detect the
optimisation and remain strictly conforming.
 On the other hand the standard also says "What constitutes
an access to an object that has volatile-qualified type is
implementation-defined."

Precisely, which is why you may be better off posting your
question to a gcc group.
 This seems to admit an implementation that specifies that
volatile- qualified types are never accessed, regardless
of what the source code looks like.

Not quite.
 But is it legal for the compiler to access the object at
some optimization levels and not at others?

Yes. As pointed out, all you've demonstrated is that higher
optimisation level will implement a no-op more efficiently.
 
J

John Regehr

I think you're unlikely to find any compile that will treat
'access' to this variable any differently from any other
non-volatile global. Since it is the compiler that allocates
space, it can determine whether there are any inherent
access considerations involved. As I said, I think most
compilers will not treat this declaration as anything
special beyond the volatile qualification in and of itself.

On the contrary, Microsoft, Intel, CodeWarrior, and all gcc4-based
compilers that I have tried turn the code I sent into a load then a
store. (The platform in question here, msp430, is not yet supported
by gcc4.)

Anyway I do not buy your reasoning. The volatile qualifier tells the
compiler not to perform precisely the kind of reasoning about storage
that you describe here.
Note that you cannot modify your program to detect the
optimisation and remain strictly conforming.

But I can, like this:

extern volatile int g_1;

void self_assign (void)
{
g_1 = g_1;
}

Now the storage allocation is not under control of the compiler, which
still emits the function doing nothing. I can detect the optimization
by using the linker to place g_1 onto a memory-mapped I/O register.
Precisely, which is why you may be better off posting your
question to a gcc group.

They know about this problem... I'm trying to get this all straight
because I do research in tools for embedded systems, not because I
want to work around specific gcc bugs.

John Regehr
 
P

Peter Nilsson

John Regehr said:
...  The volatile qualifier tells the compiler not to
perform precisely the kind of reasoning about storage
that you describe here.

There is only 1 scenario where the standard actually
precludes certain optimisations relating to volatile
objects, and that's in relation to automatic storage
in the presence of setjmp().[1] The only other case that
comes close is that objects of type sig_atomic_t type
may need to be volatile to work properly.

But appart from those, you can remove all volatile
qualifiers from any strictly conforming program and
not change the semantics. This is all the standard
cares about. Optimisation at the assembler level is
not defined by the standards. It's a QoI issue.

[1] It's not actually phrased in that way, rather it
says that in the presence of setjmp() non-volatile
automatic objects in the calling original function
have indeterminate values.
But I can, like this:

extern volatile int g_1;

This would not change the semantics of a strictly conforming
program. That it changes the assembler code is immaterial
so long as the semantics are preserved.
void self_assign (void)
{
  g_1 = g_1;
}

Now the storage allocation is not under control of the
compiler, which still emits the function doing nothing.
 I can detect the optimization by using the linker to place
g_1 onto a memory-mapped I/O register.

If you're going to define variables other than through
standard source definitions, then you are not talking about
standard C. In which case, the standard doesn't have any
guarantees, and clc cannot assist you.

We can only describe what the standard requires from the
virtual machine. However, the standard leaves the definition
of 'access' to the implementation.
They know about this problem...

You haven't demonstrated a problem. At least as far as the
standard is concerned.
I'm trying to get this all straight because I do research
in tools for embedded systems, not because I want to work
around specific gcc bugs.

It sounds like your research is delving into aspects of
C implementations that are outside the scope of the C
standard.
 
J

Jack Klein

I'm trying to figure out what -- if any -- ammunition the C standard
gives me for identifying bugs in the translation of C code containing
volatile variables. For example consider this fragment:

volatile int g_1;

void self_assign (void)
{
g_1 = g_1;
}

One of gcc's embedded ports translates this into the following asm
when invoked with -Os:

self_assign:
ret

On the other hand, when optimizations are disabled the same compiler
produces object code that properly loads from g_1 and then stores the
loaded value back into g_1.

At an informal level, the optimized code is obviously wrong in the
sense that any embedded C programmer would expect this function to
load from g_1 and then store back to it.

What I am trying to figure out is, is this output buggy from the
language lawyer point of view? On one hand the standard tells us that
"any expression referring to such an object shall be evaluated
strictly according to the rules of the abstract machine." This
appears to clearly call for a load and then a store. On the other
hand the standard also says "What constitutes an access to an object
that has volatile-qualified type is implementation-defined." This
seems to admit an implementation that specifies that volatile-
qualified types are never accessed, regardless of what the source code
looks like. But is it legal for the compiler to access the object at
some optimization levels and not at others?

Any help appreciated.

In my opinion, the compiler is broken at the optimization level where
it omits the read and subsequent write of the object. For embedded
use, I would either demand a fix from the vendor or dump the compiler.

The bit in the standard about volatile access being
implementation-defined has been explained at least once by one member
of the committee, on comp.std.c, as referring to cases like the
following:

unsigned char some_func(volatile unsigned char *vucp)
{
return *vucp;
}

Now assuming 8-bit bytes and a typical processor with greater than
8-bit bus width, it is very likely that more than just the first byte
addressed by the pointer will be read. Quite likely, 32, 64, or more
bits will actually be read.

So if vucp actually points to an array of 8-bit hardware registers,
each of which experiences a side-effect when read, C does not
guarantee to only read vucp[0] and not to read vucp[1], vucp[2],
vucp[3], etc. Many processors that have wider busses always perform
full bus width reads, and only use individual byte-enable lines on
writes.

On quite a few hardware platforms, the following function:

void func2(volatile unsigned char *vucp, int x)
{
*vucp = x;
}

....will cause a read of multiple bytes, a change to one of those
bytes, and a rewrite of all of them, one with the new value and the
others with the values read.

So the expressed intent by a member of the committee covers the fact
that hardware may generate more physical (not logical) accesses than
one might expect from just looking at an expression. It is not
intended to allow the compiler to avoid specifically programmed
accesses.

It's unfortunate that so far nobody has been able to generate any
impetus to get this cleaned up.

Nevertheless, an expression that modifies the value of a volatile
object, especially one defined outside the current translation unit,
must be performed, based on the first 3 paragraphs of 5.1.2.3:

"1 The semantic descriptions in this International Standard describe
the behavior of an abstract machine in which issues of optimization
are irrelevant.

2 Accessing a volatile object, modifying an object, modifying a file,
or calling a function that does any of those operations are all side
effects, which are changes in the state of the execution environment.
Evaluation of an expression may produce side effects. At certain
specified points in the execution sequence called sequence points,all
side effects of previous evaluations shall be complete and no side
effects of subsequent evaluations shall have taken place. (A summary
of the sequence points is given in annex C.)

3 In the abstract machine, all expressions are evaluated as specified
by the semantics. An actual implementation need not evaluate part of
an expression if it can deduce that its value is not used and that no
needed side effects are produced (including any caused by calling a
function or accessing a volatile object)."

If the volatile object is defined externally, there is no way that the
compiler can guarantee that it can accurately deduce that the two
accesses of the volatile object produce no needed side effects.

--
Jack Klein
Home: http://JK-Technology.Com
FAQs for
comp.lang.c http://c-faq.com/
comp.lang.c++ http://www.parashift.com/c++-faq-lite/
alt.comp.lang.learn.c-c++
http://www.club.cc.cmu.edu/~ajo/docs/FAQ-acllc.html
 
J

John Regehr

It sounds like your research is delving into aspects of
C implementations that are outside the scope of the C
standard.

I see your point. In that case it is depressing that the standard
describes a language that is useless for implementing operating
systems and embedded systems -- domains where C dominates and has no
obvious successors.

John Regehr
 
J

John Regehr

If the volatile object is defined externally, there is no way that the
compiler can guarantee that it can accurately deduce that the two
accesses of the volatile object produce no needed side effects.

Thanks Jack. Certainly I prefer this interpretation as it leads to a
far more useful language.

As a side note, for most compilers that we have tested, we are finding
corner cases -- generally much more complex than the self-assignment
function I sent -- that are miscompiled at high optimization levels.
In my opinion, the compiler is broken at the optimization level where
it omits the read and subsequent write of the object. For embedded
use, I would either demand a fix from the vendor or dump the compiler.

Right on. This port has other significant problems too. But it is
the only free compiler for an otherwise nice super-low-power 16-bit
architecture :|.

John Regehr
 
C

Chris Torek

[snippage of actual code, but the problem is that one version of
gcc removes entirely two references to a "volatile"-qualified object]
At an informal level, the optimized code is obviously wrong in the
sense that any embedded C programmer would expect this function to
load from g_1 and then store back to it.

Indeed, and as others have mentioned in this thread, this is in fact
a bug in that gcc port.
What I am trying to figure out is, is this output buggy from the
language lawyer point of view?

Most likely (and in the case of gcc, yes, although given the state of
some gcc documentation ... well, read on), but:
On one hand the standard tells us that
"any expression referring to such an object shall be evaluated
strictly according to the rules of the abstract machine." This
appears to clearly call for a load and then a store. On the other
hand the standard also says "What constitutes an access to an object
that has volatile-qualified type is implementation-defined."

What this means is that, in order to *prove* that this is a bug in
the implementation, you must read the documentation that the
implementation is required to provide. Somewhere in this documentation,
there must be some text defining "what constitutes an access to an
object that has volatile-qualified type". Use that wording, plus
the standard's "abstract machine" requirements, to prove the bug.
This seems to admit an implementation that specifies that volatile-
qualified types are never accessed, regardless of what the source code
looks like.

If the compiler documentation says that, the compiler is doing the
"right thing" and it is not a bug. But it (this compiler) is then
worthless for much embedded work, and -- as was once said of a book
-- is not to be put aside lightly, but rather thrown with great
force. :) Of course, the documentation does not say that. A
"good" implementation will say something much more obvious and
appropriate for the target system, and you can then use that to
prove that the compiler is broken.

Given that this is some unspecified port of some unspecified version
of gcc, the documentation may well be missing outright, though.
On the bright side, you will presumably have the gcc source, and
can go fix the bug. (In older versions of gcc, at least, this is
just a matter of testing the "volatile" flag and skipping the
unwanted optimization if it is set. The trick is finding the
place(s) that is/are doing this inappropriate optimization -- and
finding the "volatile" flag, which as I recall is not at all
obvious.)
 
R

Randy Howard

I see your point. In that case it is depressing that the standard
describes a language that is useless for implementing operating
systems and embedded systems -- domains where C dominates and has no
obvious successors.

Given that practically /all/ operating systems and the vast majority of
embedded systems use C, how can your statement possibly be true?
 
R

Randy Howard

Thanks Jack. Certainly I prefer this interpretation as it leads to a
far more useful language.

As a side note, for most compilers that we have tested, we are finding
corner cases -- generally much more complex than the self-assignment
function I sent -- that are miscompiled at high optimization levels.


Right on. This port has other significant problems too. But it is
the only free compiler for an otherwise nice super-low-power 16-bit
architecture :|.

I have used a commercial compiler for the msp430 in the past, and found
it to be quite good. Perhaps free isn't always the right answer.
 
R

Richard

Randy Howard said:
Given that practically /all/ operating systems and the vast majority of
embedded systems use C, how can your statement possibly be true?

"standard"

Few if any of those systems you mention use "standard C" which the likes
of RH and CBF would deign to discuss here.

The Linux kernel, in particular, would probably cause them to choke on
their copies of the standard.
 
C

Chris Hills

I see your point.

I don't
In that case it is depressing that the standard
describes a language that is useless for implementing operating
systems and embedded systems -- domains where C dominates and has no
obvious successors.

The standard does work on embedded systems the problem is a lot of the
time you need hardware specify extensions and there is a small group
here who go very silly if you mention something that is not [their very
own] definition of "Standard C"

You may get a better response on comp.arch.embedded where they are more
open minded.
 
K

Kenny McCormack

"standard"

Few if any of those systems you mention use "standard C" which the likes
of RH and CBF would deign to discuss here.

The Linux kernel, in particular, would probably cause them to choke on
their copies of the standard.

Actually, it is a very good question to ask RH/CBF/et.al what language
the Linux kernel is written in. Note that the(ir) answer cannot be and
will not be "C", since by their lights, that stuff is "not C". They are
explicit when people post stuff like that that this is some other
language - it is "not C" [1].

So, RH/CBF/et al, what language is the Linux kernel written in? (D?
C++, assembler, what?)

Note: You cannot answer "C with extensions" - because that clearly is
"C" - in much the same way that "cake with frosting" is (obviously) "cake".

[1] Insert obvious Godwin reference here, if you see fit.
 
K

Keith Thompson

Chris Hills said:
I see your point.

I don't
In that case it is depressing that the standard
describes a language that is useless for implementing operating
systems and embedded systems -- domains where C dominates and has no
obvious successors.

The standard does work on embedded systems the problem is a lot of the
time you need hardware specify extensions and there is a small group
here who go very silly if you mention something that is not [their
very own] definition of "Standard C"

I don't understand the "[their very own]" qualification. The
definition of "Standard C" is very clear.
You may get a better response on comp.arch.embedded where they are
more open minded.

And where, unlike here, such system-specific extensions are presumably
topical, and the newsgroup is full of experts on such things.

Nobody objects to system-specific extensions. The only objection is
to discussing them here, when there are forums (such as
comp.arch.embedded, comp.unix.programmer, et al) that deal
specifically with the systems in question.
 
J

John Regehr

Thanks Chris (and others), this is all very helpful.

The trick is finding the
place(s) that is/are doing this inappropriate optimization -- and
finding the "volatile" flag, which as I recall is not at all
obvious.)

Yeah. In fact it looks like gcc3-based compilers are deeply flawed in
their treatment of volatiles in the sense that this kind of problem
appears in multiple ports and in a variety of situations, even on very
simple inputs. On the other hand gcc4 appears to get the easy cases
right and has flaws only in some corner cases. Probably a smallish
patch would fix it right up. Anyway it is nice to see things moving
in the right direction.

John Regehr
 
K

Kelsey Bjarnason

[snips]

"standard"

Few if any of those systems you mention use "standard C" which the likes
of RH and CBF would deign to discuss here.

Actually, I suspect a fair number of those systems do, in fact, use
"standard C", plus extensions. You know, extensions, things
expressly allowed by C.

The objections you find extensions are not that these things are not
allowed by C, bur, rather, that they are not topical, as they are not
actually _part of_ C.

One could, theoretically, write a C extension to control vibrators via
cell phones; this would not make sex play topical for c.l.c.

CLC adheres, by and large, to the C language, defined by the
ISO C standard(s). Other groups exist, or can be created, to discuss
extensions not covered by C, be they extensions for programming Windows,
or sound cards, or compression techniques - or vibrators.

Why is this such a difficult concept for some folks to grasp?
 
C

CBFalconer

Kelsey said:
Richard wrote:
.... snip ...


Actually, I suspect a fair number of those systems do, in fact,
use "standard C", plus extensions. You know, extensions, things
expressly allowed by C.

I trust you realize that you are feeding the troll?
 
C

Chris Hills

Kelsey Bjarnason said:
[snips]

"standard"

Few if any of those systems you mention use "standard C" which the likes
of RH and CBF would deign to discuss here.

Actually, I suspect a fair number of those systems do, in fact, use
"standard C", plus extensions. You know, extensions, things
expressly allowed by C.

But apparently not permitted here by some pedants..
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

No members online now.

Forum statistics

Threads
473,744
Messages
2,569,484
Members
44,903
Latest member
orderPeak8CBDGummies

Latest Threads

Top