The Semantics of 'volatile'

[MikeWhy]

Do you mean a cache- or bus-lock? Good heavens, no. I'd throw out that
compiler with yesterday's bad news. If the developer wants a cache lock,
he'll have to code one explicitly. Volatile means nothing more than to
limit
the optimizations.

What volatile means is determined almost entirely by a decision
that is implementation-defined. If an implementation chooses to
interpret volatile so that it supplies a cache lock, it may do
so. Or a bus lock. Or no lock at all. The Standard grants an
enormous amount of latitude to the implementation in this area.
....


In the absence of volatile, the only limits for how the abstract
machine can map onto a physical machine is the "as if" rule and
the minimum requirements on a conforming implementation as stated
in 5.1.2.3. Using volatile tightens those limits, but (and here
is the point), the Standard /doesn't say what physical machine
model must match the abstract machine at volatile access points/.
Your comment, "[volatile] means simply to not make assumptions
about the value referenced by the variable", isn't completely
wrong, but it's an oversimplification. The reason is, when one
talks about "the" value referenced by a variable, those comments
must take place in the context of a particular memory model.
Since the Standard doesn't specify one, we can't really talk about
what volatile does without taking that implementation-defined
choice into account. A cache lock memory model is different from
a bus lock memory model, etc.

Do those comments make more sense now?

Give me an example. Given a volatile qualifier, what memory context makes it
reasonable and useful for the compiler to generate a cache lock where the
developer didn't specifically code one? I can't think of one. If the
compiler can generate opcodes to do so, the developer can also write code to
do so if he had wanted one. Or are we talking something much simpler,
something like flipping a bit in a R/W register, as on a baby PIC?

The point of standardizing a language is to give some assurance that the
same code run through different, compliant compilers will generate
functionally identical behavior. This puts a limit on just how unspecified
"implementation defined" loopholes are in the language spec. Volatile
limits the actions of the compiler, not grants more freedom through the
implementation-defined loophoole.

 

[Tim Rentsch]

Do you mean a cache- or bus-lock? Good heavens, no. I'd throw out that
compiler with yesterday's bad news. If the developer wants a cache lock,
he'll have to code one explicitly. Volatile means nothing more than to
limit
the optimizations.

What volatile means is determined almost entirely by a decision
that is implementation-defined. If an implementation chooses to
interpret volatile so that it supplies a cache lock, it may do
so. Or a bus lock. Or no lock at all. The Standard grants an
enormous amount of latitude to the implementation in this area.
...


In the absence of volatile, the only limits for how the abstract
machine can map onto a physical machine is the "as if" rule and
the minimum requirements on a conforming implementation as stated
in 5.1.2.3. Using volatile tightens those limits, but (and here
is the point), the Standard /doesn't say what physical machine
model must match the abstract machine at volatile access points/.
Your comment, "[volatile] means simply to not make assumptions
about the value referenced by the variable", isn't completely
wrong, but it's an oversimplification. The reason is, when one
talks about "the" value referenced by a variable, those comments
must take place in the context of a particular memory model.
Since the Standard doesn't specify one, we can't really talk about
what volatile does without taking that implementation-defined
choice into account. A cache lock memory model is different from
a bus lock memory model, etc.

Do those comments make more sense now?

Give me an example. Given a volatile qualifier, what memory context makes it
reasonable and useful for the compiler to generate a cache lock where the
developer didn't specifically code one? I can't think of one. If the
compiler can generate opcodes to do so, the developer can also write code to
do so if he had wanted one. Or are we talking something much simpler,
something like flipping a bit in a R/W register, as on a baby PIC?

Oh, I never said it was reasonable, only that it's allowed.

The point of standardizing a language is to give some assurance that the
same code run through different, compliant compilers will generate
functionally identical behavior. This puts a limit on just how unspecified
"implementation defined" loopholes are in the language spec. Volatile
limits the actions of the compiler, not grants more freedom through the
implementation-defined loophoole.

You're right that volatile limits the actions of the compiler, but how
much they are limited is determined by an implementation-defined
choice, and that choice has so much leeway that we cannot in general
make any guarantees about restrictions volatile imposes.

I'm not trying to advocate that the Standard adopt a particular
memory model for volatile, or even that it limit the set of
choices of memory model that an implementation can choose amongst.
Maybe that's a good idea, maybe it isn't, I just don't know.
I /do/ think what memory models each implementation supports should
be described both more explicitly and more specifically, for the
benefit of both developers and implementors.

 

[MikeWhy]

Oh, I never said it was reasonable, only that it's allowed.

Yeah, I also can't dream up a scenario where that would be justified.

As to what is allowed in context of conforming to the standard... The
specific example takes an extreme, unwarranted liberty that has noticeable
side effects and performance implications. That's a defect. A bug.

 

[Chris M. Thomasson]

Do you mean a cache- or bus-lock? Good heavens, no. I'd throw out that
compiler with yesterday's bad news. If the developer wants a cache lock,
he'll have to code one explicitly. Volatile means nothing more than to
limit
the optimizations.

What volatile means is determined almost entirely by a decision
that is implementation-defined. If an implementation chooses to
interpret volatile so that it supplies a cache lock, it may do
so. Or a bus lock. Or no lock at all. The Standard grants an
enormous amount of latitude to the implementation in this area.
...


In the absence of volatile, the only limits for how the abstract
machine can map onto a physical machine is the "as if" rule and
the minimum requirements on a conforming implementation as stated
in 5.1.2.3. Using volatile tightens those limits, but (and here
is the point), the Standard /doesn't say what physical machine
model must match the abstract machine at volatile access points/.
Your comment, "[volatile] means simply to not make assumptions
about the value referenced by the variable", isn't completely
wrong, but it's an oversimplification. The reason is, when one
talks about "the" value referenced by a variable, those comments
must take place in the context of a particular memory model.
Since the Standard doesn't specify one, we can't really talk about
what volatile does without taking that implementation-defined
choice into account. A cache lock memory model is different from
a bus lock memory model, etc.

Do those comments make more sense now?

Give me an example. Given a volatile qualifier, what memory context makes
it reasonable and useful for the compiler to generate a cache lock where
the developer didn't specifically code one? I can't think of one.

FWIW, MSVC compilers, versions 8 and above, automatically insert
load-acquire/store-release barriers for volatile loads and stores on certain
architectures (e.g., PowerPC):
_______________________________________________________________
void*
volatile_load(
void** p
) {
void* v = ATOMIC_LOAD(p);
MEMBAR #LoadStore | #LoadLoad;
return v;
}


void*
volatile_store(
void** p,
void* v
) {
MEMBAR #LoadStore | #StoreStore;
ATOMIC_STORE(p, v);
return v;
}
_______________________________________________________________




Please note that this is NOT strong enough for mutual-exclusion such that
you cannot use these memory barrier guarantees to build Petersons Algorithm.
You would need to insert a stronger membar (e.g., MEMBAR #StoreLoad |
#StoreStore) in order to prevent subsequent load from hoisting up above the
previous store in the lock acquire portion of the algorithm. However, it is
strong enough for a DCL algorithm or a producer/consumer example:
_______________________________________________________________
int data = 0;
atomic_word volatile g_flag = 0;


void single_producer() {
data = 1234;
g_flag = 1;
}


void multiple_consumers() {
while (! g_flag) backoff();
assert(data == 1234);
}
_______________________________________________________________




However, these barriers are actually too strong here such that the
#LoadStore restrain is not required for the example... On the SPARC you
could write it as:
_______________________________________________________________
int data = 0;
atomic_word g_flag = 0;


void single_producer() {
data = 1234;
MEMBAR #StoreStore;
g_flag = 1;
}


void multiple_consumers() {
while (! g_flag) backoff();
MEMBAR #LoadLoad;
assert(data == 1234);
}
_______________________________________________________________




Even then, the barriers can be too strong for architectures that support
implicit data-dependant load barriers (e.g., all but the DEC Alpha). You can
write it this way:
_______________________________________________________________
int data = 0;
int* g_flag = 0;


void single_producer() {
data = 1234;
g_flag = &data;
}


void multiple_consumers() {
while (! g_flag) backoff();
assert(*g_flag == 1234);
}
_______________________________________________________________

 

[Tim Rentsch]

Yeah, I also can't dream up a scenario where that would be justified.

As to what is allowed in context of conforming to the standard... The
specific example takes an extreme, unwarranted liberty that has noticeable
side effects and performance implications. That's a defect. A bug.

I don't agree. There are lots of ways that the Standard gives
license to implementations to do extreme, absurd, or ridiculous
things, but practically speaking the effects are minimal. For
example, there's nothing stopping an implementation from choosing
sizeof(int) == 1000000000. It's hard to grant /some/ freedoms
like this to implementations without also allowing extreme
choices that seem ridiculous.

Should the ridiculous choices be forbidden (even assuming that
suitable language could be found to do that)? I would say no,
because what seems ridiculous in one context might very well have
useful implications in another context.

Where I think the Standard falls short here is in explaining what
it is that must be documented when an implementation adopts a
particular "volatile access" model. Also, the question of what
consequences must follow given a particular choice of memory
model, which I believe the Standard /does/ mean to specify, is
not spelled out very clearly in the current Standard text.

 

[Tim Rentsch]

Give me an example. Given a volatile qualifier, what memory context makes
it reasonable and useful for the compiler to generate a cache lock where
the developer didn't specifically code one? I can't think of one.

FWIW, MSVC compilers, versions 8 and above, automatically insert
load-acquire/store-release barriers for volatile loads and stores on certain
architectures (e.g., PowerPC):
[...examples omitted...]

Thank you! An excellent set of examples.

 

[MikeWhy]

I don't agree. There are lots of ways that the Standard gives
license to implementations to do extreme, absurd, or ridiculous
things, but practically speaking the effects are minimal. For
example, there's nothing stopping an implementation from choosing
sizeof(int) == 1000000000. It's hard to grant /some/ freedoms
like this to implementations without also allowing extreme
choices that seem ridiculous.

I didn't say that wasn't conforming. Whether it is or isn't is a separate
issue. I said I consider it a bug, a product defect, and if uncorrected
makes it unusable for my needs.

 

[Tim Rentsch]

I didn't say that wasn't conforming. Whether it is or isn't is a separate
issue. I said I consider it a bug, a product defect, and if uncorrected
makes it unusable for my needs.

Did we both misunderstand each other? I was talking about
whether or not the Standard should be considered to have a bug
(and in this regard I don't think it should), not whether
the example implementation should be considered to have a bug.

In fact, I wouldn't say this particular implementation decision
merits the term "bug" either. It may be a poor decision in terms
of your needs. Possibly it's a poor decision in terms of most
other people's needs also. But as long as the implementation is
conforming and does what its developers intended, at worst the
"feature" represents poor judgment. And who knows, it may be
just what someone is looking for, and they may be quite glad
to find an implementation that implements volatile "correctly".

 

[MikeWhy]

Did we both misunderstand each other? I was talking about
whether or not the Standard should be considered to have a bug
(and in this regard I don't think it should), not whether
the example implementation should be considered to have a bug.

In fact, I wouldn't say this particular implementation decision
merits the term "bug" either. It may be a poor decision in terms
of your needs. Possibly it's a poor decision in terms of most
other people's needs also. But as long as the implementation is
conforming and does what its developers intended, at worst the
"feature" represents poor judgment. And who knows, it may be
just what someone is looking for, and they may be quite glad
to find an implementation that implements volatile "correctly".

I think we're agreeing. Conformance to the standard is orthogonal to
correctness in the implementation.

 

[James Kuyper]

MikeWhy wrote:
....

I think we're agreeing. Conformance to the standard is orthogonal to
correctness in the implementation.

Conformance with the standard is part of correctness, not orthogonal to
it, at least for any compiler which is intended to implement that standard.

 

[Tim Rentsch]

I think we're agreeing. Conformance to the standard is orthogonal to
correctness in the implementation.

I take your point, although I wouldn't use the term "correctness"
to label the attribute that (I think) you mean to reference.
Perhaps "quality" or "appropriateness" or something along those
lines. "Correctness" is measured relative to a specification,
but what (I think) you're talking about is some sort of independent
judgment, which has no specification.

 

[luserXtrog]

I take your point, although I wouldn't use the term "correctness"
to label the attribute that (I think) you mean to reference.
Perhaps "quality" or "appropriateness" or something along those
lines.  "Correctness" is measured relative to a specification,
but what (I think) you're talking about is some sort of independent
judgment, which has no specification.

Perhaps according to the "right thinking" of the eightfold path?

Now that you're awake, this gem from the archive might be interesting.
It appears to be the genesis of "volatile".

-- \nlxt

Message-ID: <bnews.n44a.144>
Newsgroups: net.lang.c
Path: utzoo!decvax!cca!ima!n44a!dan
X-Path: utzoo!decvax!cca!ima!n44a!dan
From: n44a!dan
Date: Thu Apr 28 05:47:51 1983
Subject: C and real hardware
Posted: Wed Apr 27 15:34:30 1983
Received: Thu Apr 28 05:47:51 1983


In C code such as device drivers, one often encounters code like:

while (ADDR->c_reg&BUSY)
;

where one is testing bits of a hardware device register, and waiting
for a change.

In there anything in the C specification which would prevent a good
(but obviously not perfect) optimizer from reading the value of the
device
register just once, stuffing it into a CPU general purpose register
and looping
on its value ?
I have noticed that at least some of the 4xbsd makefiles seem to
purposely avoid the optimizer - is this sort of thing the reason ?
Might not C benefit from some additional type (e.g. "unregister")
which would tell an optimizer what was up. I believe DEC's MicroPower
Pascal has
a type called "volatile" which prevents this sort of thing.

Dan Ts'o

 

[Tim Rentsch]

[SNIP]

I think we're agreeing. Conformance to the standard is orthogonal to
correctness in the implementation.

I take your point, although I wouldn't use the term "correctness"
to label the attribute that (I think) you mean to reference.
Perhaps "quality" or "appropriateness" or something along those
lines. "Correctness" is measured relative to a specification,
but what (I think) you're talking about is some sort of independent
judgment, which has no specification.

Perhaps according to the "right thinking" of the eightfold path?

Or whatever criteria the judger chooses -- that's why it's
an /independent/ judgment (and explains why there is no
specification, because each judger makes their own decision
about what requirements should be satisfied).

Now that you're awake, this gem from the archive might be interesting.
It appears to be the genesis of "volatile".
[snip historial message]

There are two pertinent aspects that may be considered relevant
here: the choice of term ('volatile'), and what it does (ie, the
intended meaning, or what it's for).

For choice of term, I think the ordinary English meaning is
close enough so that it's reasonable to consider the ordinary
word as the origin of the term. That it was used earlier in
another language simply means they found the standard meaning
close enough for their purposes -- no different than 'if',
'break', 'while', etc.

For what it does, people more familiar with the history than I am
have said (and I believe there's a fair amount of truth in this)
that members of the committee were divided between two related
but distinct meanings/uses, namely, for accessing special memory
locations or hardware registers, and for inhibiting optimization
for some sort of inter-process sychronization. Rather than
settle on a single purpose, the two different uses were conflated
into a single one, both brought into play using the keyword
'volatile'. So even if the earlier precedent may have played a
role in motivating a need, I think 'volatile' in its current form
has a broader base than just the "we need to access a special
memory location" kind of purpose. Remember, language features
put in to enable/inhibit various low-level machine details
weren't new even in the 1980's. To give just one example, PL/I
had such things going back to the 1960's.

 

[luserXtrog]

[SNIP]
I think we're agreeing. Conformance to the standard is orthogonal to
correctness in the implementation.
I take your point, although I wouldn't use the term "correctness"
to label the attribute that (I think) you mean to reference.
Perhaps "quality" or "appropriateness" or something along those
lines.  "Correctness" is measured relative to a specification,
but what (I think) you're talking about is some sort of independent
judgment, which has no specification.

Perhaps according to the "right thinking" of the eightfold path?

Or whatever criteria the judger chooses -- that's why it's
an /independent/ judgment (and explains why there is no
specification, because each judger makes their own decision
about what requirements should be satisfied).

Now that you're awake, this gem from the archive might be interesting.
It appears to be the genesis of "volatile".
[snip historial message]

There are two pertinent aspects that may be considered relevant
here:  the choice of term ('volatile'), and what it does (ie, the
intended meaning, or what it's for).

For choice of term, I think the ordinary English meaning is
close enough so that it's reasonable to consider the ordinary
word as the origin of the term.  That it was used earlier in
another language simply means they found the standard meaning
close enough for their purposes -- no different than 'if',
'break', 'while', etc.
Solid.

For what it does, people more familiar with the history than I am
have said (and I believe there's a fair amount of truth in this)
that members of the committee were divided between two related
but distinct meanings/uses, namely, for accessing special memory
locations or hardware registers, and for inhibiting optimization
for some sort of inter-process sychronization.  

Aren't these the same thing: To inhibit optimizations so as enforce
memory access? Is the distinction to be made between sharing the
memory with another process vs. a piece of hardware?

Rather than

settle on a single purpose, the two different uses were conflated
into a single one, both brought into play using the keyword
'volatile'.  So even if the earlier precedent may have played a
role in motivating a need, I think 'volatile' in its current form
has a broader base than just the "we need to access a special
memory location" kind of purpose.  Remember, language features
put in to enable/inhibit various low-level machine details
weren't new even in the 1980's.  To give just one example, PL/I
had such things going back to the 1960's.

That's so much clearer: nice and vague with an anecdotey feel.

I know I'd shouting from the back-row, here; so feel free to
be as curt as you wish. I really don't know what I'm talking
about, never having used the thing. But I thought I had this
pretty-well gelled; now it's all runny.

 

[MikeWhy]

[SNIP]

I think we're agreeing. Conformance to the standard is orthogonal to
correctness in the implementation.

I take your point, although I wouldn't use the term "correctness"
to label the attribute that (I think) you mean to reference.
Perhaps "quality" or "appropriateness" or something along those
lines. "Correctness" is measured relative to a specification,
but what (I think) you're talking about is some sort of independent
judgment, which has no specification.

Perhaps according to the "right thinking" of the eightfold path?

Or whatever criteria the judger chooses -- that's why it's
an /independent/ judgment (and explains why there is no
specification, because each judger makes their own decision
about what requirements should be satisfied).

No more so than any other part of the standard. "Volatile" means "volatile",
not "do what you damned well please".

 

[Tim Rentsch]

Aren't these the same thing: To inhibit optimizations so as enforce
memory access? Is the distinction to be made between sharing the
memory with another process vs. a piece of hardware?

I suppose they could be, depending on how generically the terms
were meant. In the particular case, I don't think they were,
because of different assumptions about what determines the
behavior on the other accesses. For hardware memory registers, it
could be just about anything; for inter-process synchronization,
the natural assumption would be that the other process would also
be a C program (and so would view 'volatile' in the same way as
this program). There's a big difference between those two.

Like I said, I wasn't there, but that's basically how I took
the comments about what actually happened.

 

[Tim Rentsch]

[SNIP]

I think we're agreeing. Conformance to the standard is orthogonal to
correctness in the implementation.

I take your point, although I wouldn't use the term "correctness"
to label the attribute that (I think) you mean to reference.
Perhaps "quality" or "appropriateness" or something along those
lines. "Correctness" is measured relative to a specification,
but what (I think) you're talking about is some sort of independent
judgment, which has no specification.

Perhaps according to the "right thinking" of the eightfold path?

Or whatever criteria the judger chooses -- that's why it's
an /independent/ judgment (and explains why there is no
specification, because each judger makes their own decision
about what requirements should be satisfied).

No more so than any other part of the standard. "Volatile" means "volatile",
not "do what you damned well please".

I think you need to make up your mind. If what you mean by
"correctness in the implementation" depends on implementing
volatile in the way the Standard says, then it is not orthogonal
to "conformance to the standard". Either the two conditions
/are/ orthogonal, in which case judging correctness is independent
of the Standard, or judging correctness depends on the Standard,
in which case they are /not/ orthogonal. It can't be both.
Or maybe you mean something different by "orthogonal" than
how the term is usually meant? If so maybe you could explain
that.

 

[Boon]

If it was the same one posted here a few months ago, it started out with
a very basic false assumption about what volatile *means*. Casting the
rest of the paper into doubt as far as I can see.

Are you both referring to the following paper?

"Volatiles Are Miscompiled, and What to Do about It"
http://www.cs.utah.edu/~regehr/papers/emsoft08-preprint.pdf

Regards.

 

[MikeWhy]

[SNIP]

I think we're agreeing. Conformance to the standard is orthogonal
to
correctness in the implementation.

I take your point, although I wouldn't use the term "correctness"
to label the attribute that (I think) you mean to reference.
Perhaps "quality" or "appropriateness" or something along those
lines. "Correctness" is measured relative to a specification,
but what (I think) you're talking about is some sort of independent
judgment, which has no specification.

Perhaps according to the "right thinking" of the eightfold path?

Or whatever criteria the judger chooses -- that's why it's
an /independent/ judgment (and explains why there is no
specification, because each judger makes their own decision
about what requirements should be satisfied).

No more so than any other part of the standard. "Volatile" means
"volatile",
not "do what you damned well please".

I think you need to make up your mind. If what you mean by
"correctness in the implementation" depends on implementing
volatile in the way the Standard says, then it is not orthogonal
to "conformance to the standard". Either the two conditions
/are/ orthogonal, in which case judging correctness is independent
of the Standard, or judging correctness depends on the Standard,
in which case they are /not/ orthogonal. It can't be both.
Or maybe you mean something different by "orthogonal" than
how the term is usually meant? If so maybe you could explain
that.

If the implementation can conform to the standard and not generate the
desired result, conformance is orthogonal to correctness. Isn't that your
argument? That implementation-defined means anything and everything? (Don't
bother. That was rhetorical. I'm outa this one.)

 

[John Devereux]

Are you both referring to the following paper?

"Volatiles Are Miscompiled, and What to Do about It"
http://www.cs.utah.edu/~regehr/papers/emsoft08-preprint.pdf

That is the one I meant. Their first example (2.1) is wrong I think:

======================================================================

volatile int buffer_ready;
char buffer[BUF_SIZE];
void buffer_init() {
int i;
for (i=0; i<BUF_SIZE; i++)
buffer = 0;
buffer_ready = 1;
}

"The for-loop does not access any volatile locations, nor does it
perform any side-effecting operations. Therefore, the compiler is free
to move the loop below the store to buffer_ready, defeating the
developer's intent."

======================================================================

The problem is that the compiler is *not* free to do this (as far as I
can see). Surely clearing the buffer *is* a side effect?

The example is meant to illustrate "what does volatile mean". If it does
not mean what they think it does, the other claims seem suspect.