Defeating Optimisation for memcmp()

[Ben Pfaff]

Ben Pfaff wrote:

I have two related questions. According to K&R2:

"Except that it should diagnose explicit attempts to change
'const' objects, a compiler may ignore these qualifiers."

This statement is given in the context of qualifiers on types,
not qualifiers on pointers. I think that this is intended to
mean that the compiler is not obligated to store const objects
in read-only memory, and that it is not obligated to put
volatile objects in a special section of memory either.

The following was my attempt to test incrementing of a void*. I
think I can imagine situations where this ability would be useful
to bypass pointer incrementation. [...]

I am struggling to understand how this is anything but a non
sequitur. Can you explain?

A void pointer has no stride and cannot be incremented. I find the sentence
very difficult to parse.

By "non sequitur", I mean that CBFalconer's reply seems to have
nothing to do with anything in the text that he quoted. Do you
see any connection?

 

[Martin]

You haven't changed anything as far as the constraint violation on the
call to memcmp is concerned. Both

volatile char x[10];
memcmp(x, ...);
memcmp(&x[3], ...);

and

volatile char *x;
memcmp(x, ...);
memcmp(&x[3], ...);

all pass a 'char pointer to volatile' where 'void pointer to const' is
expected.

OK, thanks for the response.

As has been my aim in this thread, I want to write code that will
verify that a write to CMOS has succeeded, and ensure that the
compiler does not optimise away the verification.

What follows is a solution, based on the previous comments in this
thread. As far as I can tell it looks OK. Obviously, if anyone on
c.l.c. thinks differently, I'd be grateful for the comments.

typedef struct {
...
somestruct s;
...
} CMOSTYPE;

CMOSTYPE cmos;

....

/* Create a volatile pointer to the cmos structure */
CMOSTYPE * volatile pcmos = &cmos;
somestruct s;

/* Assign some values to the members of s */

....

/* Copy s into the cmos structure */
memcpy(&pcmos->s, &s, sizeof(pcmos->s));

/* Check that the CMOS write succeeded */
if (memcmp(&pcmos->s, &s, sizeof(pcmos->s)) != 0)
/* Write failed */

 

[Martin]

I had meant to provide a rationale for the code in my previous
message, but somehow it got missed during the copy & paste process!
Here it is.

What if I were to use a volatile pointer to non-volatile data? I
think this would still prevent the compiler from optimising away the
comparison (because as far as the compiler is concerned the pointer
could change right up to the moment that the comparison is made).
This should also allow me to use memcmp to make comparisons because
instead of passing a pointer to volatile data I would be passing a
volatile pointer to the data.

 

[Eric Sosman]

You haven't changed anything as far as the constraint violation on the
call to memcmp is concerned. Both

volatile char x[10];
memcmp(x, ...);
memcmp(&x[3], ...);

and

volatile char *x;
memcmp(x, ...);
memcmp(&x[3], ...);

all pass a 'char pointer to volatile' where 'void pointer to const' is
expected.

OK, thanks for the response.

As has been my aim in this thread, I want to write code that will
verify that a write to CMOS has succeeded, and ensure that the
compiler does not optimise away the verification.

What follows is a solution, based on the previous comments in this
thread. As far as I can tell it looks OK. Obviously, if anyone on
c.l.c. thinks differently, I'd be grateful for the comments.

typedef struct {
...
somestruct s;
...
} CMOSTYPE;

CMOSTYPE cmos;

...

/* Create a volatile pointer to the cmos structure */
CMOSTYPE * volatile pcmos = &cmos;
somestruct s;

/* Assign some values to the members of s */

...

/* Copy s into the cmos structure */
memcpy(&pcmos->s, &s, sizeof(pcmos->s));

/* Check that the CMOS write succeeded */
if (memcmp(&pcmos->s, &s, sizeof(pcmos->s)) != 0)
/* Write failed */

You have been told, several times and by several people,
that it is an error to pass a pointer to volatile data to a
function that isn't aware of that volatility. What have you
done with this carefully-explained information? You ignore
it, that's what, and you pass pointers to volatile data to
memcpy() and memcmp(), functions unaware of the volatility.

I give up.

 

[Martin]

You have been told, several times and by several people,
that it is an error to pass a pointer to volatile data to a
function that isn't aware of that volatility. What have you
done with this carefully-explained information? You ignore
it, that's what, and you pass pointers to volatile data to
memcpy() and memcmp(), functions unaware of the volatility.

I give up.

Don't give up, the advice is appreciated. If I *appear* to be ignoring
the advice it's because there is something I am not understanding.

As I explained in my last post, I am passing a volatile pointer to non-
volatile data, I am not passing a pointer to volatile data; therefore,
the data dereferenced by both memcmp() and memcpy() are not volatile.
I think the distinction is important and I explained my rationale for
doing so in the message. So why is it still wrong?

 

[CBFalconer]

Ben Pfaff wrote:
.... snip ...

This statement is given in the context of qualifiers on types,
not qualifiers on pointers. I think that this is intended to
mean that the compiler is not obligated to store const objects
in read-only memory, and that it is not obligated to put
volatile objects in a special section of memory either.

The following was my attempt to test incrementing of a void*. I
think I can imagine situations where this ability would be useful
to bypass pointer incrementation. [...]

I am struggling to understand how this is anything but a non
sequitur. Can you explain?

I have to admit I am confused myself. :=) As I vaguely recall, I
had some ideas about exploring some crevice in your statement, but
after the fact I can't see it. However, my sample (that you
snipped) stands, and should maybe be considered.

 

[James Kuyper]

Don't give up, the advice is appreciated. If I *appear* to be ignoring
the advice it's because there is something I am not understanding.

As I explained in my last post, I am passing a volatile pointer to non-
volatile data, I am not passing a pointer to volatile data; therefore,
the data dereferenced by both memcmp() and memcpy() are not volatile.
I think the distinction is important and I explained my rationale for
doing so in the message. So why is it still wrong?

If that's what you think you're doing, your declaration is incorrect. Change


volatile char *parr = arr;

into

char * volatile parr = arr;

 

[Martin]

If that's what you think you're doing, your declaration is incorrect. Change

volatile char *parr = arr;

into

char * volatile parr = arr;

But that's exactly what I've done, albeit in the form:

. . .
CMOSTYPE * volatile pcmos = &cmos;
. . .

 

[Ben Pfaff]

The following was my attempt to test incrementing of a void*. I
think I can imagine situations where this ability would be useful
to bypass pointer incrementation. BTW, cc is shorthand for:
gcc -W -Wall -ansi -pedantic
and accesses gcc 3.2.1.

[1] c:\c\junk>cat junk.c
#include <stdio.h>

int main(void) {
void *p, *pb;
char by;

p = &by;

pb = p++;
if (pb == p) puts("++ uses sizeof void* == 0");
else puts("No luck here");
return 0;
}

I'm not sure what you're hoping to demonstrate here. The size of
an incomplete type cannot be taken in ISO C, and GCC with
-pedantic amply demonstrates that by giving the required
diagnostic. That GCC then allows the program to compile to an
executable is not really a kindness.

The situation when GCC is run without -pedantic is a little
different, but GCC is not an ISO C compiler in that case.

 

[James Kuyper]

But that's exactly what I've done, albeit in the form:

. . .
CMOSTYPE * volatile pcmos = &cmos;

I searched back to the message you sent with the header:

Date: Tue, 20 Nov 2007 22:41:27 -0000

In that message, you declared parr exactly as indicated above. For some
reason I skipped over the one with the header

Date: Wed, 21 Nov 2007 03:27:10 -0800 (PST)

With that declaration, the problem is indeed fixed. pcmos->s is not a
volatile object, and therefore doesn't violate 6.7.3p5. For precisely
the same reason, it doesn't achieve your goal of preventing a compiler
from optimizing away the memcmp() call.

 

[Martin]

With that declaration, the problem is indeed fixed. pcmos->s is not a
volatile object, and therefore doesn't violate 6.7.3p5. For precisely
the same reason, it doesn't achieve your goal of preventing a compiler
from optimizing away the memcmp() call.

Thanks for your response.

Surely the fact that pcmos is now a volatile pointer prevents the
compiler from optimising away the memcmp because the compiler won't
know at compile-time that pcmos won't change between the memcpy and
the memcmp?

 

[Ben Bacarisse]

OK, thanks for the response.

What follows is a solution, based on the previous comments in this
thread. As far as I can tell it looks OK. Obviously, if anyone on
c.l.c. thinks differently, I'd be grateful for the comments.

typedef struct {
...
somestruct s;
...
} CMOSTYPE;

CMOSTYPE cmos;

...

/* Create a volatile pointer to the cmos structure */
CMOSTYPE * volatile pcmos = &cmos;
somestruct s;

/* Assign some values to the members of s */

...

/* Copy s into the cmos structure */
memcpy(&pcmos->s, &s, sizeof(pcmos->s));

/* Check that the CMOS write succeeded */
if (memcmp(&pcmos->s, &s, sizeof(pcmos->s)) != 0)
/* Write failed */

You have just switched the volatile property to something else. The
pointer is now volatile (probably not true) so that call is now OK but
can be optimised away is the compiler sees fit. There is no option
but to mark the thing that is volatile as volatile. If you can't do
that for some reason then I can't see a solution.

 

[Martin]

You have just switched the volatile property to something else. The
pointer is now volatile (probably not true) so that call is now OK but
can be optimised away is the compiler sees fit. There is no option
but to mark the thing that is volatile as volatile. If you can't do
that for some reason then I can't see a solution.

Thanks for the response Ben.

Your solution (of making the CMOS structure volatile) will only work
(as others here have pointed out) in the case where the item I am
writing and verifying in the cmos structure can be compared with a
simple ==. Where I write a structure such as a date/time (using
memcpy) I need to use memcmp to check that it wrote properly and I
cannot do so if the target is volatile (or if I am pretending it is).

 

[santosh]

Thanks for the response Ben.

Your solution (of making the CMOS structure volatile) will only work
(as others here have pointed out) in the case where the item I am
writing and verifying in the cmos structure can be compared with a
simple ==. Where I write a structure such as a date/time (using
memcpy) I need to use memcmp to check that it wrote properly and I
cannot do so if the target is volatile (or if I am pretending it is).

Compare member by member individually.

 

[Chris Torek]

(... making the CMOS structure volatile) will only work
(as others here have pointed out) in the case where the item I am
writing and verifying in the cmos structure can be compared with a
simple ==. Where I write a structure such as a date/time (using
memcpy) I need to use memcmp to check that it wrote properly ...

Using memcmp() is rather likely to fail anyway, depending on
the target architecture.

Suppose memcmp() is a typical assembly-language "optimized" version,
that checks the alignment of its incoming pointers and the sizes
of the regions to compare. If the size is large enough (at least
4 or 8 bytes) and the addresses are properly aligned, it does, in
effect, the assembly equivalent of the following C code:

unsigned char *left, *right; /* then initialized from parameters */
size_t len; /* initialized from parameter */

if (properly_aligned) {
while (len >= sizeof(int) && *(int *)left == *(int *)right)
left += sizeof(int), right += sizeof(int), len -= sizeof(int);
}
while (len) {
if (*left != *right)
return *left < *right ? -1 : 1;
left++, right++, len--;
}
return 0;

Now, on a lot of hardware, *(unsigned char *)addr does a 1-byte
read on the bus, which is what is required to access an EEPROM
device; but *(int *)addr does a 4-byte read on the bus. If a 4-byte
read reaches the EEPROM ("CMOS") hardware, an incorrect value is
read from the device, or an exception ("bus error" or similar) is
delivered to the CPU.

Not all hardware works like this[%], and sometimes memcmp() really
can be used directly against "hardware memory", but exceptions
are common enough (in my experience anyway) to assume the worst,
and avoid memcmp() here.

[% Two particularly bizarre examples occur with 8-bit hardware
attached to "32-bit-only" CPUs. Here, "load byte" does a 32-bit
bus access, or there is no separate "load byte" instruction, so
the hardware designer takes one of two approaches. Either every
8-bit byte is placed on a 32-bit boundary, so that *(int *)addr is
0x000000NN or 0xNN000000 depending on endian-ness -- and sometimes
the 00s are FFs instead, or are duplicates of the NN byte; or,
instead, the 32-bit access is put on hold while the adapter does
four 8-bit accesses so as to "construct" the full 32 bit value,
which is then placed on the data bus. In this second case, it
really *can* be good idea to use 32-bit operations to access the
device -- typically to copy the entire ROM contents to RAM for
faster reference.]

 

[Willem]

Martin wrote:
) Thanks for the response Ben.
)
) Your solution (of making the CMOS structure volatile) will only work
) (as others here have pointed out) in the case where the item I am
) writing and verifying in the cmos structure can be compared with a
) simple ==. Where I write a structure such as a date/time (using
) memcpy) I need to use memcmp to check that it wrote properly and I
) cannot do so if the target is volatile (or if I am pretending it is).

Well then the solution is easy:
Write your own memcmp using volatile pointers.


SaSW, Willem
--
Disclaimer: I am in no way responsible for any of the statements
made in the above text. For all I know I might be
drugged or something..
No I'm not paranoid. You all think I'm paranoid, don't you !
#EOT

 

[James Kuyper]

Thanks for your response.

Surely the fact that pcmos is now a volatile pointer prevents the
compiler from optimising away the memcmp because the compiler won't
know at compile-time that pcmos won't change between the memcpy and
the memcmp?

Not really. if pcmos is subject to random changes between the calls to
memcpy() and memcmp(), it might no longer point at the same object it
was originally set to; it might even contain a null pointer or a trap
representation. Therefore, the call to memcmp() will have undefined
behavior. That includes the possibility of optimizing away the call to
memcmp().

I really don't understand why you want to prevent this optimization. It
is permitted only when it will make no difference, in which case there's
no point to calling memcmp() (and also no point to preventing the
optimization), or when the behavior is undefined - and there's no point
to deliberately writing code unless there's something that gives the
code defined behavior.

 

[CBFalconer]

The following was my attempt to test incrementing of a void*. I
think I can imagine situations where this ability would be useful
to bypass pointer incrementation. BTW, cc is shorthand for:
gcc -W -Wall -ansi -pedantic
and accesses gcc 3.2.1.

[1] c:\c\junk>cat junk.c
#include <stdio.h>

int main(void) {
void *p, *pb;
char by;

p = &by;

pb = p++;
if (pb == p) puts("++ uses sizeof void* == 0");
else puts("No luck here");
return 0;
}

I'm not sure what you're hoping to demonstrate here. The size of
an incomplete type cannot be taken in ISO C, and GCC with
-pedantic amply demonstrates that by giving the required
diagnostic. That GCC then allows the program to compile to an
executable is not really a kindness.

The situation when GCC is run without -pedantic is a little
different, but GCC is not an ISO C compiler in that case.

The situation I was trying to control was ridiculous anyhow.
Basically I wanted to see if the increment of a void* became a NOP,
so control of the type would enable it. Possibly it would make
more sense to have a void* point to something of size zero, but I
don't know offhand what other effects that would have. At any rate
it doesn't deserve any further consideration.

I must be getting older. I am proposing more foolish things these
days.

 

[Martin]

Not really. if pcmos is subject to random changes between the calls to
memcpy() and memcmp(), it might no longer point at the same object it
was originally set to; it might even contain a null pointer or a trap
representation. Therefore, the call to memcmp() will have undefined
behavior. That includes the possibility of optimizing away the call to
memcmp().

I really don't understand why you want to prevent this optimization. It
is permitted only when it will make no difference, in which case there's
no point to calling memcmp() (and also no point to preventing the
optimization), or when the behavior is undefined - and there's no point
to deliberately writing code unless there's something that gives the
code defined behavior.

James,

I stated at the start of this thread why I wanted to avoid optimising
out the memcmp and have restated it since. Please accept that I need
to do so.

pcmos is not subject to random change between the calls. We are making
it volatile so that we can *pretend* it is subject to random change in
order to prevent the memcmp from being optimised out (although based
on other contributors' posts to this thread that turns out to perhaps
not be a good idea).

 

[Martin]

[...]
Not all hardware works like this[%], and sometimes memcmp() really
can be used directly against "hardware memory", but exceptions
are common enough (in my experience anyway) to assume the worst,
and avoid memcmp() here.

Thanks for your detailed response, Chris.

I have checked the implementation of memcmp (and strcmp) in the
library we're using and both are done byte by byte so while the use of
memcmp on CMOS may not be totally portable it should at least be safe
in our current architecture.