memcpy and volatile

M

Mark

Hi List,

I want to write a function to copy some data out of a hardware buffer.
The hardware can change the contents of this buffer without it being
written to by my function. I want to use memcpy to unload the data.

Do I need to specify the source data as volatile in this case?

What is the correct syntax for specifying that the data pointed to by
the source pointer is volatile and not the pointer itself?

How do I ensure that all N source bytes are regarded as volatile and
not just the first byte?

Thanks for your help.

Mark
 
A

Arthur J. O'Dwyer

I want to write a function to copy some data out of a hardware buffer.
The hardware can change the contents of this buffer without it being
written to by my function. I want to use memcpy to unload the data.

Why? If you wrote memcpy yourself, then just cut-and-paste that code
into a new function, like

size_t volatile_memcpy(volatile void *d, volatile void *s, size_t n)
{
[your code here]
}

and everything will work. And if you didn't write memcpy yourself, then
how do you know that it will do what you want? For example, memcpy might
write to the destination buffer multiple times; it might read or write
in N-byte chunks instead of byte-sized chunks; it might read and write
the bytes in reverse order (starting with s[n-1] and going down to s[0]).
There are all sorts of things memcpy is allowed to do --- and /does/ do,
on many platforms --- that would play havoc with most data worth
classifying as volatile (memory-mapped registers, perhaps video memory,
and so on, handwavey handwavey).
Do I need to specify the source data as volatile in this case?

If you end up just using memcpy, it won't matter one whit. memcpy's
parameters don't have the 'volatile' qualifier; you'll have to cast it
away from both arguments before calling memcpy.

volatile char *myd = ...;
char mys[100] = { ... };
memcpy((void*)myd, mys, 100);
What is the correct syntax for specifying that the data pointed to by
the source pointer is volatile and not the pointer itself?

volatile T *pt;
T volatile *pt; both mean that *pt is volatile

T * volatile pt; means that pt is volatile (but *pt isn't)
How do I ensure that all N source bytes are regarded as volatile and
not just the first byte?

The 'volatile' qualifier applies to /all/ accesses through pt. That
is, if you write "volatile T *pt;" then *pt, pt[0], pt[1], *(pt+42) are
all volatile accesses.

-Arthur
 
E

Eric Sosman

Mark said:
Hi List,

I want to write a function to copy some data out of a hardware buffer.
The hardware can change the contents of this buffer without it being
written to by my function. I want to use memcpy to unload the data.
[...]

You're out of luck: memcpy() does not accept pointers to
volatile objects.

And no, that's not as arbitrary a restriction as it might
appear. Consider: memcpy() copies its data as if by copying
the individual bytes, one by one. What happens if memcpy() is
halfway through the operation when your hardware decides to burp
up a new batch of data? You get (at best) a mixture of old and
new bytes in the destination area; maybe you get something even
worse ("Memory Interface Syndrome: Tardy AcKnowledge Event")
and a chance to power-cycle the system.

memcpy() is the wrong hammer for driving this screw.
 
M

Mark

Thanks for all your help guys, however I would like to ask what happens
in a slightly different situation to the one that I described...

Lets say I have a hardware buffer at address BUFFER_BASE. The contents
of this buffer will remain stable, unless I access the hardware to
change the buffer page, whereby the contents of the buffer will change.

if I write...

char* ptr = BUFFER_BASE;
char a;
for(int i=0; i<10; i++)
{
SetHardwarePage(i);
a = *ptr;
printf("Page %d contains character %c\n", i, a);
}

I do not expect this to work unless ptr is defined as volatile char*
ptr;

However, what happens if I replace a=*ptr; with a function call, e.g.
memcpy?

char* ptr = BUFFER_BASE;
char a[N];
for(int i=0; i<10; i++)
{
SetHardwarePage(i);
memcpy(ptr, a, N);
printf("Page %d contains character %c at position 0\n", i, a[0]);
}

In this case, within the memcpy call, the data is not volatile as
SetHardwarePage is not called. However the data is volatile between
memcpy calls. Will it work? What happens if the compiler decides to
inline memcpy (which is unlikely, but will help me understand the
point)?

Thanks again,

Mark
 
A

Arthur J. O'Dwyer

Lets say I have a hardware buffer at address BUFFER_BASE. The contents
of this buffer will remain stable, unless I access the hardware to
change the buffer page, whereby the contents of the buffer will change.

if I write...

char* ptr = BUFFER_BASE;
char a;
for(int i=0; i<10; i++)
{
SetHardwarePage(i);
a = *ptr;
printf("Page %d contains character %c\n", i, a);
}

I do not expect this to work unless ptr is defined as volatile char*
ptr;

Sounds reasonable.
However, what happens if I replace a=*ptr; with a function call, e.g.
memcpy?

char* ptr = BUFFER_BASE;
char a[N];
for(int i=0; i<10; i++)
{
SetHardwarePage(i);
memcpy(ptr, a, N);
printf("Page %d contains character %c at position 0\n", i, a[0]);
}

In this case, within the memcpy call, the data is not volatile as
SetHardwarePage is not called. However the data is volatile between
memcpy calls. Will it work? What happens if the compiler decides to
inline memcpy (which is unlikely, but will help me understand the
point)?

Since you're casting away 'volatile' to do the memcpy, the effects
are undefined. On the one hand, I would expect a conscientious
compiler-implementor to "get it right" and make it do what you expect;
but since the language standard itself makes no such guarantee, I
would also expect a conscientious programmer to "get it right" by
avoiding the undefined behavior altogether.

N869, 6.7.3#5:
If an
attempt is made to refer to an object defined with a
volatile-qualified type through use of an lvalue with non-
volatile-qualified type, the behavior is undefined.

HTH,
-Arthur
 
M

Mark

Thanks for all your help. I will write my own suitable copy function.


Lets say I have a hardware buffer at address BUFFER_BASE. The contents
of this buffer will remain stable, unless I access the hardware to
change the buffer page, whereby the contents of the buffer will change.

if I write...

char* ptr = BUFFER_BASE;
char a;
for(int i=0; i<10; i++)
{
SetHardwarePage(i);
a = *ptr;
printf("Page %d contains character %c\n", i, a);
}

I do not expect this to work unless ptr is defined as volatile char*
ptr;

Sounds reasonable.
However, what happens if I replace a=*ptr; with a function call, e.g.
memcpy?

char* ptr = BUFFER_BASE;
char a[N];
for(int i=0; i<10; i++)
{
SetHardwarePage(i);
memcpy(ptr, a, N);
printf("Page %d contains character %c at position 0\n", i, a[0]);
}

In this case, within the memcpy call, the data is not volatile as
SetHardwarePage is not called. However the data is volatile between
memcpy calls. Will it work? What happens if the compiler decides to
inline memcpy (which is unlikely, but will help me understand the
point)?

Since you're casting away 'volatile' to do the memcpy, the effects
are undefined. On the one hand, I would expect a conscientious
compiler-implementor to "get it right" and make it do what you expect;
but since the language standard itself makes no such guarantee, I
would also expect a conscientious programmer to "get it right" by
avoiding the undefined behavior altogether.

N869, 6.7.3#5:
If an
attempt is made to refer to an object defined with a
volatile-qualified type through use of an lvalue with non-
volatile-qualified type, the behavior is undefined.

HTH,
-Arthur
 
K

Keith Thompson

David T. Ashley said:
Not always. On machines with alignment restrictions (or equivalently,
better instructions available if best alignment is guaranteed), memcpy can
take a few different forms. The most common one is:

if (both pointers are aligned)
Do best instruction copy as many times as possible;
Finish any remainder using less-efficient byte copies;
else
Use less-efficient byte copy;
[...]

Yes, but that's still *as if* by copying the individual bytes.

(Though volatile qualification might affect this; I see the word
"volatile" in the subject, but I can't see the previous article.)
 
D

David T. Ashley

Eric Sosman said:
And no, that's not as arbitrary a restriction as it might
appear. Consider: memcpy() copies its data as if by copying
the individual bytes, one by one.

Not always. On machines with alignment restrictions (or equivalently,
better instructions available if best alignment is guaranteed), memcpy can
take a few different forms. The most common one is:

if (both pointers are aligned)
Do best instruction copy as many times as possible;
Finish any remainder using less-efficient byte copies;
else
Use less-efficient byte copy;

And, as I'm sure countless subsequent posters will point out, there are
numerous variations on the above depending on modulo arithmetic and machine
specifics. [However, the form above is efficient when most pointers passed
in came from malloc() and friends ... but would get less efficient in
certain circumstances.]

And, memset() and memclr() also take on different forms when machine
alignment is an issue.

Dave.
 
A

Arthur J. O'Dwyer

David T. Ashley said:
Not always. On machines with alignment restrictions (or equivalently,
better instructions available if best alignment is guaranteed), memcpy can
take a few different forms. The most common one is:

if (both pointers are aligned)
Do best instruction copy as many times as possible;
Finish any remainder using less-efficient byte copies;
else
Use less-efficient byte copy;
[...]

Yes, but that's still *as if* by copying the individual bytes.

(Yup.)
(Though volatile qualification might affect this; I see the word
"volatile" in the subject, but I can't see the previous article.)

The OP was thinking of using 'memcpy' to copy a block of
'volatile'-qualified memory, but has since been dissuaded. And
'volatile' is irrelevant to the semantics of 'memcpy' anyway;
as Eric wrote in the previous article,

-Arthur
 
B

Ben Pfaff

David T. Ashley said:
(Function unrecognized) or (what are different forms)?

Well, there's no such function in the standard library, so what
are you talking about? It's not even in SUSv3 or (as far as I
know) Windows.
 
D

David T. Ashley

Keith Thompson said:
David T. Ashley said:
Not always. On machines with alignment restrictions (or equivalently,
better instructions available if best alignment is guaranteed), memcpy
can
take a few different forms. The most common one is:

if (both pointers are aligned)
Do best instruction copy as many times as possible;
Finish any remainder using less-efficient byte copies;
else
Use less-efficient byte copy;
[...]

Yes, but that's still *as if* by copying the individual bytes.

(Though volatile qualification might affect this; I see the word
"volatile" in the subject, but I can't see the previous article.)

Most of the posts show a lack of experience with actual hardware. In
practical applications using a buffer, usually the buffer is controlled by
other control register locations -- A-B is a typical arrangement where the
asynchronous process and the one being compiled have some other way to
coordinate which process is doing what to which memory locations at which
time (many modern communication peripherals are designed in this way). A
true situation where memcpy() can't be safely used for moving buffered data
is relatively rare.
Yes, but that's still *as if* by copying the individual bytes.

Alignment and choice of machine instructions are relevant when dealing with
"memory" locations interfaced in unusual ways. In some interface
arrangements, using the wrong instructions will "double-bounce" the location
and have unintended effects. Using "*as if*" to apply to locations declared
volatile is nonsense. That contradicts the notion of "volatile". There is
no "*as if*" with "volatile".

The last thing I need is individuals without enough experience with
practical devices opining on whether the post I made adds value.
 
E

Eric Sosman

David said:
Keith Thompson said:
David T. Ashley said:
And no, that's not as arbitrary a restriction as it might
appear. Consider: memcpy() copies its data as if by copying
the individual bytes, one by one.
Not always. On machines with alignment restrictions (or equivalently,
better instructions available if best alignment is guaranteed), memcpy
can
take a few different forms. The most common one is:

if (both pointers are aligned)
Do best instruction copy as many times as possible;
Finish any remainder using less-efficient byte copies;
else
Use less-efficient byte copy;
[...]

Yes, but that's still *as if* by copying the individual bytes.

(Though volatile qualification might affect this; I see the word
"volatile" in the subject, but I can't see the previous article.)

Most of the posts show a lack of experience with actual hardware. In
practical applications using a buffer, usually the buffer is controlled by
other control register locations -- A-B is a typical arrangement where the
asynchronous process and the one being compiled have some other way to
coordinate which process is doing what to which memory locations at which
time (many modern communication peripherals are designed in this way). A
true situation where memcpy() can't be safely used for moving buffered data
is relatively rare.
Yes, but that's still *as if* by copying the individual bytes.

Alignment and choice of machine instructions are relevant when dealing with
"memory" locations interfaced in unusual ways. In some interface
arrangements, using the wrong instructions will "double-bounce" the location
and have unintended effects. Using "*as if*" to apply to locations declared
volatile is nonsense. That contradicts the notion of "volatile". There is
no "*as if*" with "volatile".

The last thing I need is individuals without enough experience with
practical devices opining on whether the post I made adds value.

David, I chose my language with some care: I wrote "as if
by copying the individual bytes," and that is precisely what I
meant. You responded that this was "not always" the case, and
others have pointed out your error. (Which your rantings about
volatile merely perpetuate.)

Your mention of (yet more) hardware where the choice of
instruction makes a difference does not run counter to my "as
if" statement, but actually reinforces the unsuitability of
memcpy() for the use at hand. That is, the Standard's only
requirement on memcpy() is that it behave "as if" it copied the
individual bytes; implementations are free to achieve the effect
in any way they please. In particular, they are free to achieve
it in ways that do not play well with pseudo-memory buffers, or
with volatile data in general.

The situation can be even worse: Not only might memcpy() be
unsuitable, but even a plain `x = *p' may be unsuitable. "What
constitutes an access to an object that has volatile-qualified
type is implementation-defined." The practical upshot is that
the mere fact that you use an `int*' to access the buffer is no
guarantee that the compiler will generate the "obvious" int-ish
instruction to fetch from it. It might, for example, helpfully
unroll a loop and then combine two adjacent int accesses into a
single long or long long access, which would be unfortunate if
the pseudo-memory locations were only sensitive to int accesses.

Yes, I have used hardware with such characteristics. And the
last thing I need is individuals who know nothing about my
experience telling me it's inadequate.
 
D

David T. Ashley

Eric Sosman said:
David, I chose my language with some care: I wrote "as if
by copying the individual bytes," and that is precisely what I
meant. You responded that this was "not always" the case, and
others have pointed out your error. (Which your rantings about
volatile merely perpetuate.)

Hi Eric,

There is no "as if" with "volatile". This is the very definition of
volatile. It is quite possible to arrange the addressing and glue logic so
that the hardware (often a control register) will do different things
depending on which instructions are used to address the location. The most
common problem is "double bounce". Copying by character and copying using
instructions that operate with more general alignment may very well have
different results.

"As if" only applies to addressible locations which are conceptually not
volatile. With such locations, it really does not matter how the copy is
done.
You responded that this was "not always" the case, and
others have pointed out your error.

I don't believe I've made any errors. Could you be more specific? What
error did I make?

Thanks and best regards, Dave.
 
D

David T. Ashley

Ben Pfaff said:
Well, there's no such function in the standard library, so what
are you talking about? It's not even in SUSv3 or (as far as I
know) Windows.

You are right.

I've been rolling my own software for so long that I've forgotten what is in
the library and what is not. I mostly do embedded systems with under 32K of
ROM. With small embedded systems, clearing small blocks of memory occurs so
often it is helpful to have an assembly-language function (callable from
'C') to do the work without the overhead of the "0" parameter, i.e.

extern void memclr(void *ptr, unsigned char len); /* Only good to 255, but
for a small system, this is enough. */

The overhead of the "0" parameter hurts in two ways. First, it is setup
overhead on every function call (ROM waste on every call). Second, within
the memclr() function, a "store" instruction would be used (slower) when the
processor often has a "clear" instruction that does the same thing more
economically (suboptimal performance).

I am not the first person to create memclr(), i.e.

http://www-masu.ist.osaka-u.ac.jp/~kakugawa/VFlib/src-browse/VFlib3-3.6.12/HTML/D/memclr.html

however, in a small system, I wouldn't do it the way suggested by the URL
above.

My mistake. It is not in the library.

Dave.
 
R

Random832

2006-12-13 said:
Hi Eric,

There is no "as if" with "volatile".

There is always "as if". It's just that volatile means there are a lot
more things you have to take into account that might be able to "call
you out" on your optimizations.

For example, if you do know (I mean absolutely KNOW for a fact - as in,
the architecture provides no such mechanism) that nothing's going to be
able to find out that it was copied a word at a time, there's no reason
you can't do that.

But... regardless, memcpy's arguments aren't qualified as volatile.
 
E

Eric Sosman

David said:
Hi Eric,

There is no "as if" with "volatile". This is the very definition of
volatile. It is quite possible to arrange the addressing and glue logic so
that the hardware (often a control register) will do different things
depending on which instructions are used to address the location. The most
common problem is "double bounce". Copying by character and copying using
instructions that operate with more general alignment may very well have
different results.

"As if" only applies to addressible locations which are conceptually not
volatile. With such locations, it really does not matter how the copy is
done.

You've asserted this at least twice, so you presumably have
some reason for believing it. If you'd reveal the reason, maybe
someone could disabuse you of it -- or, perhaps, suddenly grasp
what you are actually trying to say. On the face of it, what you
claim seems to be purest nonsense: The definition of volatile says
nothing about suspending the "as if rule." It talks about requiring
the actual machine to mimic the abstract machine in some aspects,
but that's all.
I don't believe I've made any errors. Could you be more specific? What
error did I make?

Well, there's the Standard's description of how memcpy
operates (7.21.2.1/2):

The memcpy function copies n characters from the object
pointed to by s2 into the object pointed to by s1. [...]

You will note that the operation is described in terms of copying
characters, not larger units. The Standard does not say that
memcpy copies an n-character array, but that it copies "n characters."

This cannot be dismissed as a mere quirk of expression. Consider
7.19.8.1/2, where the Standard speaks of operations on bunches of
bytes considered as large units. The Standard is perfectly able to
describe operations on multi-character "units" when it wants to --
but it did not do so when defining memcpy.

Given that memcpy is defined in terms of copying characters,
it is erroneous to claim that it does not operate "as if" it
copied characters.
 
K

Keith Thompson

David T. Ashley said:
Keith Thompson said:
David T. Ashley said:
And no, that's not as arbitrary a restriction as it might
appear. Consider: memcpy() copies its data as if by copying
the individual bytes, one by one.

Not always. On machines with alignment restrictions (or equivalently,
better instructions available if best alignment is guaranteed), memcpy
can
take a few different forms. The most common one is:

if (both pointers are aligned)
Do best instruction copy as many times as possible;
Finish any remainder using less-efficient byte copies;
else
Use less-efficient byte copy;
[...]

Yes, but that's still *as if* by copying the individual bytes.

(Though volatile qualification might affect this; I see the word
"volatile" in the subject, but I can't see the previous article.)

Most of the posts show a lack of experience with actual hardware.

In my case, you're right. I have little experience with actual
hardware, and I don't think I've ever actually used "volatile".
In
practical applications using a buffer, usually the buffer is controlled by
other control register locations -- A-B is a typical arrangement where the
asynchronous process and the one being compiled have some other way to
coordinate which process is doing what to which memory locations at which
time (many modern communication peripherals are designed in this way). A
true situation where memcpy() can't be safely used for moving buffered data
is relatively rare.


Alignment and choice of machine instructions are relevant when dealing with
"memory" locations interfaced in unusual ways. In some interface
arrangements, using the wrong instructions will "double-bounce" the location
and have unintended effects. Using "*as if*" to apply to locations declared
volatile is nonsense. That contradicts the notion of "volatile". There is
no "*as if*" with "volatile".

Here's what the standard says about memcpy() (C99 7.21.2.1p2):

The memcpy function copies n characters from the object pointed to
by s2 into the object pointed to by s1. If copying takes place
between objects that overlap, the behavior is undefined.

It's certainly the case that a conforming implementation of memcpy()
*may* work by copying bytes one at a time. I believe it's the case
that some (most?) implementations of memcpy() copy memory in bigger
chunks if the source and target buffers are appropriately aligned. I
don't know how this might interact with "volatile".
The last thing I need is individuals without enough experience with
practical devices opining on whether the post I made adds value.

I expressed no such opinion. I did, however, explicitly acknowledge
that "volatile" might affect the behavior of memcpy(). (I also
mentioned that I wasn't able to see the entire thread, so I could
easily have missed something.)
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Similar Threads

The Semantics of 'volatile' 73
silly with memcpy() 4
volatile and multiple threads 10
Is memcpy with len=0 a NOP? 16
Volatile and code reordering 1
how to use volatile key word? 24
memcpy 6
volatile 8

Members online

No members online now.

Forum statistics

Threads
473,770
Messages
2,569,583
Members
45,074
Latest member
StanleyFra

Latest Threads

Top