Pointer Converts to Direct Memory

[Bryan Parkoff]

We discussed about the union keyword on the previous threads 2-3 days
earlier. I want to add. Two byte data inside struct and one word data are
inside union. You modify word data. Then two byte data are modified
automatically because two byte data and one word data are sharing the same
memory address. Let's move on and forget union because we claim that union
keyword is not the language of C/C++ Compiler.
Here is a pointer variable code below.

enum
{
Low = 0,
High = 1
};

U_BYTE B[4] = { 0xFF, 0x20, 0x02, 0x22 };
PU_WORD W = (PU_WORD)&B;
PU_DWORD DW = (PU_DWORD)&B;

B[Low] += 0x0A;
B[High] += 0x01;
W[Low] += 0x0100;
*DW = 0x41424344; // *DW and DW[0] are same.
DW[0] = 0x51525354;

I decide to define four elements in an array of B variable. I define W
as the pointer to B variable. This allows to modify individual two byte
data. Then W variable is modified automatically from B variable. W
variable is modified causing two byte data to be modified automatically. DW
variable does the same to modify 4 byte data. Sometimes, I want to modify
individual byte data and word data, but I depend on word data often. I load
two byte data from big array in memory and they are stored in B[L] and B[H].
I use addition to put two byte data together into word data so I don't need
to use addition on both byte data. It depends according to my decision and
design of my source code.
You may be surprised to see direct variable and pointer variable. You
set debug mode. You go step by step looking at each variable in watch
window. You go into disassemble window like x86 machine and non-x86
machine. You can see MOV instruction on direct variable. Also, you can see
LEA instruction and MOV instruction on pointer variable.
You switch from debug mode to release mode. You look at disassemble
window again. Do you see disappearing LEA instruction? W variable is
converted to direct variable from pointer variable. Do you know why? The
C/C++ Compiler uses optimization and it decides that two variables share the
same memory address so it does not need LEA instruction. It reduces time
and improve performance.
I choose word data instead of dword data, but you claim that word data
is slow than dword data. What happen if word data has 0xFFFF? It is added
by 0x0004. The result shows 0x0001-0003. I do not want data on 31th
through 16th bits. 32th through 16th bits can always be ignored. Word data
can be used. It is pain to use "AND" on dword data if you want to mask 16
bit data or 8 bit data.
Sometimes, C/C++ Compiler ignores "AND" like dword dw &= 0xFFFF and it
uses MOV instruction to store 16 bit data or 8 bit data into memory address
directly.
Please let me know what you think. It can be interesting discussion. I
may do not need to post more so I can continue writing my C++ source code
according to my design so two variables can share the same memory address.

 

[Juha Nieminen]

U_BYTE B[4] = { 0xFF, 0x20, 0x02, 0x22 };
PU_WORD W = (PU_WORD)&B;
PU_DWORD DW = (PU_DWORD)&B;

B[Low] += 0x0A;
B[High] += 0x01;
W[Low] += 0x0100;
*DW = 0x41424344; // *DW and DW[0] are same.
DW[0] = 0x51525354;

This is not portable code. The contents of B, when read and modified
using W and DW, will depend on the endianess of the system. In a
low-endian system (such as an Intel PC) the result will be different
than with a high-endian system (such as an UltraSparc).

 

[James Kanze]

We discussed about the union keyword on the previous threads
2-3 days earlier. I want to add. Two byte data inside struct
and one word data are inside union. You modify word data.
Then two byte data are modified automatically because two byte
data and one word data are sharing the same memory address.
Let's move on and forget union because we claim that union
keyword is not the language of C/C++ Compiler.

No one ever said that union wasn't a keyword in C++ (or in C).

Here is a pointer variable code below.

enum
{
Low = 0,
High = 1
};

U_BYTE B[4] = { 0xFF, 0x20, 0x02, 0x22 };
PU_WORD W = (PU_WORD)&B;
PU_DWORD DW = (PU_DWORD)&B;

B[Low] += 0x0A;
B[High] += 0x01;
W[Low] += 0x0100;
*DW = 0x41424344; // *DW and DW[0] are same.
DW[0] = 0x51525354;

U_BYTE, PU_WORD and PU_DWORD, on the other hand, definitely
aren't keywords in C++. I'm just guessing, but I suppose that
they're typedefs:

typedef unsigned char U_BYTE ;
typedef unsigned int* PU_WORD ;
typedef unsigned long* PU_DWORD ;

or something like that. If that's true, your code above may
core dump, at least on my machine, because you're not
guaranteeing necessary alignment.

Note too that: 1) you probably shouldn't use all caps for
this---all caps is best reserved for macros, and 2) the exact
size of each type might vary: traditionally, a word is four
bytes, but I can imagine that on machines where there aren't any
four byte entities (e.g. where sizeof( int ) is something like
6), it means something else. Before going any further, you
should specify your portability.

I decide to define four elements in an array of B variable. I
define W as the pointer to B variable. This allows to modify
individual two byte data. Then W variable is modified
automatically from B variable.

You're still being very confusing. First, W is a pointer
variable, whose value is independent of whatever B contains.
Second, you're not guaranteed to be able to do anything with the
pointer---B may not be sufficiently aligned to be accessed as a
word, for example, or a word may be more than four bytes.

W variable is modified causing two byte data to be modified
automatically.

Four or eight bytes on my machine. Pointers are four or eight
bytes, depending on the mode I've compiled in.

And *W will probably core dump from time to time, because B
won't be correctly aligned. (But I'm just guessing as to how
PU_WORD is really defined. If it's a pointer to a character
type, char, unsigned char or signed char, it should be OK. But
the use of the word "WORD" certainly suggests four bytes.)

DW variable does the same to modify 4 byte data.

As for W. DW is a pointer. And we don't know what *DW is,
although the use of DWORD strongly suggests eight bytes (in
which case, not only is B not necessarily correctly aligned, but
it is also not large enough).

Sometimes, I want to modify individual byte data and word
data, but I depend on word data often. I load two byte data
from big array in memory and they are stored in B[L] and B[H].
I use addition to put two byte data together into word data so
I don't need to use addition on both byte data. It depends
according to my decision and design of my source code.

All this to do what? You're being very mysterious with regards
to the semantics you're trying to achieve.

Please let me know what you think.

I think we need to know what you are trying to do. For the
moment, I can't make any sense of it.

If your point is just that given something like:

int i ;
... i & 0xFF ...

the compiler loads a byte, rather than loading a word and then
anding 0xFF, that's certainly nothing new. It's a very common
idiom, and compilers have recognized it for ages. (If memory
serves me correctly, this optimization was already present in
Johnson's pcc, back before 1980.)

 

[Juha Nieminen]

And *W will probably core dump from time to time, because B
won't be correctly aligned.

Is there any compiler in any architecture which creates char arrays at
non-aligned memory locations (by double-word or even larger alignment)?

The major problem with his code is that it's not portable between
architectures of different endianess.

 

[REH]

Is there any compiler in any architecture which creates char arrays at
non-aligned memory locations (by double-word or even larger alignment)?

Yes, I work with several compilers will not align character arrays.
Unfortunately, I also work with people like the OP that are constantly
getting bit by this.

REH

 

[James Kanze]

Is there any compiler in any architecture which creates char arrays at
non-aligned memory locations (by double-word or even larger alignment)?

Are there any that don't? Neither Sun CC nor g++ guarantee that
char arrays are aligned for more than char, and I'd be surprised
that any other compiler does as well.

The major problem with his code is that it's not portable between
architectures of different endianess.

Amongst other things. There are, in fact, several major
problems. The biggest one is that we don't know what the code
is trying to do, so we can't really analyse it for more
problems. The fact remains that it core dumps on a Sun Sparc,
at least if his types correspond the what I think they do.

 

[Bryan Parkoff]

No one ever said that union wasn't a keyword in C++ (or in C).

Here is a pointer variable code below.
enum
{
Low = 0,
High = 1
};
U_BYTE B[4] = { 0xFF, 0x20, 0x02, 0x22 };
PU_WORD W = (PU_WORD)&B;
PU_DWORD DW = (PU_DWORD)&B;
B[Low] += 0x0A;
B[High] += 0x01;
W[Low] += 0x0100;
*DW = 0x41424344; // *DW and DW[0] are same.
DW[0] = 0x51525354;

U_BYTE, PU_WORD and PU_DWORD, on the other hand, definitely
aren't keywords in C++. I'm just guessing, but I suppose that
they're typedefs:

typedef unsigned char U_BYTE ;
typedef unsigned int* PU_WORD ;
typedef unsigned long* PU_DWORD ;

Yes, it is true. You should always do not say U_BYTE is not keyword of
C/C++ Compiler. You see these words. It is simple that you guess to know
what I mean perfectly.

or something like that. If that's true, your code above may
core dump, at least on my machine, because you're not
guaranteeing necessary alignment.

I am aware that my code may be incompatible to another machine and needs
to be redesigned from scratch. There are many ways to avoid doing from
scratch. You can always use preprocessor like #ifndef, #define, #else, and
#endif. It can recognize x86 machine and other non-x86 machine. It does
the same with little endian and big endian.
Alignment? Have you heard KW, MW, and GW? They stand for KiloWord,
MegaWord, and GigaWord. Like vs KiloByte, MegaByte, and GigaByte. It
depends what the company designs electronic circuit. Think of 16M space on
RAM. It has 16M cells on one chip. All 8 chips output 8 cells as 8 bit
each memory address. Also, it has 16 chips. The memory address can hold 16
cells like 16M Word.
Attempts to translate from 16-bit RAM to 8-bit RAM is always
incompatible unless you do clever design to make it compatible. (24-bit
address bus / 8-bit data bus vs 24-bit address bus / 16-bit data bus)
Twenty-four bit address bus and 16-bit data bus are not true x86 machine.
All x86 machine are designed to put four bytes on 32-bit data bus unlike
four words on 32-bit databus. I guess that it is non-standard, but it will
be at later time. Who knows? Word in each memory address is necessary to
expand 65,536 instructions instead of using pair of byte. It can execute
much faster and higher performance.

All caps? Why? It makes readable easier. Recognize word with all
upper caps so you know them are the definition of yours. Recognize word
with all lower caps are the keyword of C/C++ Compiler. Starting one upper
cap letter before all lower caps are easier to be named variables.

You're still being very confusing. First, W is a pointer
variable, whose value is independent of whatever B contains.
Second, you're not guaranteed to be able to do anything with the
pointer---B may not be sufficiently aligned to be accessed as a
word, for example, or a word may be more than four bytes.

No, I do not. I understand how pointer work. If you are referring
pointer as 4 bytes, you are using 32-bit processor machine, otherwise
pointer as 8 bytes are on 64-bit processor machine.

All this to do what? You're being very mysterious with regards
to the semantics you're trying to achieve.

Very funny thinking mysterious. I talk about memory address how each
byte, word, and dword are loaded from RAM into register or direct variable.
Then, you can manipulate individual byte or word or dword. It depends. I
guess that you are thinking about the emulator project. It may be true. I
write simulator by translating true electronic circuit into simulated
software for testing experiment.

I think we need to know what you are trying to do. For the
moment, I can't make any sense of it.

If your point is just that given something like:

int i ;
... i & 0xFF ...

the compiler loads a byte, rather than loading a word and then
anding 0xFF, that's certainly nothing new. It's a very common
idiom, and compilers have recognized it for ages. (If memory
serves me correctly, this optimization was already present in
Johnson's pcc, back before 1980.)

I can tell. C/C++ Compiler does reocgnize i & 0xFF. In release form,
it does not have "AND" instruction. & 0xFF is recognized and converted to
MOV instruction with 8 bit like this mov byte ptr [...], al

Bryan Parkoff

 

[James Kanze]

We discussed about the union keyword on the previous threads
2-3 days earlier. I want to add. Two byte data inside struct
and one word data are inside union. You modify word data.
Then two byte data are modified automatically because two byte
data and one word data are sharing the same memory address.
Let's move on and forget union because we claim that union
keyword is not the language of C/C++ Compiler.

No one ever said that union wasn't a keyword in C++ (or in C).

Here is a pointer variable code below.
enum
{
Low = 0,
High = 1
};
U_BYTE B[4] = { 0xFF, 0x20, 0x02, 0x22 };
PU_WORD W = (PU_WORD)&B;
PU_DWORD DW = (PU_DWORD)&B;
B[Low] += 0x0A;
B[High] += 0x01;
W[Low] += 0x0100;
*DW = 0x41424344; // *DW and DW[0] are same.
DW[0] = 0x51525354;

U_BYTE, PU_WORD and PU_DWORD, on the other hand, definitely
aren't keywords in C++. I'm just guessing, but I suppose that
they're typedefs:
typedef unsigned char U_BYTE ;
typedef unsigned int* PU_WORD ;
typedef unsigned long* PU_DWORD ;

Yes, it is true. You should always do not say U_BYTE is not
keyword of C/C++ Compiler. You see these words. It is simple
that you guess to know what I mean perfectly.

But it's not simple. My guess is only one possibility; U_BYTE
could easily have been a typedef to uint8_t instead, for
example.

I am aware that my code may be incompatible to another machine
and needs to be redesigned from scratch. There are many ways
to avoid doing from scratch. You can always use preprocessor
like #ifndef, #define, #else, and #endif. It can recognize
x86 machine and other non-x86 machine. It does the same with
little endian and big endian.

What can recognize x86 and not x86? You've lost me. (And of
course, you generally don't want to use #ifndef, etc. for
anything but include guards. They're a sure recepe for
unreadable code---just look at some of the Solaris headers, for
example.)

Alignment? Have you heard KW, MW, and GW? They stand for
KiloWord, MegaWord, and GigaWord. Like vs KiloByte, MegaByte,
and GigaByte. It depends what the company designs electronic
circuit. Think of 16M space on RAM. It has 16M cells on one
chip. All 8 chips output 8 cells as 8 bit each memory
address. Also, it has 16 chips. The memory address can hold
16 cells like 16M Word.

And? Again, I really don't know what you're talking about. I'm
not aware of any hardware that requires gigabyte alignment; I
don't think it would be usable. On the other hand, my compilers
(on a Sun Sparc) will put a char array at any address that's
convenient. Accessing an int at an address which is not a
multiple of four causes a core dump. Like many (most?)
architectures, the Sparc hardware imposes alignment
restrictions. x86 is one of the few that don't (but most
compilers will still align, because misalignment slows the
program down noticeably).

Attempts to translate from 16-bit RAM to 8-bit RAM is always
incompatible unless you do clever design to make it compatible. (24-bit
address bus / 8-bit data bus vs 24-bit address bus / 16-bit data bus)
Twenty-four bit address bus and 16-bit data bus are not true x86 machine.
All x86 machine are designed to put four bytes on 32-bit data bus unlike
four words on 32-bit databus. I guess that it is non-standard, but it will
be at later time. Who knows? Word in each memory address is necessary to
expand 65,536 instructions instead of using pair of byte. It can execute
much faster and higher performance.

Do you have any idea what you're talking about? I don't. (And
for what it's worth, I've actually designed computer hardware,
although that was a long time ago.)

All caps? Why?

Because it is a more or less generally accepted convention that
all caps are reserved for macros.

It makes readable easier.

All caps does not make readability easier. It's actually harder
to read all caps than lower case.

Recognize word with all upper caps so you know them are the
definition of yours. Recognize word with all lower caps are
the keyword of C/C++ Compiler. Starting one upper cap letter
before all lower caps are easier to be named variables.

That goes against all usual conventions. You recognize keywords
of the C++ compiler because you know them. (There aren't that
many, after all.) And of course, the editor displays them in a
different color or a different font as well. You recognize
macros because they are all caps. (And you definitly want to
make macros especially identifiable, since they don't obey any
of the usual rules.)

No, I do not. I understand how pointer work. If you are
referring pointer as 4 bytes, you are using 32-bit processor
machine, otherwise pointer as 8 bytes are on 64-bit processor
machine.

You do not what? I still can't really figure out what you're
talking about. Nothing you do with the variable B in your code
will ever modify W; they're two distinct variables, at different
addresses, and with different types.

Very funny thinking mysterious. I talk about memory address
how each byte, word, and dword are loaded from RAM into
register or direct variable. Then, you can manipulate
individual byte or word or dword.

Why? That's what I'm asking. I can't think of any problem for
which this would be the correct solution.

It depends. I guess that you are thinking about the emulator
project. It may be true. I write simulator by translating
true electronic circuit into simulated software for testing
experiment.

Aha. You're trying to emulate hardware. And since the register
set is a union on the hardware... (That's true for Intel, at
least, and probably some others. It's certainly not generally
true, of course.)

Emulating hardware portably is tricky business, of course: what
happens if the hardware you're emulating has 9 bit bytes, and
the machine you're emulating on only has 8 bit bytes? If I were
trying to emulate an 8086 portably, I'd probably use something
like regAX & 0xFF to access AL. But a lot would depend on the
exact portability constraints.

 

[Bryan Parkoff]

Alignment? Have you heard KW, MW, and GW? They stand for

And? Again, I really don't know what you're talking about. I'm
not aware of any hardware that requires gigabyte alignment; I
don't think it would be usable. On the other hand, my compilers
(on a Sun Sparc) will put a char array at any address that's
convenient. Accessing an int at an address which is not a
multiple of four causes a core dump. Like many (most?)
architectures, the Sparc hardware imposes alignment
restrictions. x86 is one of the few that don't (but most
compilers will still align, because misalignment slows the
program down noticeably).

Well, I agree.

Do you have any idea what you're talking about? I don't. (And
for what it's worth, I've actually designed computer hardware,
although that was a long time ago.)

For example, most architectures have 8-bit size in byte. One of them
has 9-bit size in byte. Also, there is another architecture that it has
16-bit size on byte!! It is what I called KiloWord vs KiloByte. You hold
1,024 bytes with 16-bit size so it looks like one KiloWord. How would you
translate 1,024 9-bit size byte to 1,024 8-bit size byte? You can't unless
you decide to put 9-bit size byte into 16-bit size word.
It does the same with 1,024 16-bit size byte to be translated into 1,024
32-bit size dword. It is very new and it is under development to design
hardware. It is never disclosed to the public.

Because it is a more or less generally accepted convention that
all caps are reserved for macros.

All caps does not make readability easier. It's actually harder
to read all caps than lower case.

That goes against all usual conventions. You recognize keywords
of the C++ compiler because you know them. (There aren't that
many, after all.) And of course, the editor displays them in a
different color or a different font as well. You recognize
macros because they are all caps. (And you definitly want to
make macros especially identifiable, since they don't obey any
of the usual rules.)

Store functions into macro is bad idea like C-style. C++ provides
inline keyword. Using macro is harder to find bug than inline does.

Aha. You're trying to emulate hardware. And since the register
set is a union on the hardware... (That's true for Intel, at
least, and probably some others. It's certainly not generally
true, of course.)

Emulating hardware portably is tricky business, of course: what
happens if the hardware you're emulating has 9 bit bytes, and
the machine you're emulating on only has 8 bit bytes? If I were
trying to emulate an 8086 portably, I'd probably use something
like regAX & 0xFF to access AL. But a lot would depend on the
exact portability constraints.

Then you say using #ifndef to choose little endian or big endian is bad
idea. Also, byte and word share the same memory address using union or
pointer is bad idea, too. Your solution is to recommend using "AND" and
"left / right shift" if you want code to be portable to all computer
machines.

It would look like this.

unsigned char* ram[0x10000] = { 0 };

unsigned char low_byte = ram[0x2000] & 0xFF;
unsigned char high_byte = ram[0x2001] & 0xFF;

unsigned short word = (high_byte << 8) | low_byte;

It may be possible that ram variable may have 9-bit size or higher. You
want to limit 8-bit size. This code above is the solution for best
portability. Why not create two separate source codes. First source code
is to use union for Intel / AMD. Second source code is to use "AND" and
"left / right shift" for other non-x86 machine. Use #define and #ifndef to
detect __INTEL_AND_AMD__ for first source code otherwise __NON_X86_MACHINE__
for second source code. Does it make sense?

Bryan Parkoff

 

[James Kanze]

[...]

For example, most architectures have 8-bit size in byte. One
of them has 9-bit size in byte. Also, there is another
architecture that it has 16-bit size on byte!!

I've also heard of 32 bit bytes. (In such cases, it's probably
best to specify that one means "byte" in the sense the standard
uses it, and not in its general computer science meaning, which
implies something less than word size... and we've also had 6
and 7 bit bytes, which the standard doesn't allow.)

It is what I called KiloWord vs KiloByte. You hold
1,024 bytes with 16-bit size so it looks like one KiloWord. How would you
translate 1,024 9-bit size byte to 1,024 8-bit size byte? You can't unless
you decide to put 9-bit size byte into 16-bit size word.
It does the same with 1,024 16-bit size byte to be translated into 1,024
32-bit size dword. It is very new and it is under development to design
hardware. It is never disclosed to the public.

So on some machines, a kiloword is the same thing as a kilobyte.
What does this have to do with alignment?

Store functions into macro is bad idea like C-style. C++ provides
inline keyword. Using macro is harder to find bug than inline does.

And? That's really part of the argument in C++. In C, a macro
that works exactly like a function should be written as one. By
definition, in well written C++, if you use a macro, it's
because the functionality cannot be expressed in a function.
And since it's doing something particular, you want it to stand
out and be recognized.

Obviously, naming conventions are just that: conventions. But
you're usually better off not fighting those which are generally
accepted.

Then you say using #ifndef to choose little endian or big endian is bad
idea.

Very. It's a quick path to almost unmaintainable code.

Having said that, it's really very, very rare to have code which
depends on endianness anyway. Except in very special
circumstances, there's just no need for it.

Also, byte and word share the same memory address using union or
pointer is bad idea, too. Your solution is to recommend using "AND" and
"left / right shift" if you want code to be portable to all computer
machines.

More or less. Simulating hardware that acts like a union might
be a special case, but in general, you define what you want to
do in terms of mathematical (perhaps bit-mathematical) functions
on values, and implement those. On one hand, it's far easier to
analyse, and to prove correctness, and on the other, the results
are garanteed portable.

If I had to emulate an 8086, on a single compiler, which did
guarantee that the union trick would work, I don't know.
Because if the problem is emulating the registers of an 8086,
you're very much talking about something that is conceptually a
union: you write to AX, and you read from AL, and they happend
to occupy the same bits. Conceptually, in this case, the union
actually fits better. Practically, portability concerns may
mean that you can't use it.

(You could get fancy, and make the register set a class, in
which the function al() returned a proxy object that did
whatever was necessary. But I'm a little sceptical about using
something this complicated to emulate such a low level
abstraction.)

It would look like this.

unsigned char* ram[0x10000] = { 0 };

unsigned char low_byte = ram[0x2000] & 0xFF;
unsigned char high_byte = ram[0x2001] & 0xFF;

unsigned short word = (high_byte << 8) | low_byte;

It may be possible that ram variable may have 9-bit size or
higher. You want to limit 8-bit size. This code above is the
solution for best portability. Why not create two separate
source codes. First source code is to use union for Intel /
AMD. Second source code is to use "AND" and "left / right
shift" for other non-x86 machine. Use #define and #ifndef to
detect __INTEL_AND_AMD__ for first source code otherwise
__NON_X86_MACHINE__ for second source code. Does it make
sense?

I would first define my portability needs very, very precisely.
Practically speaking, there's a good chance that I can assume 8
bit bytes; in which case, I'd use uint8_t, uint16_t, etc. (which
won't be available unless the compiler supports 8 bit 2's
complement bytes---so if ever the code is ported to a 9 bit
machine, it won't compile, rather than give a bad results). If
I did have to support a machine with 9 bit bytes, or might have
to in the future, I'd still only and in the 0xFF when
necessary; the extra bit would be treated as "don't care", not
as "must be zero". But that's more a personal choice.

As for storing and reading the data, yes, I'd only read and
write bytes, combining them into a word on an as needed basis.