How to enforce pointer alignment in a portable way (i.e. w/ 64 bit pointers)

K

Ken Turkowski

The construct
(void*)(((long)ptr + 3) & ~3)
worked well until now to enforce alignment of the pointer to long
boundaries. However, now VC++ warns about it, undoubtedly to help things
work on 64 bit machines, i.e. with 64 bit pointers.

In the early days of C, where there were problems with the size of int
being 16 or 32 bits, the response was that an int was guaranteed to hold
a pointer (yes, there were 64Kb address spaces at one time!). However, I
understand that this convention is being violated with the modern
compilers, so that
(void*)(((int)ptr + 3) & ~3)
doesn't work either. Is there an portable way of coercing pointers,
regardless of their size?
 
A

Arthur J. O'Dwyer

The construct
(void*)(((long)ptr + 3) & ~3)
worked well until now to enforce alignment of the pointer to long
boundaries.

Didn't work in 1989, doesn't work today. Get a better C tutorial.
However, now VC++ warns about it, undoubtedly to help things
work on 64 bit machines, i.e. with 64 bit pointers.

Good for them.
In the early days of C, where there were problems with the size of int
being 16 or 32 bits, the response was that an int was guaranteed to hold
a pointer (yes, there were 64Kb address spaces at one time!). However, I
understand that this convention is being violated with the modern
compilers, so that
(void*)(((int)ptr + 3) & ~3)
doesn't work either. Is there an portable way of coercing pointers,
regardless of their size?

Of course not. Luckily, you don't need to. If you think you do,
then explain your problem, and we can try to help you solve it. But
the only reason to be worried about pointer alignment is if you're
implementing 'malloc' and friends, and in that case you don't need
to worry about portability (and there are well-known tricks with
'union' to help you cope).

Post your real problem, if you've got one. If you were just
curious about bitwise tricks on pointers, the answer is even simpler:
Pointers don't allow bitwise tricks. Forget it.

HTH,
-Arthur
 
B

Ben Pfaff

Ken Turkowski said:
Is there an portable way of coercing pointers, regardless of
their size?

intptr_t and uintptr_r are C99 integer types defined in
<stdint.h>that are guaranteed large enough to hold a pointer, but
they are optional.
 
K

Ken Turkowski

"Arthur J. O'Dwyer said:
Didn't work in 1989, doesn't work today. Get a better C tutorial.

On the contrary, a lot of the software you use every day uses constructs
like this.
Good for them.


Of course not. Luckily, you don't need to. If you think you do,
then explain your problem, and we can try to help you solve it. But
the only reason to be worried about pointer alignment is if you're
implementing 'malloc' and friends, and in that case you don't need
to worry about portability (and there are well-known tricks with
'union' to help you cope).

Post your real problem, if you've got one. If you were just
curious about bitwise tricks on pointers, the answer is even simpler:
Pointers don't allow bitwise tricks. Forget it.

The speed of copying one image to another is faster when you use a
larger data type. If the source and destinations coincide in 8-byte
alignment, for example, you can do
double *f, *t; long n;
... setup ...
while (n--)
*t++ = *f++;
It is faster, though not quite 8X faster, to copy doubles rather than
bytes.
 
J

Jack Klein

The construct
(void*)(((long)ptr + 3) & ~3)
worked well until now to enforce alignment of the pointer to long
boundaries. However, now VC++ warns about it, undoubtedly to help things
work on 64 bit machines, i.e. with 64 bit pointers.

In the early days of C, where there were problems with the size of int
being 16 or 32 bits, the response was that an int was guaranteed to hold
a pointer

No it was not, not even in the early days.
 
K

Ken Turkowski

intptr_t and uintptr_r are C99 integer types defined in
<stdint.h>that are guaranteed large enough to hold a pointer, but
they are optional.

Great. I have this on my Macintosh, but don't on Windows.

The Windows documentation defines SIZE_T an d SSIZE_T to be integers the
size of a pointer.

This suggests that size_t might be a cross-platform way of achieving the
same thing as uintptr_t, and I suspect that off_t works like intptr_t.

I tested this in my code. It silences Windows warnings, and doesn't
introduce warnings on the Macintosh. So at least it's a short-term
solution for my problem.
 
K

Keith Thompson

Ken Turkowski said:
On the contrary, a lot of the software you use every day uses constructs
like this.

It may "work", but it's not guaranteed by the standard. It invokes
undefined behavior.

[...]
The speed of copying one image to another is faster when you use a
larger data type. If the source and destinations coincide in 8-byte
alignment, for example, you can do
double *f, *t; long n;
... setup ...
while (n--)
*t++ = *f++;
It is faster, though not quite 8X faster, to copy doubles rather than
bytes.

Have you tried using memcpy()? It's very likely that it already has
whatever optimizations you're trying to implement.
 
J

Jim

Even ignoring alignment issues, the code above doesn't work.
Some of the bit patterns you are loading are not guaranteed to
represent valid double values. Depending on your CPU architecture,
this could cause problems.

Jim
 
X

xarax

Jim said:
Even ignoring alignment issues, the code above doesn't work.
Some of the bit patterns you are loading are not guaranteed to
represent valid double values. Depending on your CPU architecture,
this could cause problems.

Jim

Exactly. Many implementations will load the double
into a floating point register, then store that
register back to memory. Floating point registers
may or may not fudge the loaded value, or perhaps
throw an exception on a NaN (Not A Number) bit pattern.

Using a 64-bit integer type may work, but it's
safer to use memcpy() which will generate an efficient
instruction stream for your implementation.

--
----------------------------
Jeffrey D. Smith
Farsight Systems Corporation
24 BURLINGTON DRIVE
LONGMONT, CO 80501-6906
http://www.farsight-systems.com
z/Debug debugs your Systems/C programs running on IBM z/OS for FREE!
 
T

Thomas Matthews

Ken said:
The construct
(void*)(((long)ptr + 3) & ~3)
worked well until now to enforce alignment of the pointer to long
boundaries. However, now VC++ warns about it, undoubtedly to help things
work on 64 bit machines, i.e. with 64 bit pointers.

In the early days of C, where there were problems with the size of int
being 16 or 32 bits, the response was that an int was guaranteed to hold
a pointer (yes, there were 64Kb address spaces at one time!). However, I
understand that this convention is being violated with the modern
compilers, so that
(void*)(((int)ptr + 3) & ~3)
doesn't work either. Is there an portable way of coercing pointers,
regardless of their size?

Pointers on different platforms may have different structures.
For example, an Intel 80186 processor still uses the segment:eek:ffset
notation for addresses. The 80386 (and above) processors have
the ability to have a flat (unsegmented) architecture. The ARM
processor has a 32-bit flat address model.

More complex processors have things like paging bits in their
pointers. Some may have different sized pointers based on
where the data is or the size of the data. A classic example
is a "near" pointer and a "far" pointer. The Signetics 2650
8-bit processor had relative (7-bit) addressing and 16-bit
full addressing.

So basically, you can't portably convert a pointer to an
integral type. As far as alignment goes, leave it to the
compiler or use a platform specific casting hack.

--
Thomas Matthews

C++ newsgroup welcome message:
http://www.slack.net/~shiva/welcome.txt
C++ Faq: http://www.parashift.com/c++-faq-lite
C Faq: http://www.eskimo.com/~scs/c-faq/top.html
alt.comp.lang.learn.c-c++ faq:
http://www.raos.demon.uk/acllc-c++/faq.html
Other sites:
http://www.josuttis.com -- C++ STL Library book
 
S

S.Tobias

Arthur J. O'Dwyer said:
Of course not. Luckily, you don't need to. If you think you do,
then explain your problem, and we can try to help you solve it. But
the only reason to be worried about pointer alignment is if you're
implementing 'malloc' and friends, and in that case you don't need
to worry about portability (and there are well-known tricks with
'union' to help you cope).

I have a similar problem, that I'll have to resolve soon.
I build a transactional storage and I store my data on disk pages.
My system is layered, ie. paging and caching is on the bottom,
then transaction processing, and logical structure (eg. hash table,
btree etc.) on top. Each higher level layer obtains a page (or
rather part of a page that it is allowed to manage) from lower
level; in this sense it is malloc-like problem.

The page may look more-or-less like this (symbolically):
[address +0]
struct transaction_header t_h;
struct structure_header s_h;
struct data0_header d_h0;
char data0[*];
size_t int0;
struct data1_header d_h1;
char data1[*];
size_t int1;
[address +free_offset]
... ("free" space)
[address +free_offset + free_size]
size_t ptr1;
size_t ptr0;
struct transaction_footer t_f;
[address +page_size]

I don't want to do manual data "packing", because that would cost
additional cpu cycles (currently memory transfers take most of
the time); I want to access the data in machine native format
directly on page (cross-platform portability of the data is irrelevant).

Problems to solve:
(Suppose transaction_header starts at malloc()'ed address, thus is
aligned for any data type, and it is known.)
1. When transactional layer passes a page to structure layer,
it has to pass an address (here: for structure_header) aligned
for any data type (this is easy, I have to embed a most alignment
requiring type via a union into transaction_header).
2. I have to somehow calculate the address for transaction_footer, which
has to be fit at the very end of the page.
I could do this like this:
(struct transaction_footer*)address
+ (page_size/sizeof(struct transaction_footer) - 1)
3. After I write data0, I have to find an address for int0 and d_h1.
The general issue is that I might have a sequence of different
types. The solution is practically same as in 2., but I have
to keep "most aligned" reference pointer (here: transaction_header,
or structure_header), which might be a little nuisance.
4. I have to pre-calculate the needed space. This seems the most
difficult part to make in portable C.
Suppose I have d_hN, dataN and intN (and ptrN) to store on a page.
The transactional layer can return free_size and free_offset before
it takes any other action (suppose they're cached somewhere);
it doesn't know about structure's alignment requirements.
Now, how could I check if my data would fit in the free space?
For the first thing: the actual size needed depends on the value
of [address+free_offset] (for d_hN might not start exactly there);
for the second: I cannot do pointer arithmetic without having
a *valid* pointer (to make it Standard compliant I would have
to allocate memory for arithmetic purposes only, which is out of
the question). Doing "integer pointer" arithmetic seems
the only solution.

Is there a C solution to a more generic problem: Having a pointer (void*)
and a data_type, calculate the nearest position after the pointer
at which we may put a datum of data_type? If yes, can we do it (ie.
calculate) without having (enough) valid memmory allocated?

I'd be more than happy to hear ideas on these issues. TIA
 
K

Kenneth Brody

Keith said:
It may "work", but it's not guaranteed by the standard. It invokes
undefined behavior.
[...]

Can a compiler vendor "define" what happens when you invoke "undefined
behavior"?

In other words, can a compiler vendor say "the above construct will align
the pointer to a 32-bit boundary"? Of course, this will only apply to this
particular vendor's particular compiler. But, does "defining" the behavior
break the standard if the standard says it's "undefined behavior"?
 
L

Lew Pitcher

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Kenneth Brody wrote:
[snip]
Can a compiler vendor "define" what happens when you invoke "undefined
behavior"?

In other words, can a compiler vendor say "the above construct will align
the pointer to a 32-bit boundary"? Of course, this will only apply to this
particular vendor's particular compiler. But, does "defining" the behavior
break the standard if the standard says it's "undefined behavior"?

As I understand it, when the standard says that something induces
"undefined behaviour", it means that the behaviour induced is not
defined /by the standard/.

So, a particular compiler /can/ define the behaviour of something that
the standard says results in "undefined behaviour" without violating the
standard. It just means that a program that performs such functions will
result in behaviour not defined by the standard, and thus the behaviour
will not be consistant across all platforms.


- --

Lew Pitcher, IT Consultant, Enterprise Application Architecture
Enterprise Technology Solutions, TD Bank Financial Group

(Opinions expressed here are my own, not my employer's)
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.4 (MingW32)

iD8DBQFA7r4aagVFX4UWr64RAsECAJ9wTgy0YBm4VWkpnOcp6l+3EsDphgCfcwvl
ZX2+gG9+CRrUm9J+d9NJf7g=
=n3sA
-----END PGP SIGNATURE-----
 
C

Chris Torek

Can a compiler vendor "define" what happens when you invoke "undefined
behavior"?

Certainly.

["the above construct" was something like (((int)ptr + 3) & ~3), here]
In other words, can a compiler vendor say "the above construct will align
the pointer to a 32-bit boundary"? Of course, this will only apply to this
particular vendor's particular compiler. But, does "defining" the behavior
break the standard if the standard says it's "undefined behavior"?

Not at all -- "undefined behavior" is a two-way escape hatch, both
down into the Hell of "this code will never work" *and* up into the
Heaven of "now you can achieve this thing that the Standard fails
to provide".

The problem is, the endpoint of each hatch is typically unmarked. :)
All too often, compiler-vendors leave "undefined but useful" behavior
in their compilers, while people get hooked on the useful aspect;
then the vendor will *change* it in a future release -- often even
for a good reason -- and those who are hooked now discover that the
actual direction of that particular escape hatch was "down" (assuming
of course the traditional placement for the two destinations I
mentioned :) ).
 
G

Gordon Burditt

It may "work", but it's not guaranteed by the standard. It invokes
undefined behavior.
[...]

Can a compiler vendor "define" what happens when you invoke "undefined
behavior"?

Yes. Chances are the compiler vendor will not define what happens
in the NEXT VERSION of said compiler, though. And it might not be
a USEFUL definition (e.g. this works the way you want if the pointer
is in an even-numbered register at the point where you use it -
where you don't get to control register allocation at all. The
"register" keyword does not allow you to select between odd and
even registers, and it may not do anything at all.)

If you depend on what the vendor defines, you are limited to the
compilers with that guarantee. You may get stuck not being able
to upgrade without having things break. You may get stuck having
to upgrade the OS (due to, say, security vulnerabilities or the OS
no longer has support), having the old compiler version no longer
work, and having the new version no longer provide the behavior
your code depends on.
In other words, can a compiler vendor say "the above construct will align
the pointer to a 32-bit boundary"? Of course, this will only apply to this
particular vendor's particular compiler.

It probably applies to this particular vendor's particular compiler
of given version, patchlevel, and perhaps also the version of the
OS it's running on, the compiler serial number, date and time of
registration, and amount of sales tax paid.
But, does "defining" the behavior
break the standard if the standard says it's "undefined behavior"?

No. For example, in many systems the OS and the hardware will give
you a hardware trap (mapped by the compiler into a Segmentation
Fault) if you attempt to access memory that's not mapped into your
address space. The compiler vendor would have to work really hard
to make the behavior something else. Lots of systems do this and
there's nothing in the C standard preventing them from writing this
information down. Intel's manuals for the Pentium processors go
into considerable detail about this. Compiler manuals may mention
what hardware traps get mapped into a SIGSEGV signal.

Another area where Intel goes into excrutiating detail is in the
handling of exceptional conditions in floating point. In this case
the compiler vendor may actually provide functions that can access
hardware registers that control the behavior (e.g. silent underflow
vs. a significance loss trap), and distinguish trapping vs.
non-trapping NaNs. However, ANSI C does not require IEEE floating
point at all.

Gordon L. Burditt
 
D

Dan Pop

In said:
Can a compiler vendor "define" what happens when you invoke "undefined
behavior"?

Yes, and contrary to what other people have told you, once the vendor
actually documents it, it becomes a feature of the VendorX C language,
which is highly unlikely to change in the future.

It may change, but the change will definitely not happen overnight and
catch everybody by surprise. OTOH, the same is true for standard C:
C99 does break correct and portable C89 code (or merely renders its
behaviour undefined), so even if you stay away from any kind of
undefined behaviour and never change hardware platforms and OS, you
still have no full guarantee that your code will continue to work on
the next compiler released by VendorX.

Try compiling the following strictly conforming C89 program with a C99
compiler (or gcc in C99 mode):

int main()
{
int restrict = 0;
return restrict;
}

Dan
 
C

Christian Bau

Ken Turkowski said:
The construct
(void*)(((long)ptr + 3) & ~3)
worked well until now to enforce alignment of the pointer to long
boundaries. However, now VC++ warns about it, undoubtedly to help things
work on 64 bit machines, i.e. with 64 bit pointers.

In the early days of C, where there were problems with the size of int
being 16 or 32 bits, the response was that an int was guaranteed to hold
a pointer (yes, there were 64Kb address spaces at one time!). However, I
understand that this convention is being violated with the modern
compilers, so that
(void*)(((int)ptr + 3) & ~3)
doesn't work either. Is there an portable way of coercing pointers,
regardless of their size?

The mistake that you make is trying to cast an integer type to a pointer
type which is highly non-portable. For what you want to achieve, there
is absolutely no need to do this: Cast the pointer to char*, then
subtract ((unsigned int) ptr) & 0x3, then cast to whatever type you
wanted.

The only casts of pointer types are from the original type to char* and
back, and that is portable. The calculation ((unsigned int) ptr) & 0x3
has no problems with 64bit-ness because the result is in the range from
0 to 3 and fits into unsigned int; whether you use unsigned int,
unsigned long, unsigned long long, unsigned char or unsigned short, it
doesn't matter at all (as long as you align to a multiple of 256 bytes
or less; unsigned short is fine up to 65536 bytes).

The only non-portable part is that ((unsigned int) ptr) & 0x03 might
have nothing to do with the alignment of the pointer at all.

char* p = ...;
p -= ((unsigned int) p) & 0x3;

will subtract a value from 0 to 3 from p, and in most implementations p
will be aligned to a multiple of four bytes afterwards.
 
D

Dave Thompson

Great. I have this on my Macintosh, but don't on Windows.

The Windows documentation defines SIZE_T an d SSIZE_T to be integers the
size of a pointer.

This suggests that size_t might be a cross-platform way of achieving the
same thing as uintptr_t, and I suspect that off_t works like intptr_t.
The C standard requires size_t and ptroff_t be provided that can
represent respectively the size of any valid (declared or malloc'ed)
object which is also the largest positive-only valid difference of
character pointers, and any positive-or-negative difference of
character pointers. These don't have be as large as, or even have as
large a range as, pointers, although in simple flat-memory models
(which includes Win32 and both old and new MacOS) they probably do.
(It is possible with calloc's two arguments to specify a size larger
than the range of size_t, but it's not clear such sizes must work.)

POSIX/SUS, but not C, requires off_t that can represent any position
in a (stdio supported) file. It is difficult to come up with a
reasonable platform, other(?) than AS/400, where that is not as large
as any in-memory pointer, but nothing explicitly requires it.
I tested this in my code. It silences Windows warnings, and doesn't
introduce warnings on the Macintosh. So at least it's a short-term
solution for my problem.

- David.Thompson1 at worldnet.att.net
 
P

pete

Dave said:
The C standard requires size_t and ptroff_t be provided that can
represent respectively the size of any valid (declared or malloc'ed)
object which is also the largest positive-only valid difference of
character pointers, and any positive-or-negative difference of
character pointers.

You misspelled "ptrdiff_t".
ptrdiff_t is *not* guaranteed to be able to represent
any positive-or-negative difference of character pointers.
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

Forum statistics

Threads
473,744
Messages
2,569,483
Members
44,901
Latest member
Noble71S45

Latest Threads

Top