alligned memory allocation

Discussion in 'C Programming' started by Kutty Banerjee, Mar 4, 2004.

  1. Hi,
    I ve got the following piece of code which does the role of allocating
    aligned memory (not sure what aligned memory allocation is either).

    void *
    _align_calloc(size_t bytes, unsigned long alignment)
    {
    unsigned long ptr, buf;

    ASSERT( alignment > 0 );

    ptr = (unsigned long) calloc(bytes + alignment + sizeof(void *));
    if (!ptr)
    return NULL;

    buf = (ptr + alignment + sizeof(void *)) & ~(unsigned long)(alignment -
    1);
    *(unsigned long *)(buf - sizeof(void *)) = ptr;

    #ifdef DEBUG
    /* mark the non-aligned area */
    while ( ptr < buf - sizeof(void *) ) {
    *(unsigned long *)ptr = 0xcdcdcdcd;
    ptr += sizeof(unsigned long);
    }
    #endif

    return (void *)buf;
    }

    Can somebody please explain what this code is doing? I m a little confused
    with the memory being allocated to a non pointer data namely unsigned long
    ptr ?


    kutty
    Kutty Banerjee, Mar 4, 2004
    #1
    1. Advertising

  2. In article <c26nfc$1pot$>,
    "Kutty Banerjee" <> wrote:

    > Hi,
    > I ve got the following piece of code which does the role of allocating
    > aligned memory (not sure what aligned memory allocation is either).
    >
    > void *
    > _align_calloc(size_t bytes, unsigned long alignment)
    > {
    > unsigned long ptr, buf;
    >
    > ASSERT( alignment > 0 );
    >
    > ptr = (unsigned long) calloc(bytes + alignment + sizeof(void *));
    > if (!ptr)
    > return NULL;
    >
    > buf = (ptr + alignment + sizeof(void *)) & ~(unsigned long)(alignment -
    > 1);
    > *(unsigned long *)(buf - sizeof(void *)) = ptr;
    >
    > #ifdef DEBUG
    > /* mark the non-aligned area */
    > while ( ptr < buf - sizeof(void *) ) {
    > *(unsigned long *)ptr = 0xcdcdcdcd;
    > ptr += sizeof(unsigned long);
    > }
    > #endif
    >
    > return (void *)buf;
    > }
    >
    > Can somebody please explain what this code is doing? I m a little confused
    > with the memory being allocated to a non pointer data namely unsigned long
    > ptr ?


    It is a case of a clueless programmer trying to return a pointer that is
    aligned in a certain way. I'll give you an overview what he is trying to
    do and where he gets it wrong:

    malloc and calloc return pointers that are properly aligned for _any_ C
    type. Now there might be reasons why you would want a pointer that is
    aligned to multiples of 128 bytes, for example, and malloc won't do
    that.

    So what he does is allocate a pointer that is bigger, lets say a pointer
    "char* p", and in the case of 128 byte alignment you would add some
    value from 0 to 127 to make it aligned on a multiple of 128 bytes. But
    if you did just that and returned the result pointer, then you wouldn't
    be able to free the memory later: To be able to call free, you must have
    the original pointer. So you store the original pointer just in front of
    the pointer that you return.

    Now your original author does a few remarkably stupid things. He makes
    assumptions that are completely unwarranted in portable C and actually
    wrong on more and more computers that are in actual use. No wonder you
    are confused :-(

    First, he assumes that "alignment" is not only greater than zero, but
    also a power of two, AND at least as large as the alignment requirement
    of a void* value. Why doesn't he write this in the ASSERT? To test
    whether x is a power of two: If you write x and x-1 in binary, then if x
    is a power of two no bit will be set both in x and x-1. If x is nonzero
    and not a power of two, then the highest bit of x is also set in x-1. So
    you could write

    ASSERT(align >= sizeof (void *) && (align & (align-1)) != 0);

    Next, he assumes that you can cast a void* to unsigned long and perform
    pointer arithmetic on the unsigned long. Bullshit. Take an Athlon 64 and
    you may be surprised to see that void* = 64 bit, unsigned long = 32 bit,
    and casting void* to unsigned long produces a whole lot of garbage. Take
    16 bit DOS with "huge" memory model, and adding to an unsigned long to
    do pointer arithmetic will produce garbage. Try it on a Cray and heaven
    knows what you get. But in fact it is very easy to do what he wants in a
    much more portable way:

    char* p = calloc (bytes + alignment + sizeof (void *));
    char* q = p + sizeof (void *) + alignment;
    q -= ((unsigned long) q) % alignment;

    // If you want your code unreadable like the original author,
    // then change the last line to
    // q -= ((unsigned long) q) & (alignment - 1);

    Now it doesn't matter what the relative sizes of pointers and unsigned
    long are.

    The next mistake that he makes is that you can store an unsigned long if
    you have space for a void*. Bad mistake. Just as you have systems where
    sizeof (void *) > sizeof (unsigned long), you have systems where it is
    the other way round. Buy a new Macintosh and you will get a machine that
    uses 64 bit integers and 32 bit pointers. So he has this pointer "buf",
    subtracts four byte = sizeof (void *), and stores an eight byte unsigned
    long. Congratulations. The last four bytes of that unsigned long will be
    overwritten by the caller as soon as they use the memory. Crash when you
    try to free the memory later on.

    Have fun with this guy's code. Expect the worst. People who think they
    can program and can't are the worst kind.
    Christian Bau, Mar 4, 2004
    #2
    1. Advertising

  3. Kutty Banerjee

    CBFalconer Guest

    Christian Bau wrote:
    > "Kutty Banerjee" <> wrote:
    >
    > > I ve got the following piece of code which does the role of
    > > allocating aligned memory (not sure what aligned memory
    > > allocation is either).
    > >
    > > void *
    > > _align_calloc(size_t bytes, unsigned long alignment)
    > > {

    .... snip code ...
    > >
    > > Can somebody please explain what this code is doing? I m a
    > > little confused with the memory being allocated to a non
    > > pointer data namely unsigned long ptr ?

    >
    > It is a case of a clueless programmer trying to return a pointer
    > that is aligned in a certain way. I'll give you an overview what
    > he is trying to do and where he gets it wrong:
    >

    .... snip ...
    >
    > Have fun with this guy's code. Expect the worst. People who
    > think they can program and can't are the worst kind.


    The code is obviously not portable, and is also obviously intended
    for system use (the _align... name is not in the user name
    space). This is a problem that is simply not soluble with
    portable code, so the only requirement is that it work, somehow or
    other, with the original implementation. Your caustic criticisms
    are probably undeserved.

    To the OP: The internal workings of a particular system are often
    wierd and wonderful, and highly non-standard. You are not
    expected to understand them on the basis of the language.

    --
    Chuck F () ()
    Available for consulting/temporary embedded and systems.
    <http://cbfalconer.home.att.net> USE worldnet address!
    CBFalconer, Mar 4, 2004
    #3
  4. In article <>,
    CBFalconer <> wrote:

    > Christian Bau wrote:
    > > Have fun with this guy's code. Expect the worst. People who
    > > think they can program and can't are the worst kind.

    >
    > The code is obviously not portable, and is also obviously intended
    > for system use (the _align... name is not in the user name
    > space). This is a problem that is simply not soluble with
    > portable code, so the only requirement is that it work, somehow or
    > other, with the original implementation. Your caustic criticisms
    > are probably undeserved.


    The point is that this code is unportable without any good reason. And
    assuming that unsigned long and void* have the same size, today, is just
    utterly stupid. Not the "it works everywhere except on the DeathStation
    9000" kind of stupid, but the "I bought this brand new computer and my
    code crashes all over the place" kind of stupid.
    Christian Bau, Mar 4, 2004
    #4
  5. Kutty Banerjee

    CBFalconer Guest

    Christian Bau wrote:
    > CBFalconer <> wrote:
    > > Christian Bau wrote:

    >
    > > > Have fun with this guy's code. Expect the worst. People who
    > > > think they can program and can't are the worst kind.

    > >
    > > The code is obviously not portable, and is also obviously intended
    > > for system use (the _align... name is not in the user name
    > > space). This is a problem that is simply not soluble with
    > > portable code, so the only requirement is that it work, somehow or
    > > other, with the original implementation. Your caustic criticisms
    > > are probably undeserved.

    >
    > The point is that this code is unportable without any good reason.
    > And assuming that unsigned long and void* have the same size, today,
    > is just utterly stupid. Not the "it works everywhere except on the
    > DeathStation 9000" kind of stupid, but the "I bought this brand new
    > computer and my code crashes all over the place" kind of stupid.


    You yourself pointed out a reason for it - to provide alignment to
    larger values than does malloc. Unsigned long and void* may well
    have the same size on that system, they do under DJGPP. I believe
    gcc requires such system routines (although I don't approve).
    This is probably not the forum to ask questions about it, although
    I am hard pressed to think of a suitable newsgroup.

    --
    Chuck F () ()
    Available for consulting/temporary embedded and systems.
    <http://cbfalconer.home.att.net> USE worldnet address!
    CBFalconer, Mar 5, 2004
    #5
  6. CBFalconer <> wrote in message news:<>...
    > Christian Bau wrote:

    <snip>
    > >
    > > Have fun with this guy's code. Expect the worst. People who
    > > think they can program and can't are the worst kind.

    >
    > The code is obviously not portable, and is also obviously intended
    > for system use (the _align... name is not in the user name
    > space). This is a problem that is simply not soluble with
    > portable code, so the only requirement is that it work, somehow or
    > other, with the original implementation. Your caustic criticisms
    > are probably undeserved.


    I strongly disagree. As Christian pointed out, the original code
    chooses unportability even though choosing (more) portability is
    essentially free.

    In fact, to me it's more unmaintainable. If you're doing something
    like this, I immediately wonder why the author did not simply call
    memalign then memset. Or simply wrote a small function to round the
    alignment to the nearest power of two and then do a memalign and
    memset. Now I just spent time that I shouldn't have had to spend.

    Bad code is bad code, whether it works on the original implementation
    or not. The mark of a bad programmer is that they write portable code
    when there's a reason to. The mark of a good programmer is that they
    write portable code when there is no reason not to.

    Mark F. Haigh
    Mark F. Haigh, Mar 5, 2004
    #6
  7. In article <>,
    CBFalconer <> wrote:

    > Christian Bau wrote:
    > > CBFalconer <> wrote:
    > > > Christian Bau wrote:

    > >
    > > > > Have fun with this guy's code. Expect the worst. People who
    > > > > think they can program and can't are the worst kind.
    > > >
    > > > The code is obviously not portable, and is also obviously intended
    > > > for system use (the _align... name is not in the user name
    > > > space). This is a problem that is simply not soluble with
    > > > portable code, so the only requirement is that it work, somehow or
    > > > other, with the original implementation. Your caustic criticisms
    > > > are probably undeserved.

    > >
    > > The point is that this code is unportable without any good reason.
    > > And assuming that unsigned long and void* have the same size, today,
    > > is just utterly stupid. Not the "it works everywhere except on the
    > > DeathStation 9000" kind of stupid, but the "I bought this brand new
    > > computer and my code crashes all over the place" kind of stupid.

    >
    > You yourself pointed out a reason for it - to provide alignment to
    > larger values than does malloc. Unsigned long and void* may well
    > have the same size on that system, they do under DJGPP.


    > I believe
    > gcc requires such system routines (although I don't approve).


    And I hope sincerely that they have been written better.

    > This is probably not the forum to ask questions about it, although
    > I am hard pressed to think of a suitable newsgroup.


    char* p;
    unsigned long x;

    * (unsigned long *) (p - sizeof (void *)) = x;

    is not non-portable, it is just plain stupid. The original poster was
    confused _by the stupidity_ of the code. There were other errors in the
    code that had nothing to do with portability, but just with stupidity.
    Christian Bau, Mar 5, 2004
    #7
  8. Mark F. Haigh wrote:

    > CBFalconer <> wrote in message news:<>...
    >
    >>Christian Bau wrote:

    >
    > <snip>
    >
    >>>Have fun with this guy's code. Expect the worst. People who
    >>>think they can program and can't are the worst kind.

    >>
    >>The code is obviously not portable, and is also obviously intended
    >>for system use (the _align... name is not in the user name
    >>space). This is a problem that is simply not soluble with
    >>portable code, so the only requirement is that it work, somehow or
    >>other, with the original implementation. Your caustic criticisms
    >>are probably undeserved.

    >
    >
    > I strongly disagree. As Christian pointed out, the original code
    > chooses unportability even though choosing (more) portability is
    > essentially free.
    >
    > In fact, to me it's more unmaintainable. If you're doing something
    > like this, I immediately wonder why the author did not simply call
    > memalign then memset. Or simply wrote a small function to round the
    > alignment to the nearest power of two and then do a memalign and
    > memset. Now I just spent time that I shouldn't have had to spend.
    >
    > Bad code is bad code, whether it works on the original implementation
    > or not. The mark of a bad programmer is that they write portable code
    > when there's a reason to. The mark of a good programmer is that they
    > write portable code when there is no reason not to.
    >


    yeah, yeah, memalign not in C standard, or POSIX for that matter, yeah
    yeah. My point remains essentially the same. If memalign, snprintf,
    etc, are not implemented on your system, then implement 'em and use 'em.
    Even at a glance, the entire _align_calloc thing the OP quoted should
    jump out at you and say 'probably a bad idea'.

    Some of these supposed systems programming guys come up with this kind
    of thing and then throw casts around until the compiler shuts up. This
    is neither weird nor wonderful, just annoying to those of us that have
    to clean up afterwards.

    Or maybe I'm just cynical, you decide.



    Mark F. Haigh
    Mark F. Haigh, Mar 5, 2004
    #8
  9. Kutty Banerjee

    Dan Pop Guest

    In <> CBFalconer <> writes:

    >You yourself pointed out a reason for it - to provide alignment to
    >larger values than does malloc.


    malloc provides maximally aligned blocks, *by definition*:

    The pointer returned if the allocation succeeds
    is suitably aligned so that it may be assigned to a pointer to
    any type of object and then used to access such an object or an
    ^^^^^^^^^^^^^^^^^^
    array of such objects in the space allocated (until the space
    is explicitly deallocated).

    Dan
    --
    Dan Pop
    DESY Zeuthen, RZ group
    Email:
    Dan Pop, Mar 5, 2004
    #9
  10. Kutty Banerjee

    CBFalconer Guest

    Dan Pop wrote:
    > CBFalconer <> writes:
    >
    >> You yourself pointed out a reason for it - to provide alignment to
    >> larger values than does malloc.

    >
    > malloc provides maximally aligned blocks, *by definition*:
    >
    > The pointer returned if the allocation succeeds
    > is suitably aligned so that it may be assigned to a pointer to
    > any type of object and then used to access such an object or an
    > ^^^^^^^^^^^^^^^^^^
    > array of such objects in the space allocated (until the space
    > is explicitly deallocated).


    For the C language. There are entities in the world other than C
    language objects, and sometimes C code is used to organize and
    manipulate them.

    --
    Chuck F () ()
    Available for consulting/temporary embedded and systems.
    <http://cbfalconer.home.att.net> USE worldnet address!
    CBFalconer, Mar 5, 2004
    #10
  11. Kutty Banerjee

    Dan Pop Guest

    In <> CBFalconer <> writes:

    >Dan Pop wrote:
    >> CBFalconer <> writes:
    >>
    >>> You yourself pointed out a reason for it - to provide alignment to
    >>> larger values than does malloc.

    >>
    >> malloc provides maximally aligned blocks, *by definition*:
    >>
    >> The pointer returned if the allocation succeeds
    >> is suitably aligned so that it may be assigned to a pointer to
    >> any type of object and then used to access such an object or an
    >> ^^^^^^^^^^^^^^^^^^
    >> array of such objects in the space allocated (until the space
    >> is explicitly deallocated).

    >
    >For the C language. There are entities in the world other than C
    >language objects, and sometimes C code is used to organize and
    >manipulate them.


    How is this possible, without treating them as C objects?

    Dan
    --
    Dan Pop
    DESY Zeuthen, RZ group
    Email:
    Dan Pop, Mar 5, 2004
    #11
  12. Kutty Banerjee

    xarax Guest

    "Dan Pop" <> wrote in message
    news:c2a98e$a5h$...
    > In <> CBFalconer <> writes:
    >
    > >Dan Pop wrote:
    > >> CBFalconer <> writes:
    > >>
    > >>> You yourself pointed out a reason for it - to provide alignment to
    > >>> larger values than does malloc.
    > >>
    > >> malloc provides maximally aligned blocks, *by definition*:
    > >>
    > >> The pointer returned if the allocation succeeds
    > >> is suitably aligned so that it may be assigned to a pointer to
    > >> any type of object and then used to access such an object or an
    > >> ^^^^^^^^^^^^^^^^^^
    > >> array of such objects in the space allocated (until the space
    > >> is explicitly deallocated).

    > >
    > >For the C language. There are entities in the world other than C
    > >language objects, and sometimes C code is used to organize and
    > >manipulate them.

    >
    > How is this possible, without treating them as C objects?


    Depends on the hardware on which the C program is
    running, which means that the C code is not intended
    to be portable to other platforms, yet still have
    defined behavior and not use implementation-dependent
    compiler features.

    The IBM Mainframe has hardware structures of various
    alignments, like 64-byte, 256-byte, 4096-byte,
    1MB alignment, and larger. There is no C compiler
    "object" that conforms to those alignments, yet
    it's necessary to represent such requirements in
    C code. That requires pointer<->integer conversions
    and arithmetic that yield defined behavior. In such
    cases, malloc() will not yield a properly aligned
    "object". The usual approach is to "over allocate"
    with malloc() and massage the pointer upward to an
    appropriate alignment boundary, saving somewhere the
    original pointer for a subsequent free().

    A smart implementation of calloc() and malloc()
    would examine the length to determine a plausible
    maximum alignment and attempt to return a pointer
    that conforms to the plausible max alignment, but
    that is not a requirement. They only need to conform
    to maximum alignment of a primitive C object (like
    a "long long" or a "double", whatever is largest in
    the implementation).


    --
    ----------------------------
    Jeffrey D. Smith
    Farsight Systems Corporation
    24 BURLINGTON DRIVE
    LONGMONT, CO 80501-6906
    http://www.farsight-systems.com
    z/Debug debugs your Systems/C programs running on IBM z/OS!
    Are ISV upgrade fees too high? Check our custom product development!
    xarax, Mar 5, 2004
    #12
  13. Kutty Banerjee

    CBFalconer Guest

    Dan Pop wrote:
    > CBFalconer <> writes:
    >> Dan Pop wrote:
    >>> CBFalconer <> writes:
    >>>
    >>>> You yourself pointed out a reason for it - to provide alignment to
    >>>> larger values than does malloc.
    >>>
    >>> malloc provides maximally aligned blocks, *by definition*:
    >>>
    >>> The pointer returned if the allocation succeeds
    >>> is suitably aligned so that it may be assigned to a pointer to
    >>> any type of object and then used to access such an object or an
    >>> ^^^^^^^^^^^^^^^^^^
    >>> array of such objects in the space allocated (until the space
    >>> is explicitly deallocated).

    >>
    >> For the C language. There are entities in the world other than C
    >> language objects, and sometimes C code is used to organize and
    >> manipulate them.

    >
    > How is this possible, without treating them as C objects?


    By taking advantage of system specific knowledge, and accepting
    the non-portability of the result. Your system implementation of
    malloc, free, and realloc is almost certainly a case in point. My
    implementation of the same for DJGPP is certainly such. You can
    find it on my site if curious.

    --
    Chuck F () ()
    Available for consulting/temporary embedded and systems.
    <http://cbfalconer.home.att.net> USE worldnet address!
    CBFalconer, Mar 5, 2004
    #13
  14. Kutty Banerjee

    Chris Torek Guest

    >>> CBFalconer <> writes:
    >>>> You yourself pointed out a reason for it - to provide alignment to
    >>>> larger values than does malloc.


    >>Dan Pop wrote:
    >>> malloc provides maximally aligned blocks, *by definition*:

    [snippage]

    >In <> CBFalconer <> writes:
    >>For the C language. There are entities in the world other than C
    >>language objects, and sometimes C code is used to organize and
    >>manipulate them.


    In article <news:c2a98e$a5h$>
    Dan Pop <> writes:
    >How is this possible, without treating them as C objects?


    By faking it, of course.

    Typically one might build a struct or array that (by chance *and*
    purpose both) corresponds to the hardware's requirements, then use
    non-portable trickery to cause the existing mechanisms to produce
    values that work with this.

    The result is not guaranteed by the language, certainly, and often
    even the compiler-vendor may not guarantee it (other than internally
    if the vendor is also in the hardware or hardware-support business).
    But if it works, and is more maintainable than (e.g.) writing the
    entire thing in assembly, that may be enough to cause one to use it.

    The fact that a "struct page_table" (for instance) can only be
    allocated by the special "page table allocator", and:

    struct page_table tmp;
    ... use &tmp ...

    fails miserably, is relevant to someone who thinks the system is
    written in ANSI/ISO C, but not so important one who knows it is
    much more restricted than that. In other words, these non-C-objects
    *sometimes* work like real C objects, and sometimes do not, and we
    (I can speak for more than just myself here :) ) pay this price
    willingly for the value returned -- namely, some measure of
    convenience. When it stops working well, we change the code
    (and/or the compiler, in those cases where we control it).
    --
    In-Real-Life: Chris Torek, Wind River Systems
    Salt Lake City, UT, USA (40°39.22'N, 111°50.29'W) +1 801 277 2603
    email: forget about it http://web.torek.net/torek/index.html
    Reading email is like searching for food in the garbage, thanks to spammers.
    Chris Torek, Mar 5, 2004
    #14
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Regena
    Replies:
    0
    Views:
    378
    Regena
    Jan 19, 2006
  2. Ken
    Replies:
    24
    Views:
    3,857
    Ben Bacarisse
    Nov 30, 2006
  3. chris
    Replies:
    6
    Views:
    986
    chris
    Oct 28, 2005
  4. raj
    Replies:
    1
    Views:
    104
    mick white
    Jan 19, 2006
  5. Regena

    list box values to be alligned

    Regena, Jan 19, 2006, in forum: Javascript
    Replies:
    4
    Views:
    133
    Thomas 'PointedEars' Lahn
    Jan 19, 2006
Loading...

Share This Page