how to figure out the size of buffer from malloc

Discussion in 'C Programming' started by pembed2012, Feb 16, 2012.

  1. pembed2012

    pembed2012 Guest

    I am wondering how does 'free' know how many bytes to free? For example:

    char* s = (char*)malloc(10);
    free(s); // how many bytes to free?

    How does free know how many bytes to free? I have heard that malloc
    also allocate additional memory to hold some header information. Where
    is this header information stored? If we have access to the header, we
    can find that out?

    Thanks!
     
    pembed2012, Feb 16, 2012
    #1
    1. Advertising

  2. pembed2012

    James Kuyper Guest

    On 02/16/2012 03:21 PM, pembed2012 wrote:
    > I am wondering how does 'free' know how many bytes to free? For example:
    >
    > char* s = (char*)malloc(10);
    > free(s); // how many bytes to free?
    >
    > How does free know how many bytes to free? I have heard that malloc
    > also allocate additional memory to hold some header information. Where
    > is this header information stored? If we have access to the header, we
    > can find that out?


    Howe that information is stored is an implementation detail, likely to
    be different on different implementations. As a result, there's no
    portable way you can obtain that information.

    One common technique is to have malloc() allocate a little bit more than
    the requested amount of memory, and to store the size of the allocation
    in that extra piece of memory. This extra memory is usually stored at a
    fixed negative offset from the pointer that you get from malloc().

    Another technique is round each allocation request up to a power of 2,
    and to maintain a separate pool of memory for each power of 2. By
    looking at the address, free() can tell which pool it came from, and
    therefore, how much memory to free.

    There's several other, more sophisticated techniques that are in actual
    use, but I hope that those two are sufficient for you to understand why
    there's no portable way to get this information.
    --
    James Kuyper
     
    James Kuyper, Feb 16, 2012
    #2
    1. Advertising

  3. pembed2012

    John Gordon Guest

    In <jhjofu$jer$> pembed2012 <> writes:

    > I am wondering how does 'free' know how many bytes to free? For example:


    > char* s = (char*)malloc(10);
    > free(s); // how many bytes to free?


    A common method is for malloc to allocate a few extra bytes, store the
    size in the extra space at the beginning, and return a pointer to the
    remaining area.

    In your sample code above, malloc might actually allocate fourteen bytes,
    store the size information in the first four bytes and return a pointer
    to the remaining ten bytes.

    Then when free is called, it knows it can back up four bytes from the
    given address to find the size of the area to be freed.

    > How does free know how many bytes to free? I have heard that malloc
    > also allocate additional memory to hold some header information. Where
    > is this header information stored? If we have access to the header, we
    > can find that out?


    It's all compiler- and library-specific. The language itself doesn't
    specify how it is done.

    --
    John Gordon A is for Amy, who fell down the stairs
    B is for Basil, assaulted by bears
    -- Edward Gorey, "The Gashlycrumb Tinies"
     
    John Gordon, Feb 16, 2012
    #3
  4. pembed2012

    Lew Pitcher Guest

    On Thursday 16 February 2012 15:21, in comp.lang.c, d
    wrote:

    > I am wondering how does 'free' know how many bytes to free?


    As others have said, malloc() and free() are part of the language
    definition, and /how/ they work is considered to be "magic" as far as we
    mortal userspace programmers go.

    OTOH, you can get a feel for what sort of code /might/ go into malloc() and
    free() by checking out K&R "The C Programming Language". The original K&R
    (and, I believe, the K&R rewritten for C89) has a chapter called "The Unix
    System Interface", in which an example "Storage Allocator" is detailed.

    HTH
    --
    Lew Pitcher
     
    Lew Pitcher, Feb 16, 2012
    #4
  5. pembed2012 <> writes:
    > I am wondering how does 'free' know how many bytes to free? For example:
    >
    > char* s = (char*)malloc(10);


    The cast is unnecessary and can hide errors.

    char *s = malloc(10);

    Note that this can fail; you should always check whether malloc()
    returned a null pointer.

    > free(s); // how many bytes to free?
    >
    > How does free know how many bytes to free? I have heard that malloc
    > also allocate additional memory to hold some header information. Where
    > is this header information stored? If we have access to the header, we
    > can find that out?


    The comp.lang.c FAQ is at <http://c-faq.com/>. You've just asked
    question 7.26.

    --
    Keith Thompson (The_Other_Keith) <http://www.ghoti.net/~kst>
    Will write code for food.
    "We must do something. This is something. Therefore, we must do this."
    -- Antony Jay and Jonathan Lynn, "Yes Minister"
     
    Keith Thompson, Feb 16, 2012
    #5
  6. pembed2012

    pembed2012 Guest

    On Thu, 16 Feb 2012 20:48:06 +0000, John Gordon wrote:

    > In <jhjofu$jer$> pembed2012 <>
    > writes:
    >
    >> I am wondering how does 'free' know how many bytes to free? For
    >> example:

    >
    >> char* s = (char*)malloc(10);
    >> free(s); // how many bytes to free?

    >
    > A common method is for malloc to allocate a few extra bytes, store the
    > size in the extra space at the beginning, and return a pointer to the
    > remaining area.
    >
    > In your sample code above, malloc might actually allocate fourteen
    > bytes, store the size information in the first four bytes and return a
    > pointer to the remaining ten bytes.
    >
    > Then when free is called, it knows it can back up four bytes from the
    > given address to find the size of the area to be freed.
    >
    >> How does free know how many bytes to free? I have heard that malloc
    >> also allocate additional memory to hold some header information. Where
    >> is this header information stored? If we have access to the header, we
    >> can find that out?

    >
    > It's all compiler- and library-specific. The language itself doesn't
    > specify how it is done.


    Thanks, I've tried this but it does NOT work :(

    This code prints: 0, it should print: 10

    int get_malloc_size(void *vp)
    {
    char *cp = (char*)vp;
    static int __done = -1;
    static int __offset = -1;
    if(__done != 0) {
    char *p;
    int n,i;
    while(1) {
    n=random()%10000;
    p=(char*)malloc(n);
    for(i=1;i<100;i++) {
    if(*(int*)(p-i)==n) {
    __offset=i;
    break;
    }
    }
    free(p);
    if(__offset != -1)
    break;
    }
    __done=0;
    }
    return *(int*)(cp-__offset);
    }

    main()
    {
    char *p=(char*)malloc(10);
    printf("%d\n", get_malloc_size((void*)p));
    free(p);
    }
     
    pembed2012, Feb 16, 2012
    #6
  7. pembed2012

    John Gordon Guest

    In <jhk0oi$8cu$> pembed2012 <> writes:

    > Thanks, I've tried this but it does NOT work :(


    I suggested one possible way that malloc might store the size of the
    allocated area. There are lots of different ways this can be done. Your
    malloc implementation must use a different mechanism.

    I thought I was clear when I said:

    > > It's all compiler- and library-specific. The language itself doesn't
    > > specify how it is done.


    If you want to know how it's done on your system, you'll have to look
    at the system documentation.

    --
    John Gordon A is for Amy, who fell down the stairs
    B is for Basil, assaulted by bears
    -- Edward Gorey, "The Gashlycrumb Tinies"
     
    John Gordon, Feb 16, 2012
    #7
  8. pembed2012 <> writes:
    > On Thu, 16 Feb 2012 20:48:06 +0000, John Gordon wrote:
    >
    >> In <jhjofu$jer$> pembed2012 <>
    >> writes:

    [...]
    >>> How does free know how many bytes to free? I have heard that malloc
    >>> also allocate additional memory to hold some header information. Where
    >>> is this header information stored? If we have access to the header, we
    >>> can find that out?

    >>
    >> It's all compiler- and library-specific. The language itself doesn't
    >> specify how it is done.

    >
    > Thanks, I've tried this but it does NOT work :(
    >
    > This code prints: 0, it should print: 10
    >
    > int get_malloc_size(void *vp)
    > {
    > char *cp = (char*)vp;
    > static int __done = -1;
    > static int __offset = -1;


    Identifiers starting with two underscores are reserved to the
    implementation. You should *never* define such identifiers in
    your own code. The rules are a little more complex for identifiers
    starting with a single underscore but you should still avoid them.

    [snip]

    > return *(int*)(cp-__offset);
    > }
    >
    > main()


    Should be "int main(void)".

    > {
    > char *p=(char*)malloc(10);
    > printf("%d\n", get_malloc_size((void*)p));
    > free(p);
    > }


    Why would you expect it to work? There is *no* portable way to
    determine the size of a malloc()ed chunk of memory.

    Actually, there is: just remember the argument you passed to malloc():

    size_t requested_size = 10;
    char *p = malloc(requested_size);
    if (p != NULL) {
    /* You know you've allocated requested_size bytes. */
    }

    Note that the implementation could allocate more memory than you
    request; it won't necessarily store the requested size anywhere.
    But that shouldn't matter to your program.

    If you think it does matter, explain what you're trying to accomplish
    and we can probably help you find another way.

    --
    Keith Thompson (The_Other_Keith) <http://www.ghoti.net/~kst>
    Will write code for food.
    "We must do something. This is something. Therefore, we must do this."
    -- Antony Jay and Jonathan Lynn, "Yes Minister"
     
    Keith Thompson, Feb 16, 2012
    #8
  9. pembed2012

    Shao Miller Guest

    On 2/16/2012 17:42, pembed2012 wrote:
    > On Thu, 16 Feb 2012 20:48:06 +0000, John Gordon wrote:
    >
    >> In<jhjofu$jer$> pembed2012<>
    >> writes:
    >>
    >>> I am wondering how does 'free' know how many bytes to free? For
    >>> example:

    >>
    >>> char* s = (char*)malloc(10);
    >>> free(s); // how many bytes to free?

    >>
    >> A common method is for malloc to allocate a few extra bytes, store the
    >> size in the extra space at the beginning, and return a pointer to the
    >> remaining area.
    >>
    >> In your sample code above, malloc might actually allocate fourteen
    >> bytes, store the size information in the first four bytes and return a
    >> pointer to the remaining ten bytes.
    >>
    >> Then when free is called, it knows it can back up four bytes from the
    >> given address to find the size of the area to be freed.
    >>
    >>> How does free know how many bytes to free? I have heard that malloc
    >>> also allocate additional memory to hold some header information. Where
    >>> is this header information stored? If we have access to the header, we
    >>> can find that out?

    >>
    >> It's all compiler- and library-specific. The language itself doesn't
    >> specify how it is done.

    >
    > Thanks, I've tried this but it does NOT work :(
    >


    You can make your own allocator and freer and use them instead of
    'malloc' and 'free'. In fact, you could even do:

    #include <standard_stuff1.h>
    #include <standard_stuff2.h>

    #include <system_stuff.h>

    #include "my_stuff.h"

    #define malloc my_malloc
    #define free my_free

    Then you could use 'malloc' and 'free' but the result would be calls to
    your functions instead of the implementation's.

    Here's an example alternative to using 'malloc' that I haven't looked at
    for a while:

    /**
    * Delete all lines containing
    * 'ALL_IN_ONE' if you split this example
    * into separate files.
    */
    #define ALL_IN_ONE 1


    /****
    * pfxalloc.h
    *
    * It took several minutes to find
    * a free *alloc. (No pun intended.)
    */
    #ifndef PFXALLOC_H_

    /* For 'offsetof' and 'size_t' */
    #include <stddef.h>

    /*** Macros */
    #define PFXALLOC_H_

    #ifndef alignof

    /* Derived from Mr. Chris M. Thomasson */
    #define alignof(type) \
    (offsetof(struct {char c; type t;}, t))

    #ifdef __alignof_is_defined
    #undef __alignof_is_defined
    #endif /* __alignof_is_defined */

    #define __alignof_is_defined 1
    #endif /* alignof */

    /*** Functions */

    /**
    * Allocate memory for an object and an
    * associated "prefix" object.
    *
    * @v pfx_size The "prefix" size.
    * @v obj_alignment The object's alignment
    * requirement.
    * @v obj_size The object's size.
    * @ret void * Points to the object,
    * or is NULL for failure.
    */
    extern void * pfxalloc(size_t, size_t, size_t);

    /**
    * Get an object's associated "prefix" object.
    *
    * @v obj The object to fetch the
    * associated "prefix" for.
    * @ret void * Points to the "prefix",
    * or is NULL for failure.
    */
    extern void * pfxget(void *);

    #endif /* PFXALLOC_H_ */


    /**** pfxalloc.c */

    /* For 'malloc' */
    #include <stdlib.h>
    #if !ALL_IN_ONE
    /* For 'size_t', 'ptrdiff_t' and 'NULL' */
    #include <stddef.h>
    /* For 'alignof' */
    #include "pfxalloc.h"
    #endif /* !ALL_IN_ONE */

    /*** Constants */
    enum cv {
    cv_ptrdiff_alignment =
    alignof(ptrdiff_t),
    cv_zero = 0
    };

    /*** Functions */

    /**
    * The layout looks like:
    * +-----+-------------+--------+--------+
    * | pfx | opt_padding | offset | |
    * +- -+- -+- -+ object |
    * | header | |
    * +----------------------------+--------+
    */
    void * pfxalloc(
    size_t pfx_size,
    size_t obj_alignment,
    size_t obj_size
    ) {
    size_t offset_at;
    ptrdiff_t * offset;
    size_t header_size;
    size_t total_size;
    unsigned char * mem;
    unsigned char * obj;

    /* Sanity-check alignment value */
    if (obj_alignment & (obj_alignment - 1))
    return NULL;

    /* Calculate the offset position */
    offset_at = pfx_size;
    offset_at += sizeof *offset;
    /* Check for wrap-around */
    if (offset_at < pfx_size)
    return NULL;
    --offset_at;
    offset_at /= sizeof *offset;

    /* Calculate the header size */
    header_size = (offset_at + 1) * sizeof *offset;
    /* Check for wrap-around */
    if (header_size < pfx_size)
    return NULL;

    /* Calculate padding */
    if (obj_alignment > cv_ptrdiff_alignment) {
    size_t new_hdr_size = header_size;
    new_hdr_size += obj_alignment;
    /* Check for wrap-around */
    if (new_hdr_size < header_size)
    return NULL;
    --new_hdr_size;
    new_hdr_size /= obj_alignment;
    new_hdr_size *= obj_alignment;
    header_size = new_hdr_size;
    }

    /* Allocate storage */
    total_size = header_size + obj_size;
    /* Check for wrap-around */
    if (total_size < pfx_size || total_size < obj_size)
    return NULL;
    mem = malloc(total_size);
    if (!mem)
    return mem;

    /* Point to the object */
    obj = mem + header_size;

    /* Note the offset to the prefix */
    offset = (ptrdiff_t *) obj - 1;
    *offset = obj - mem;

    return obj;
    }

    /* Fetch an asociated prefix for an object */
    void * pfxget(void * obj) {
    ptrdiff_t * offset;
    unsigned char * prefix;

    if (!obj)
    return obj;

    offset = obj;
    --offset;
    prefix = obj;
    prefix -= *offset;

    return prefix;
    }

    (It could probably be improved.)
     
    Shao Miller, Feb 17, 2012
    #9
  10. pembed2012

    Mark Adler Guest

    If you're not too worried about portability, you can use
    malloc_usable_size(), malloc_size(), or _msize(), depending on the
    system. The value returned is the actual size of the block, which may
    be a bit larger than what was requested.

    Mark
     
    Mark Adler, Feb 17, 2012
    #10
  11. pembed2012

    pembed2012 Guest

    On Thu, 16 Feb 2012 14:50:13 -0800, Keith Thompson wrote:

    > pembed2012 <> writes:
    >> On Thu, 16 Feb 2012 20:48:06 +0000, John Gordon wrote:
    >>
    >>> In <jhjofu$jer$> pembed2012
    >>> <> writes:

    > [...]
    >>>> How does free know how many bytes to free? I have heard that malloc
    >>>> also allocate additional memory to hold some header information.
    >>>> Where is this header information stored? If we have access to the
    >>>> header, we can find that out?
    >>>
    >>> It's all compiler- and library-specific. The language itself doesn't
    >>> specify how it is done.

    >>
    >> Thanks, I've tried this but it does NOT work :(
    >>
    >> This code prints: 0, it should print: 10
    >>
    >> int get_malloc_size(void *vp)
    >> {
    >> char *cp = (char*)vp;
    >> static int __done = -1;
    >> static int __offset = -1;

    >
    > Identifiers starting with two underscores are reserved to the
    > implementation. You should *never* define such identifiers in your own
    > code. The rules are a little more complex for identifiers starting with
    > a single underscore but you should still avoid them.
    >
    > [snip]
    >
    >> return *(int*)(cp-__offset);
    >> }
    >>
    >> main()

    >
    > Should be "int main(void)".
    >
    >> {
    >> char *p=(char*)malloc(10);
    >> printf("%d\n", get_malloc_size((void*)p)); free(p);
    >> }

    >
    > Why would you expect it to work? There is *no* portable way to
    > determine the size of a malloc()ed chunk of memory.
    >
    > Actually, there is: just remember the argument you passed to malloc():
    >
    > size_t requested_size = 10;
    > char *p = malloc(requested_size);
    > if (p != NULL) {
    > /* You know you've allocated requested_size bytes. */
    > }
    >
    > Note that the implementation could allocate more memory than you
    > request; it won't necessarily store the requested size anywhere. But
    > that shouldn't matter to your program.
    >
    > If you think it does matter, explain what you're trying to accomplish
    > and we can probably help you find another way.


    Sorry it wasn't clear, I need to take a pointer from malloc and find out
    the size of the allocation. To allow for dynamic pointers as well as
    stack arrays. Like this:

    myfunc(void* buf){
    int bufsiz=-1;
    if(sizeof(buf)==sizeof(void*)) { // pointer
    bufsiz=get_malloc_size(buf)'
    }
    if(bufsize==-1){ // array
    bufsiz=sizeof(buf);
    }
    // .... process buffer ...
    }

    Fully portable is best, but a solution working on 90% of compilers would
    be OK if I understood right for you say it can't be done 100%
     
    pembed2012, Feb 17, 2012
    #11
  12. pembed2012

    Ike Naar Guest

    On 2012-02-17, pembed2012 <> wrote:
    > myfunc(void* buf){
    > int bufsiz=-1;
    > if(sizeof(buf)==sizeof(void*)) { // pointer


    Since the type of buf is void*, the condition is always true.

    > bufsiz=get_malloc_size(buf)'
    > }
    > if(bufsize==-1){ // array
    > bufsiz=sizeof(buf);
    > }
    > // .... process buffer ...
    > }
    >
    > Fully portable is best, but a solution working on 90% of compilers would
    > be OK if I understood right for you say it can't be done 100%


    One method that works 100% is to remember the size of the buffer
    in a variable, and pass that information to myfunc along with the
    pointer to the buffer.

    #include <stdlib.h>

    void myfunc(void *buf, size_t bufsize);

    void using_pointer(void)
    {
    size_t size = 684;
    char *pointer = malloc(size);
    if (pointer != NULL)
    {
    myfunc(pointer, size);
    }
    }

    void using_array(void)
    {
    char array[684];
    myfunc(array, sizeof array);
    }
     
    Ike Naar, Feb 17, 2012
    #12
  13. pembed2012

    James Kuyper Guest

    On 02/17/2012 02:45 PM, pembed2012 wrote:
    ....
    > Sorry it wasn't clear, I need to take a pointer from malloc and find out
    > the size of the allocation. To allow for dynamic pointers as well as
    > stack arrays. Like this:
    >
    > myfunc(void* buf){
    > int bufsiz=-1;


    It's absolutely guaranteed, at this point, that sizeof(buf) ==
    sizeof(void*). As a result, 2 of your next three lines are pointless,
    which suggests that you were not aware of that fact.

    > if(sizeof(buf)==sizeof(void*)) { // pointer
    > bufsiz=get_malloc_size(buf)'
    > }


    Why did you think it was possible for the condition of that if() to not
    be met?
    --
    James Kuyper
     
    James Kuyper, Feb 17, 2012
    #13
  14. pembed2012

    pembed2012 Guest

    On Fri, 17 Feb 2012 15:33:58 -0500, James Kuyper wrote:
    > On 02/17/2012 02:45 PM, pembed2012 wrote: ...
    >> Sorry it wasn't clear, I need to take a pointer from malloc and find
    >> out the size of the allocation. To allow for dynamic pointers as well
    >> as stack arrays. Like this:
    >>
    >> myfunc(void* buf){
    >> int bufsiz=-1;

    >
    > It's absolutely guaranteed, at this point, that sizeof(buf) ==
    > sizeof(void*). As a result, 2 of your next three lines are pointless,
    > which suggests that you were not aware of that fact.
    >
    >> if(sizeof(buf)==sizeof(void*)) { // pointer
    >> bufsiz=get_malloc_size(buf)'
    >> }

    >
    > Why did you think it was possible for the condition of that if() to not
    > be met?


    Often this function will be called with a dynamic pointer from malloc.
    However, it is also possible to call it with a stack array:

    char ary[10];
    char* ptr=malloc(10);
    myfunc((void*)ary); // should work
    myfunc((void*)ptr); // should also work

    It started off with the buffers being NULL-free strings. However now I
    need to work with strings with embedded NULL characters. This means I
    can't find the length of the buffer with strlen. And it would be too much
    change to add a new argument to pass the length of the buffer. So the
    only good solution is for myfunc to work out by it's self 1) if the
    buffer is a dynamic pointer or stack array and 2) the size of the buffer.
     
    pembed2012, Feb 17, 2012
    #14
  15. pembed2012

    James Kuyper Guest

    On 02/17/2012 05:13 PM, pembed2012 wrote:
    > On Fri, 17 Feb 2012 15:33:58 -0500, James Kuyper wrote:
    >> On 02/17/2012 02:45 PM, pembed2012 wrote: ...
    >>> Sorry it wasn't clear, I need to take a pointer from malloc and find
    >>> out the size of the allocation. To allow for dynamic pointers as well
    >>> as stack arrays. Like this:
    >>>
    >>> myfunc(void* buf){
    >>> int bufsiz=-1;

    >>
    >> It's absolutely guaranteed, at this point, that sizeof(buf) ==
    >> sizeof(void*). As a result, 2 of your next three lines are pointless,
    >> which suggests that you were not aware of that fact.
    >>
    >>> if(sizeof(buf)==sizeof(void*)) { // pointer
    >>> bufsiz=get_malloc_size(buf)'
    >>> }

    >>
    >> Why did you think it was possible for the condition of that if() to not
    >> be met?

    >
    > Often this function will be called with a dynamic pointer from malloc.
    > However, it is also possible to call it with a stack array:
    >
    > char ary[10];
    > char* ptr=malloc(10);
    > myfunc((void*)ary); // should work
    > myfunc((void*)ptr); // should also work


    Assuming that the function's declaration matches the definition you've
    given above, the casts are unnecessary in your code above; they specify
    the same conversion that will happen anyway even if they weren't present.

    Experienced programmers treat every occurrence of a cast as a potential
    time bomb that needs to be examined carefully to make sure it's not an
    actual bomb. While your code has mistakes, that's not one of them, so
    you're better off removing those unnecessary casts.

    Again, why do you think sizeof(buf)==sizeof(void*) will be false for one
    of those two cases (presumably the first one)?

    Incidentally, it's the wrong test even if it could work the way you were
    expecting it to do. Consider:

    char arr[sizeof(void*)];

    > It started off with the buffers being NULL-free strings. However now I
    > need to work with strings with embedded NULL characters. This means I
    > can't find the length of the buffer with strlen. And it would be too much
    > change to add a new argument to pass the length of the buffer. So the
    > only good solution is for myfunc to work out by it's self 1) if the
    > buffer is a dynamic pointer or stack array and 2) the size of the buffer.


    Unfortunately, it's not possible for your function to figure out either
    of those things in the way that you want it to. Your best solution is to
    make the changes needed to add a new argument containing the length of
    the buffer.
    --
    James Kuyper
     
    James Kuyper, Feb 17, 2012
    #15
  16. pembed2012

    Shao Miller Guest

    On 2/17/2012 14:45, pembed2012 wrote:
    >
    > Sorry it wasn't clear, I need to take a pointer from malloc and find out
    > the size of the allocation. To allow for dynamic pointers as well as
    > stack arrays. Like this:
    >
    > myfunc(void* buf){
    > int bufsiz=-1;

    /* The following condition is always true */
    > if(sizeof(buf)==sizeof(void*)) { // pointer

    /* Oops. Semi-colon needed instead of ' */
    > bufsiz=get_malloc_size(buf)'
    > }

    /* Oops. 'bufsiz', not 'bufsize' */
    > if(bufsize==-1){ // array

    /* This will never happen */
    > bufsiz=sizeof(buf);
    > }
    > // .... process buffer ...
    > }
    >
    > Fully portable is best, but a solution working on 90% of compilers would
    > be OK if I understood right for you say it can't be done 100%


    I don't think it matters what kind of pointer-to-object type you
    originally pass to 'myfunc'. 'buf' is a 'void *' and 'sizeof buf'
    should be equal to 'sizeof (void *)'.

    Did you understand my example allocator that I posted? It ought to be
    at least 90% portable.
     
    Shao Miller, Feb 17, 2012
    #16
  17. pembed2012

    Shao Miller Guest

    On 2/17/2012 17:13, pembed2012 wrote:
    > On Fri, 17 Feb 2012 15:33:58 -0500, James Kuyper wrote:
    >> On 02/17/2012 02:45 PM, pembed2012 wrote: ...
    >>> Sorry it wasn't clear, I need to take a pointer from malloc and find
    >>> out the size of the allocation. To allow for dynamic pointers as well
    >>> as stack arrays. Like this:
    >>>
    >>> myfunc(void* buf){
    >>> int bufsiz=-1;

    >>
    >> It's absolutely guaranteed, at this point, that sizeof(buf) ==
    >> sizeof(void*). As a result, 2 of your next three lines are pointless,
    >> which suggests that you were not aware of that fact.
    >>
    >>> if(sizeof(buf)==sizeof(void*)) { // pointer
    >>> bufsiz=get_malloc_size(buf)'
    >>> }

    >>
    >> Why did you think it was possible for the condition of that if() to not
    >> be met?

    >
    > Often this function will be called with a dynamic pointer from malloc.
    > However, it is also possible to call it with a stack array:
    >
    > char ary[10];
    > char* ptr=malloc(10);
    > myfunc((void*)ary); // should work
    > myfunc((void*)ptr); // should also work
    >
    > It started off with the buffers being NULL-free strings. However now I
    > need to work with strings with embedded NULL characters. This means I
    > can't find the length of the buffer with strlen. And it would be too much
    > change to add a new argument to pass the length of the buffer. So the
    > only good solution is for myfunc to work out by it's self 1) if the
    > buffer is a dynamic pointer or stack array and 2) the size of the buffer.


    I don't think a string has 'NULL' in it. I think a string can have a
    null terminator. I think that 'NULL' is an invalid pointer value.

    I think you are mistaken about what you are doing and expecting.

    It looks like 'ary' has type 'char[10]'. When 'ary' is used in your
    examples, it yields a 'char *' value. All pointer-to-object types can
    be implicitly converted to 'void *', so your cast is redundant.

    It looks like the type of 'ptr' is 'char *'. All pointer-to-object
    types can be implicitly converted to 'void *', so your cast is redundant.

    In C, if you have a pointer into some object (single object, array
    object, "stack," allocated, static-duration; no difference), I do not
    believe that you can expect that a pointer value carries size
    information nor that even if it does, that you can access that information.
     
    Shao Miller, Feb 17, 2012
    #17
  18. pembed2012 <> writes:
    [...]
    > Often this function will be called with a dynamic pointer from malloc.
    > However, it is also possible to call it with a stack array:
    >
    > char ary[10];
    > char* ptr=malloc(10);
    > myfunc((void*)ary); // should work
    > myfunc((void*)ptr); // should also work


    Not possible. It's not just not 100% portable, it's not possible in any
    implementation I know of. You'll need to find a way to keep track of
    the size yourself.

    > It started off with the buffers being NULL-free strings. However now I
    > need to work with strings with embedded NULL characters.


    NULL is (a macro that expands to) a null *pointer* constant. You're
    talking about null *characters*. I know that some character set
    standards use the term "NULL", but you shouldn't use that term to refer
    to null characters in the context of C.

    A "string" in C is, by definition, "a contiguous sequence of characters
    terminated by and including the first null character", so there's no
    such thing as a "null-free string". A character array that doesn't
    contain any null characters is not a string; it's simply a character
    array.

    > This means I
    > can't find the length of the buffer with strlen.


    strlen doesn't give you the size of a buffer; it searches for the
    first null character. You need to decide whether you want the
    length of a string contained in a buffer, or the size of the buffer
    itself, and you can't reasonably decide that based on its content.
    If the buffer holds arbitrary data, it might still contain a null
    character that's not meant to be a string terminator; what should
    your function return in that case, and how can it decide?

    > And it would be too much
    > change to add a new argument to pass the length of the buffer. So the
    > only good solution is for myfunc to work out by it's self 1) if the
    > buffer is a dynamic pointer or stack array and 2) the size of the buffer.


    Then I'm afraid you've got "too much change" ahead of you -- unless
    perhaps you can reformulate the problem somehow. (We don't know *why*
    you need this functionality.)

    --
    Keith Thompson (The_Other_Keith) <http://www.ghoti.net/~kst>
    Will write code for food.
    "We must do something. This is something. Therefore, we must do this."
    -- Antony Jay and Jonathan Lynn, "Yes Minister"
     
    Keith Thompson, Feb 17, 2012
    #18
  19. pembed2012

    Shao Miller Guest

    On 2/17/2012 18:01, Keith Thompson wrote:
    >> And it would be too much
    >> change to add a new argument to pass the length of the buffer. So the
    >> only good solution is for myfunc to work out by it's self 1) if the
    >> buffer is a dynamic pointer or stack array and 2) the size of the buffer.

    >
    > Then I'm afraid you've got "too much change" ahead of you -- unless
    > perhaps you can reformulate the problem somehow. (We don't know *why*
    > you need this functionality.)
    >


    Or the OP could wrap the *alloc functions and 'free' so that the size is
    tracked. They could even #define over the *alloc functions and 'free'.
    Then they could also implement their 'alloc_size' function or whatever
    it was called.

    Of course, doing that doesn't help with the misunderstanding that the
    size of an _array_ is somehow available from a pointer _value_.
     
    Shao Miller, Feb 17, 2012
    #19
  20. pembed2012

    Ian Pilcher Guest

    On 02/17/2012 04:13 PM, pembed2012 wrote:
    > It started off with the buffers being NULL-free strings. However now I
    > need to work with strings with embedded NULL characters. This means I
    > can't find the length of the buffer with strlen. And it would be too much
    > change to add a new argument to pass the length of the buffer. So the
    > only good solution is for myfunc to work out by it's self 1) if the
    > buffer is a dynamic pointer or stack array and 2) the size of the buffer.


    Prepare to be disappointed.

    --
    ========================================================================
    Ian Pilcher
    "If you're going to shift my paradigm ... at least buy me dinner first."
    ========================================================================
     
    Ian Pilcher, Feb 18, 2012
    #20
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Skyhorse
    Replies:
    4
    Views:
    4,241
    EventHelix.com
    Apr 16, 2004
  2. Raja
    Replies:
    12
    Views:
    24,546
    John Harrison
    Jun 21, 2004
  3. pembed2003
    Replies:
    10
    Views:
    543
    Alan Balmer
    Jun 24, 2004
  4. Replies:
    2
    Views:
    3,300
    Ron Natalie
    Aug 11, 2006
  5. James Harris
    Replies:
    26
    Views:
    782
    santosh
    Jul 29, 2008
Loading...

Share This Page