how to figure out the size of buffer returned by malloc

[pembed2003]

Hi,
If I have the folllowing:

char* p = malloc(5);
memset(p,-1,5);
*p = 0;
printf("%d\n",strlen(p));
free(p);

It will print 0. Is there a way to retrive the initial size of memory
returned by malloc? Other than using an extra variable to hold the
initial size like:

size_t i = 5;
char* p = malloc(i);
....

The initial requested size is then stored in i. Is there a
end-of-malloc-marker?

Thanks!

 

[Mark A. Odell]

(e-mail address removed) (pembed2003) wrote in

Hi,
If I have the folllowing:

char* p = malloc(5);
memset(p,-1,5);
*p = 0;
printf("%d\n",strlen(p));
free(p);

It will print 0. Is there a way to retrive the initial size of memory
returned by malloc? Other than using an extra variable to hold the
initial size like:

size_t i = 5;
char* p = malloc(i);

No. You must track it yourself.

The initial requested size is then stored in i. Is there a
end-of-malloc-marker?

Not that C knows about.

 

[Eric Sosman]

Hi,
If I have the folllowing:

char* p = malloc(5);
memset(p,-1,5);
*p = 0;
printf("%d\n",strlen(p));
free(p);

It will print 0. Is there a way to retrive the initial size of memory
returned by malloc? Other than using an extra variable to hold the
initial size like:

size_t i = 5;
char* p = malloc(i);
...

The initial requested size is then stored in i. Is there a
end-of-malloc-marker?

This is Question 7.27 in the comp.lang.c Frequently
Asked Questions (FAQ) list

http://www.eskimo.com/~scs/C-faq/top.html

See also Question 7.26.

 

[Mike Wahler]

Hi,
If I have the folllowing:

char* p = malloc(5);
memset(p,-1,5);
*p = 0;
printf("%d\n",strlen(p));

Should be:

printf("%lu\n", (unsigned long)strlen(p));

free(p);

It will print 0. Is there a way to retrive the initial size of memory
returned by malloc?

'malloc()' does not return the size of what it allocates,
but a pointer to it.

The only guarantee you have is that 'malloc()' (if it succeeds,
indicated by non-NULL return value), will allocate *at least* the
number of bytes requested. It's allowed to allocate more (but
you must not access more than you requested).

Other than using an extra variable to hold the
initial size like:

size_t i = 5;
char* p = malloc(i);
...

The initial requested size is then stored in i. Is there a
end-of-malloc-marker?

No. But you can easily 'simulate' one by adding one more
byte to your requested size, and storing some special 'sentinel'
value there, e.g.:

size_t size = 5;
char marker(0xFF);
char *p = malloc(size + 1);
*(p + size) = marker;

What are you trying to do?

-Mike

 

[Default User]

Hi,
If I have the folllowing:

char* p = malloc(5);
memset(p,-1,5);
*p = 0;
printf("%d\n",strlen(p));
free(p);

It will print 0.

That's good.

Is there a way to retrive the initial size of memory
returned by malloc?
http://www.eskimo.com/~scs/C-faq/q7.27.html


Other than using an extra variable to hold the initial size

That's pretty much the only way portably.



Brian Rodenborn

 

[Mantorok Redgormor]

Hi,
If I have the folllowing:

char* p = malloc(5);
memset(p,-1,5);
*p = 0;
printf("%d\n",strlen(p));
free(p);

It will print 0. Is there a way to retrive the initial size of memory
returned by malloc? Other than using an extra variable to hold the
initial size like:

size_t i = 5;
char* p = malloc(i);
...

The initial requested size is then stored in i. Is there a
end-of-malloc-marker?

Thanks!

You keep track of this in your program.
But implementations of malloc do provide ways
of figuring out the size allocated by malloc.
However, this is non-portable since standard C's
malloc does not mention of a way to do this.

For example though, with Doug Lea's malloc, you
can get the size by inspecting the address returned - 4
which contains the size allocated. Though this is off
topic.

 

[pembed2003]

Should be:

printf("%lu\n", (unsigned long)strlen(p));


'malloc()' does not return the size of what it allocates,
but a pointer to it.

The only guarantee you have is that 'malloc()' (if it succeeds,
indicated by non-NULL return value), will allocate *at least* the
number of bytes requested. It's allowed to allocate more (but
you must not access more than you requested).


No. But you can easily 'simulate' one by adding one more
byte to your requested size, and storing some special 'sentinel'
value there, e.g.:

size_t size = 5;
char marker(0xFF);
char *p = malloc(size + 1);
*(p + size) = marker;

What are you trying to do?

Thanks for everyone who answered my question. To answer Mike's
question:

I don't really have the "need" to do that yet but I am wondering how
does 'free' know how many bytes to free? For example:

char* s = malloc(10);
free(s); // how many bytes to free?

How does free know how many bytes to free? I have heard that malloc
also allocate additional memory to hold some header information. Where
is this header information stored? If we have access to the header, we
can find that out?

Thanks!

 

[Eric Sosman]

[...]
I don't really have the "need" to do that yet but I am wondering how
does 'free' know how many bytes to free? For example:

char* s = malloc(10);
free(s); // how many bytes to free?

How does free know how many bytes to free? I have heard that malloc
also allocate additional memory to hold some header information. Where
is this header information stored? If we have access to the header, we
can find that out?

The actual size of the memory pointed to by `s' is
etched in mystic runes on a tablet of finest jade, kept
in a diamond-encrusted box with silver hinges and golden
locks, hidden in a sacred cave in the Tibetan highlands
and guarded by a crack regiment of three hundred Abominable
Snowmen, each mounted upon a hippogriff with carnivorous
tendencies. Seven immortal unsleeping one-eyed sorcerers
stand upon the seven peaks that surround the Valley of the
Cave, each ready to hurl magical destruction on any who
venture near. The stream that flows from the valley teems
with invisible piranhas who eat only man-flesh, and is so
cold that if a single drop should touch your skin you would
instantly freeze to superconducting temperatures.

Or to put it another way, "There are some things Man
was not Meant To Know."

 

[Gordon Burditt]

Thanks for everyone who answered my question. To answer Mike's
question:

I don't really have the "need" to do that yet but I am wondering how
does 'free' know how many bytes to free?

It knows. It doesn't have to tell you HOW it knows, it just does.

For example:

char* s = malloc(10);
free(s); // how many bytes to free?

How does free know how many bytes to free? I have heard that malloc
also allocate additional memory to hold some header information. Where
is this header information stored? If we have access to the header, we
can find that out?

The information might be stored in a large SQL database in a
sub-dungeon of Microsoft Corporation, where you will be taxed on
the memory you use (and more on the memory you don't give back).
The ANSI C standard doesn't say HOW, it just says to DO IT.

There are various trade-offs of speed vs. memory utilization
efficiency (aka fragmentation). For example, some implementations
add a little overhead, then round the result up to the next power
of two, so if you ask for a block of 4096, you might end up using
8192. Some implementations keep a linked list of free and allocated
memory, using various unportable mechanisms such as using the
low-order bit of a pointer to indicate whether the block is free
or not. Linked lists can get VERY slow if you allocate lots of
really tiny strings.

The information might be kept in a table somewhere, not at the
beginning and/or end of the allocated memory block.

Gordon L. Burditt

 

[John Cochran]

SNIP...

I don't really have the "need" to do that yet but I am wondering how
does 'free' know how many bytes to free? For example:

char* s = malloc(10);
free(s); // how many bytes to free?

How does free know how many bytes to free? I have heard that malloc
also allocate additional memory to hold some header information. Where
is this header information stored? If we have access to the header, we
can find that out?

Thanks!

The answer to that question depends on the implementation of the malloc(),
realloc(), et. all package. Two common implementations are:

1. Have malloc allocate slightly more data than requested and place the
length at the front of the memory block. The pointer that malloc
returns is to the byte just following the length. This method has the
advantage of being very simple to do. However in a virtual memory
system, it frequently requires a page in request in order for free()
to determine the size of the memory block just freed. A rather silly
thing to do if you're not going to modify or otherwise use the page
of memory just read in.

2. Have a "index" page of memory with page addresses and allocation sizes
for those pages. Keep track of which section are allocated using a
bit array on the index page. For example, FreeBSD gets memory in 4K byte
pages. Each page is then used for only allocations of a specified size.
You may have a page that is allocated 16 bytes at a time and used for
any memory request less than or equal to 16 bytes. Another page is
used 32 bytes at a time and handles requests for 17 to 32 bytes. This
pattern continues for 64, 128, 256, ... 4096 bytes. For requests over
4096 bytes, several pages are chained together. When freeing a chunk
of allocated memory, the index page simply needs to have a bit set or
reset so if the allocated memory is paged out, it doesn't need to be
paged back in. Since the index page is frequently accessed, it typically
remains active and isn't paged out.

I am certain that you can think of many more methods of keeping track of
allocated memory. I am also certain that attempting to use internal
implementation information about malloc() and company is the wrong thing to
do and is likely to change from system to system.

 

[Alan Balmer]

I don't really have the "need" to do that yet but I am wondering how
does 'free' know how many bytes to free? For example:

char* s = malloc(10);
free(s); // how many bytes to free?

It's a secret ;-) The implementation knows, but you don't, unless
you're writing the implementation. What's more, you don't want to know
- acting on such knowledge can only get you in trouble.

How does free know how many bytes to free? I have heard that malloc
also allocate additional memory to hold some header information. Where
is this header information stored? If we have access to the header, we
can find that out?

Not portably. The header is quite likely to be different from compiler
to compiler, platform to platform and version to version. If you use
such knowledge even for a particular program on a particular
implementation on a particular machine, it may break if you upgrade
the compiler or the operating system (since runtime libraries may be
shipped with the OS.)