malloc and free

[Joris Adriaenssens]

This is my first posting, please excuse me if it is off-topic.

I'm learning to program in C. It's been almost ten years I've been
programming and a lot of things have changed apparently.

I understand from other postings that casting a result from malloc
isn't good. In the past I have always been casting the malloc. I
think it was even necessary. (But that's a long time ago, I hadn't
heard of a standard for C these days). Was it really necessary or did
we do it only to silence the warnings ? (cfr. FAQ q7.7 - I can't
look in the manuals from these days : I don't have them any longer)


When I use malloc for the allocation of a small piece of memory, e.g.
a single node of a linked list, I use 'free' to release the memory
afterwards. When using 'free', I don't have to worry about the size
of the allocated block the pointer is pointing to.
Is it the operating-system that is responsible to remember the size of
that piece of memory ?
In that case, when I build a large linked-list by allocating memory
for each node, I get a lot of overhead in the operating system. Is
this right ? or is this implementation-specific or operating-system
specific ? (FAQ q7.26 says that the implementation 'remembers' the
size. Does this mean that every implementation may do it in his own
way ? )
If the computer has to remember for each single allocation the size of
the allocated block, is it better than, to allocate a bigger piece of
memory and keep track of the use of it within the program, when I try
to create a linked-list or a binary tree or something like that ?

Thank you.
Joris Adriaenssens

 

[Barry Schwarz]

This is my first posting, please excuse me if it is off-topic.

I'm learning to program in C. It's been almost ten years I've been
programming and a lot of things have changed apparently.

I understand from other postings that casting a result from malloc
isn't good. In the past I have always been casting the malloc. I
think it was even necessary. (But that's a long time ago, I hadn't
heard of a standard for C these days). Was it really necessary or did
we do it only to silence the warnings ? (cfr. FAQ q7.7 - I can't
look in the manuals from these days : I don't have them any longer)

Things change over time. The cast became unnecessary when malloc
started returning void*.

Using a cast to silence a warning is a common but not foolproof
technique. The compiler is free to give any (superfluous) diagnostic
it likes as long as it correctly compiles a conforming program.

When I use malloc for the allocation of a small piece of memory, e.g.
a single node of a linked list, I use 'free' to release the memory
afterwards. When using 'free', I don't have to worry about the size
of the allocated block the pointer is pointing to.
Is it the operating-system that is responsible to remember the size of
that piece of memory ?

It is an implementation responsibility. Whether that falls on the
operating system of the C run-time library is an implementation issue.

In that case, when I build a large linked-list by allocating memory
for each node, I get a lot of overhead in the operating system. Is

"lot" is a matter of opinion. Some operating systems may do this so
efficiently that the amount of overhead is not unacceptable *to you*.

this right ? or is this implementation-specific or operating-system
specific ? (FAQ q7.26 says that the implementation 'remembers' the
size. Does this mean that every implementation may do it in his own
way ? )
Yes

If the computer has to remember for each single allocation the size of
the allocated block, is it better than, to allocate a bigger piece of
memory and keep track of the use of it within the program, when I try
to create a linked-list or a binary tree or something like that ?

"better" is also a matter of opinion. It also probably depends on the
quality of implementation for your system (which is determined by both
the run-time library and the operating system).

In a system with a "good" malloc, individual calls for each node may
be preferable (less code to write, easier to maintain, etc). In a
system with a "poor" malloc, a single call for a large area which you
subdivide yourself may be preferable (program will run noticeably
faster, less fragmentation of memory, etc).


<<Remove the del for email>>

 

[Ben Pfaff]

I understand from other postings that casting a result from malloc
isn't good. In the past I have always been casting the malloc. I
think it was even necessary. (But that's a long time ago, I hadn't
heard of a standard for C these days). Was it really necessary or did
we do it only to silence the warnings ? (cfr. FAQ q7.7 - I can't
look in the manuals from these days : I don't have them any longer)

It was necessary with pre-ANSI compilers. Casting the result of
malloc() is not necessary with any ANSI C89 or later compliant
compiler. However, you'll normally get a warning if you forget

When I use malloc for the allocation of a small piece of memory, e.g.
a single node of a linked list, I use 'free' to release the memory
afterwards. When using 'free', I don't have to worry about the size
of the allocated block the pointer is pointing to.
Is it the operating-system that is responsible to remember the size of
that piece of memory ?

Somebody has to (more or less). It may be a library linked into
your program or it may be the operating system.

In that case, when I build a large linked-list by allocating memory
for each node, I get a lot of overhead in the operating system. Is
this right ? or is this implementation-specific or operating-system
specific ?

Yes, both.

(FAQ q7.26 says that the implementation 'remembers' the size.
Does this mean that every implementation may do it in his own
way ? )
Yes.

If the computer has to remember for each single allocation the
size of the allocated block, is it better than, to allocate a
bigger piece of memory and keep track of the use of it within
the program, when I try to create a linked-list or a binary
tree or something like that ?

If you think you can do the tracking with lower overhead than the
implementation, that can be a perfectly good idea.

 

[Malcolm]

I understand from other postings that casting a result from malloc
isn't good. In the past I have always been casting the malloc. I
think it was even necessary. (But that's a long time ago, I hadn't
heard of a standard for C these days). Was it really necessary or did
we do it only to silence the warnings ? (cfr. FAQ q7.7 - I can't
look in the manuals from these days : I don't have them any longer)

C implicitly casts a void * to another type of pointer. C++ does not. The
main reason for casting the return from malloc() is to allow the code to
compile under C++. This may be necessary or even undesireable, depending on
the environment in which you work.

.
Is it the operating-system that is responsible to remember the size of
that piece of memory ?

The operating system just has to allocate and free memory on demand. In
practise it usually does this by storing the size of the block in the memory
immediately before the pointer returned from malloc(). However a weird and
wonderful system may do things differently.

If the computer has to remember for each single allocation the size of
the allocated block, is it better than, to allocate a bigger piece of
memory and keep track of the use of it within the program, when I try
to create a linked-list or a binary tree or something like that ?

The best rule is to use malloc() to allocate your nodes. Then if the program
runs too slowly, you can see if you can optimise. A fixed-block allocator
can be written much more efficiently than one that has to deal with any
size. Unfortunatley the meory it uses cannot normally be recycled for
general use, so it is not a panacea.

 

[Jack Klein]

C implicitly casts a void * to another type of pointer. C++ does not. The

^^^^^^^^^^^^^^^^

No, it doesn't. C automatically converts a pointer to void to any
type of object pointer. There is no such thing as an "implicit cast"
in C, as a cast is defined as an "explicit conversion".

And there is no conversion from pointer to void to pointer to any type
of function. Only to pointer to other object types.

 

[RCollins]

^^^^^^^^^^^^^^^^

No, it doesn't. C automatically converts a pointer to void to any

^^^^^^^^^^^^^^^^^^^^^^

type of object pointer. There is no such thing as an "implicit cast"
in C, as a cast is defined as an "explicit conversion".

^^^^^^^^^^^^^^^^^^^

And there is no conversion from pointer to void to pointer to any type
of function. Only to pointer to other object types.

I don't understand this. As I read it, you are saying that C performs
an "automatic conversion" of the void *, but does not do an "explicit
conversion" (see underlined sections, above).

I understand that you are talking strictly about a cast operation; but
it seems to me that the compiler must *explicitly* convert the "void *"
to a "struct foo *" (for example).

What is the difference? Most important, what is the difference to the
*programmer* ?

 

[Ben Pfaff]

I don't understand this. As I read it, you are saying that C performs
an "automatic conversion" of the void *, but does not do an "explicit
conversion" (see underlined sections, above).

I understand that you are talking strictly about a cast operation; but
it seems to me that the compiler must *explicitly* convert the "void *"
to a "struct foo *" (for example).

What is the difference? Most important, what is the difference to the
*programmer* ?

An explicit conversion is one that you explicitly tell the
implementation to do, i.e. a cast. An automatic or implicit
conversion is done by the implementation without an explicit
request. The implementation has to do the same work regardless
of whether a conversion is implicit or explicit. The difference
is in the syntax, not the semantics.

 

[Jack Klein]

I don't understand this. As I read it, you are saying that C performs
an "automatic conversion" of the void *, but does not do an "explicit
conversion" (see underlined sections, above).

I understand that you are talking strictly about a cast operation; but
it seems to me that the compiler must *explicitly* convert the "void *"
to a "struct foo *" (for example).

What is the difference? Most important, what is the difference to the
*programmer* ?

From dictionary.com, the first entry for each respective word:

implicit: Implied or understood though not directly expressed.

explicit: Fully and clearly expressed; leaving nothing implied.

Now let's look at an implicit conversion in C:

int i;
char ch = 'a';
i = ch; /* implicit conversion of char to int */

And an explicit conversion:

int i;
char ch = 'a';
i = (int)ch;

See how the first is "implied or understood but not directly
expressed" and the second is "fully and clearly expressed; leaving
nothing implied"?

The use of a cast operator is an explicit directive to the compiler to
perform a conversion. In some cases, as in the two little examples
above, the cast specifies a conversion that would be implicit anyway.
In that case, it is just what I call a "redundant cast", which is
usually but not always just useless noise.

Consider another case, assume <limits.h> is included:

int i;
double d = (double)INT_MAX + 10.0;
i = d; /* implicit automatic conversion */
i = (int)d; /* explicit "redundant" cast */

In the second example, the redundant cast gains you absolutely
nothing. d contains a value outside the range of an unsigned int, so
the implicit conversion caused by the assignment produces undefined
behavior. The explicit cast causes exactly the same undefined
behavior, so it buys nothing at all.

There are some cases where you must use a cast to cause an explicit
conversion because there is no automatic conversion, for example
between any two types of object pointers if neither is a pointer to
void:

int i = 12;
int *ip = &i; /* extra variable for documentation */
char *cp = ip; /* diagnostic required */
char *cp = (char *)ip; /* OK */

And there are some conversions that are not allowed even with a cast:

struct s1 { int i1; double d2; };
struct s2 { double d2; int i2; };

struct s1 { 3, 42.0 };
struct s2;
s2 = s1;
s2 = (struct st)si;

And as for my assertion that a cast is an explicit conversion, rather
than an automatic conversion being a implicit cast, that comes from
the wording of the C standard itself:

6.3 Conversions

1 Several operators convert operand values from one type to another
automatically. This subclause specifies the result required from such
an implicit conversion, as well as those that result from a cast
operation (an explicit conversion). The list in 6.3.1.8 summarizes
the conversions performed by most ordinary operators; it is
supplemented as required by the discussion of each operator in 6.5.

 

[Joris Adriaenssens]

It was necessary with pre-ANSI compilers. Casting the result of
malloc() is not necessary with any ANSI C89 or later compliant
compiler. However, you'll normally get a warning if you forget
to #include <stdlib.h>. In that case, the correct fix is to add
the #include, not to add a cast.

....

Thanks for the fast replies. They are very clear to me.

 

[Kenneth Brody]

Joris Adriaenssens wrote:
[...]

I understand from other postings that casting a result from malloc
isn't good. In the past I have always been casting the malloc. I
think it was even necessary. (But that's a long time ago, I hadn't
heard of a standard for C these days). Was it really necessary or did
we do it only to silence the warnings ? (cfr. FAQ q7.7 - I can't
look in the manuals from these days : I don't have them any longer)

[...]

It was necessary back in the days when malloc() returned char* because
there was no such thing as "void" yet.

Now that malloc() returns void*, which can be implicitly converted to
anything*, it is considered "bad" to cast the return value, as this can
mask a missing header file. (ie: if malloc() isn't defined in any of
the header files, the compiler will think it returns int, and the cast
will mask the warning about converting int to a pointer.)

 

[jnf]

Joris Adriaenssens said:

When using 'free', I don't have to worry about the size
of the allocated block the pointer is pointing to.
Is it the operating-system that is responsible to remember the size of
that piece of memory ?

It's less OS dependant and more malloc-dependant, I realize that sounds
funny and I probably am using the wrong term there, so anyone feel free to
correct me there if necessary (or anywhere ;]).

To explain it a little better, the most common malloc()/free()
implementation provided under Linux, is called dl-malloc, or Doug Lea's
malloc(). It 'remembers' chunk size by allocating a few extra bytes of
space per chunk allocated, and then using the extra space to keep track of
size and whether the previous block is in use. If you were to malloc(0),
if I remember correctly it actually allocates 8, it rounds up whatever you
allocate up to the next double-word boundary. Because of this, the lower
end 3 bits of size to determine the status of the previous chunk.
prev_inuse is only used by dlmalloc if the previous block is not in use.

When a block is free()'d the ?deallocator? comes through and check's the
blocks size element to see if the previous one is in use or not, and also
checks the next block, and groups them altogether in bin's (grouped by
size), the effort here is to have as many large chunks as we can.

Really, it has been a long time since ive played with the inner working's
of malloc, and I am sure there are plenty of errors in what I said- but
basically that is how it works, think of it kinda like an advanced version
of the stack. This is also why keeping it pristine is so important, you
can corrupt the whole thing with a single bad chunk.

If you are really interested in it, feel free to read:

'A memory allocator' by doug lea

http://gee.cs.oswego.edu/dl/html/malloc.html


'vudo malloc tricks' by maxx

http://phrack.org/phrack/57/p57-0x08

'Once upon a free()' by anonymous

http://phrack.org/phrack/57/p57-0x09

I hope that gives you a better idea of how the implementation 'remembers',
it's important to not that across different operating systems, and even on
the same operating systems, you can find the semantics to be different,
but generally they are more or less the same with little differences.


jnf

You're running me on a live system. That's incredibly stupid.
-- RedHat Alpha installer.

 

[CBFalconer]

.... snip ...

Really, it has been a long time since ive played with the inner
working's of malloc, and I am sure there are plenty of errors in
what I said- but basically that is how it works, think of it
kinda like an advanced version of the stack. This is also why
keeping it pristine is so important, you can corrupt the whole
thing with a single bad chunk.

If you are really interested in it, feel free to read:

'A memory allocator' by doug lea
http://gee.cs.oswego.edu/dl/html/malloc.html

'vudo malloc tricks' by maxx
http://phrack.org/phrack/57/p57-0x08

'Once upon a free()' by anonymous
http://phrack.org/phrack/57/p57-0x09

You can also try my nmalloc for DJGPP, at:

<http://cbfalconer.home.att.net/download/nmalloc.zip>

 

[Malcolm]

6.3 Conversions

1 Several operators convert operand values from one type to another
automatically. This subclause specifies the result required from such
an implicit conversion, as well as those that result from a cast
operation (an explicit conversion). The list in 6.3.1.8 summarizes
the conversions performed by most ordinary operators; it is
supplemented as required by the discussion of each operator in 6.5.

Casts (use of the parentheses notation) are explicit. Therefore the same
operation without such notation can be called an "implicit cast". That's how
language works.

 

[RCollins]

Casts (use of the parentheses notation) are explicit. Therefore the same
operation without such notation can be called an "implicit cast". That's how
language works.

Yes ... that was the point I was trying to make. I fully understand
the meanings of the words "explicit" and "implicit". However, (my
point being), when the compiler *automatically* makes the cast for the
programmer, wouldn't be called an "implicit cast" ?

This was the wording that Jack Klein disputed in a post from Malcolm:

Malcom said:

>C implicitly casts a void * to another type of pointer. C++ does not.

Jack Klein replied:

>No, it doesn't. C automatically converts a pointer to void to any
>type of object pointer. There is no such thing as an "implicit cast"

So ... what *do* you call it when the compiler automatically performs
the cast for you?

 

[Keith Thompson]

Yes ... that was the point I was trying to make. I fully understand
the meanings of the words "explicit" and "implicit". However, (my
point being), when the compiler *automatically* makes the cast for the
programmer, wouldn't be called an "implicit cast" ?

No, it's an implicit conversion. There's no such thing as an implicit
cast.

A conversion is an action that occurs during program execution. It
can be either explicit (i.e., specified by a cast operator in the
program source) or implicit (i.e., not specified by a cast operator).

A cast is a source construct, consisting of a parenthesized type-name
followed by an expression. It specifies an explicit conversion.

I admit it's a bit confusing. For most other constructs, we use the
same name for the syntactic construct as for the action that occurs
during execution. For example, we commonly refer to both the source
construct "foo(42)" and to the action that it specifies as a "function
call", and there can be implicit function calls that don't correspond
to any source-code function call. For casts and conversions, we use
two different terms.

 

[Jack Klein]

Casts (use of the parentheses notation) are explicit. Therefore the same
operation without such notation can be called an "implicit cast". That's how
language works.

You are entitled to your opinion, however incorrect it is. The C
language is defined by an ANSI/ISO/IEC standard, however, not by your
opinions, even if they happen to be correct. Which they are not, in
this case.

A cast is something generated by a cast operator. Anything else is
not a cast.

========
6.5.4 Cast operators

Syntax
1 cast-expression:
unary-expression
( type-name ) cast-expression

Constraints

2 Unless the type name specifies a void type, the type name shall
specify qualified or unqualified scalar type and the operand shall
have scalar type.

3 Conversions that involve pointers, other than where permitted by the
constraints of 6.5.16.1, shall be specified by means of an explicit
cast.

Semantics

4 Preceding an expression by a parenthesized type name converts the
value of the expression to the named type. This construction is called
a cast. A cast that specifies no conversion has no effect on the type
or value of an expression.
========

Note specifically the first two sentences of paragraph 4. They are
quite clear. The only thing that the standard defines as being called
a cast is exactly defined in the first one.

So until and unless you can quote chapter and verse from the C
standard that anything else is called a cast, with or without the use
of the adjective 'implicit', STFU.

 

[Eric Sosman]

Casts (use of the parentheses notation) are explicit. Therefore the same
operation without such notation can be called an "implicit cast". That's how
language works.

Balderdash.

int main(void) {
return 0;
}

How many "implicit unary + operators" does this program use?

 

[Chris Torek]

(Abraham Lincoln is supposed to have said: "How many legs does a
dog have, if you call a tail a leg? Four -- calling a tail a leg
doesn't make it one.")

Balderdash.

int main(void) {
return 0;
}

How many "implicit unary + operators" does this program use?

42, obviously.

As for Malcolm's remark: people do indeed use sloppy language. It
is often worth correcting, as in this case. "Jargon" -- technical
terminology with very specific meanings (definition 2 at www.m-w.com)
-- needs to be used with care, lest it change into the other kind
of "jargon", meaning "confused unintelligible language" (definition
1a at www.m-w.com).

 

[Malcolm]

Balderdash.

int main(void) {
return 0;
}

How many "implicit unary + operators" does this program use?

An arbitrary number.
Sometimes you want to include unary +es, for instance where you have a list
of numbers of either sign and you want to make the formatting more readable.
Very occasionally you might want two or more, for instance if you need to
add a list of sines, which you replace by constnats as part of your
optimisation procedure.

 

[Merrill & Michele]

... snip ...

You can also try my nmalloc for DJGPP, at:

<http://cbfalconer.home.att.net/download/nmalloc.zip>

...snip...

I was just reading Stroustrup's home page and was astonished at the
belligerent tone towards casts. MPJ