realloc() implicit free() ?

[Walter Roberson]

If realloc() finds it necessary to move the memory block, then
does it free() the previously allocated block?

The C89 standard has some reference to undefined behaviour if one
realloc()'s memory that was freed by realloc(), but the only way
explicitly mentioned in the C89 standard to free memory via realloc()
is to realloc() it down to 0 bytes.

I had always assumed it would automatically free the previous memory,
but is the behaviour instead undefined [or defined as not happening] ?

 

[tigervamp]

If realloc() finds it necessary to move the memory block, then
does it free() the previously allocated block?
Yes.

The C89 standard has some reference to undefined behaviour if one
realloc()'s memory that was freed by realloc(), but the only way
explicitly mentioned in the C89 standard to free memory via realloc()
is to realloc() it down to 0 bytes.

Passing a pointer to realloc that points to memory free'd by _any_
function results in undefined behavior. If you successfully realloc
memory which results in the original object being deallocated, you must
provide a pointer to the new object created by realloc in subsequent
realloc calls as the old object has been freed.

I had always assumed it would automatically free the previous memory,
but is the behaviour instead undefined [or defined as not happening]

?

No, the behavior is well defined by the Standard, see section 7.20.3 of
C99, I don't have a C89 copy handy.

Rob Gamble

 

[Dave Vandervies]

If realloc() finds it necessary to move the memory block, then
does it free() the previously allocated block?

Quoth n869 (7.20.3.4):

[#2] The realloc function deallocates the old object pointed |
^^^^^^^^^^^^^^^^^^^^^^^^^^
to by ptr and returns a pointer to a new object that has the |
size specified by size. The contents of the new object |
shall be the same as that of the old object prior to |
deallocation, up to the lesser of the new and old sizes. |
Any bytes in the new object beyond the size of the old |
object have indeterminate values. |



[#4] The realloc function returns a pointer to the new |
object (which may have the same value as a pointer to the |
^^^^^^^^^^^^^^^^^^^^^^^^
old object), or a null pointer if the new object could not |
be allocated.

So the old one always gets (conceptually) freed, though in practice it's
Highly Unlikely that you'll get a non-moving call to realloc actually
freeing anything (this is an excellent example of the as-if rule).

The C89 standard has some reference to undefined behaviour if one
realloc()'s memory that was freed by realloc(), but the only way
explicitly mentioned in the C89 standard to free memory via realloc()
is to realloc() it down to 0 bytes.

The C89 draft I have here describes the same behavior, but it's less
clear that if the memory block fgets moved the old memory is deallocated.

I had always assumed it would automatically free the previous memory,
but is the behaviour instead undefined [or defined as not happening] ?

a call to realloc can do one of three things:
-fail to resize the memory block and return NULL without deallocating
the old one
-resize the block in-place and return the same pointer you gave it
-resize-and-move the block with behavior equivalent to doing malloc-
memcpy-free yourself (except that the old and new blocks are allowed
to overlap - I don't know if there's any reasonable way to implement
it that actually would overlap them) and return a pointer to the new
memory block

None of these involve leaving old memory blocks unfree()d.


dave

 

[Eric Sosman]

If realloc() finds it necessary to move the memory block, then
does it free() the previously allocated block?

Yes. (Well, it might not actually call free(), but
the outcome will be the same.) C99 7.20.3.4/2:

The realloc function deallocates the old object [...]

The C89 standard has some reference to undefined behaviour if one
realloc()'s memory that was freed by realloc(), but the only way
explicitly mentioned in the C89 standard to free memory via realloc()
is to realloc() it down to 0 bytes.

C89's language isn't as explicit, but throughout section
4.3.2 it keeps mentioning realloc() as one of the ways memory
can be deallocated, and in 4.10.3.4 it describes realloc() as
returning a pointer to the "possibly moved" memory. I think
the only reasonable reading of "moved" involves deallocation
of the previously-occupied space.

I had always assumed it would automatically free the previous memory,
but is the behaviour instead undefined [or defined as not happening] ?

A proper realloc() call can't invoke undefined behavior.
Even if C89 could be construed as not requiring the old memory
to be released, the behavior wouldn't be undefined -- you can't
refer to that old memory or to any pointer to it without invoking
undefined behavior, but simply leaking it isn't U.B. At worst
it'd be a QoI issue, with any implementation that leaked the
memory exhibiting negative QoI ...

 

[Walter Roberson]

C89's language isn't as explicit,

Thanks, everyone; and thanks for recognizing that C89 isn't precise
on the point.

 

[SM Ryan]

(e-mail address removed)-cnrc.gc.ca (Walter Roberson) wrote:
# If realloc() finds it necessary to move the memory block, then
# does it free() the previously allocated block?

You're over-specifying the function. What happens to the block is irrelevant
to understanding the function. Use the address returned by realloc; don't
worry about what it was before.

# I had always assumed it would automatically free the previous memory,
# but is the behaviour instead undefined [or defined as not happening] ?

I did a debugging realloc that left the original block alone so that it could
dump an allocation history. There could be all kinds of exotic things happenning
to the previous memory: that's why you shouldn't concern yourself with it.

 

[CBFalconer]

If realloc() finds it necessary to move the memory block, then
does it free() the previously allocated block?
Yes.

The C89 standard has some reference to undefined behaviour if
one realloc()'s memory that was freed by realloc(), but the only
way explicitly mentioned in the C89 standard to free memory via
realloc() is to realloc() it down to 0 bytes.

Passing a pointer to realloc that points to memory free'd by _any_
function results in undefined behavior. If you successfully
realloc memory which results in the original object being
deallocated, you must provide a pointer to the new object created
by realloc in subsequent realloc calls as the old object has been
freed.

I had always assumed it would automatically free the previous
memory, but is the behaviour instead undefined [or defined as
not happening] ?

No, the behavior is well defined by the Standard, see section
7.20.3 of C99, I don't have a C89 copy handy.

No, I don't consider it well defined. It leaves too much up to the
implementation in the case of zero block size requests. This means
that NULL is not always a sign of failure, and that a non-NULL
return is not necessarily freeable or reallocable.

In my nmalloc implementation for DJGPP I have chosen to treat a
zero block size as a request for one byte, and let the alignment
mechanisms raise it as they will. This ensures that all successful
requests return a non-NULL pointer that can safely be used in
subsequent frees or reallocs. (It also has the convenient to me
side effect of ensuring space for some record-keeping in my code)

I suspect the standard is written to not invalidate the sloppier
malloc implementations already out there. In cases like this I
think it should include a recommended practice.

 

[Walter Roberson]

(e-mail address removed)-cnrc.gc.ca (Walter Roberson) wrote:
# If realloc() finds it necessary to move the memory block, then
# does it free() the previously allocated block?

You're over-specifying the function. What happens to the block is irrelevant
to understanding the function. Use the address returned by realloc; don't
worry about what it was before.

The question is not over-specified if one is concerned about
whether one is leaking memory. When your datasets are ~1 Gb each
and you are handling them in a loop, you can't afford to allow memory
to leak.

 

[SM Ryan]

(e-mail address removed)-cnrc.gc.ca (Walter Roberson) wrote:
# In article <[email protected]>,
# >[email protected] (Walter Roberson) wrote:
# ># If realloc() finds it necessary to move the memory block, then
# ># does it free() the previously allocated block?
#
# >You're over-specifying the function. What happens to the block is irrelevant
# >to understanding the function. Use the address returned by realloc; don't
# >worry about what it was before.
#
# The question is not over-specified if one is concerned about
# whether one is leaking memory. When your datasets are ~1 Gb each
# and you are handling them in a loop, you can't afford to allow memory
# to leak.

All you can do to prevent leaks is to match frees to allocs; you don't need to
know how the library is implemented to do that much. It's still leaking
you're stuck unless you can get the library source code.

 

[Robert Gamble]

If realloc() finds it necessary to move the memory block, then
does it free() the previously allocated block?
Yes.

The C89 standard has some reference to undefined behaviour if
one realloc()'s memory that was freed by realloc(), but the only
way explicitly mentioned in the C89 standard to free memory via
realloc() is to realloc() it down to 0 bytes.

Passing a pointer to realloc that points to memory free'd by _any_
function results in undefined behavior. If you successfully
realloc memory which results in the original object being
deallocated, you must provide a pointer to the new object created
by realloc in subsequent realloc calls as the old object has been
freed.

I had always assumed it would automatically free the previous
memory, but is the behaviour instead undefined [or defined as
not happening] ?

No, the behavior is well defined by the Standard, see section
7.20.3 of C99, I don't have a C89 copy handy.

No, I don't consider it well defined. It leaves too much up to the
implementation in the case of zero block size requests.

I was referring specifically to the behavior of realloc deallocating the
original object which is well defined.

This means that NULL is not always a sign of failure, and that a non-NULL
return is not necessarily freeable or reallocable.

NULL is always a sign of failure when the size provided is non-zero, if
you want to provide a size of zero is is easy enough to handle the return

What evidence do you have that a non-NULL value returned by realloc, when
provided a size of zero, cannot safely be passed to free() or realloc()?

Rob Gamble

 

[CBFalconer]

I was referring specifically to the behavior of realloc
deallocating the original object which is well defined.


NULL is always a sign of failure when the size provided is
non-zero, if you want to provide a size of zero is is easy
enough to handle the return

What evidence do you have that a non-NULL value returned by
realloc, when provided a size of zero, cannot safely be passed
to free() or realloc()?

As I read the following from N869:

7.20.3 Memory management functions

[#1] The order and contiguity of storage allocated by
successive calls to the calloc, malloc, and realloc
functions is unspecified. The pointer returned if the
allocation succeeds is suitably aligned so that it may be
assigned to a pointer to any type of object and then used to
access such an object or an array of such objects in the
space allocated (until the space is explicitly freed or
reallocated). Each such allocation shall yield a pointer to
an object disjoint from any other object. The pointer
returned points to the start (lowest byte address) of the
allocated space. If the space cannot be allocated, a null
pointer is returned. If the size of the space requested is
zero, the behavior is implementation-defined: either a null

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
pointer is returned, or the behavior is as if the size were
some nonzero value, except that the returned pointer shall
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
not be used to access an object. The value of a pointer
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
that refers to freed space is indeterminate.

Is freeing NULL so generated accessing an object? Is freeing
non-NULL so generated accessing an object? When things are buried
in layers of code we can easily guarantee that the pointer was
created by malloc/realloc, and we can even guard it by making it
NULL after passing to free, but that guard won't work if realloc
can return a non-freeable pointer.

--
Some informative links:
http://www.geocities.com/nnqweb/
http://www.catb.org/~esr/faqs/smart-questions.html
http://www.caliburn.nl/topposting.html
http://www.netmeister.org/news/learn2quote.html

 

[S.Tobias]

I think you must be misreading something (or I don't understand
what your saying). I'm sure
free(malloc(0));
is _always_ valid.

Have a look at the Rationale, page 160. If `calloc' returned
a non-reallocable pointer, then subsequent reallocations
could no work, could they?

[snip]

As I read the following from N869:

7.20.3 Memory management functions
[snip]
pointer is returned. If the size of the space requested is
zero, the behavior is implementation-defined: either a null

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
pointer is returned, or the behavior is as if the size were
some nonzero value, except that the returned pointer shall
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
not be used to access an object. The value of a pointer
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
that refers to freed space is indeterminate.

Is freeing NULL so generated accessing an object?

Null ptrs are special cases. free(NULL) is no-op by definition.

Is freeing
non-NULL so generated accessing an object?

No, because the Standard doesn't specify that `free' accesses
the object before deallocation.

I think that `malloc(0)' also allocates an object (if returns non-null),
which may be freed later. The compiler knows the object won't be
accessed, therefore the returned pointer may "point" into some
read-only memory, or may have some invalid/trap value which blows up
when dereferenced.

In my nmalloc implementation for DJGPP I have chosen to treat a
zero block size as a request for one byte, and let the alignment
mechanisms raise it as they will.

I'm not criticizing this, there're good reasons to do it like that,
but the Standard does not require the allocated object to have
alignment more than its size, right? Subsequent `malloc(1)'-s could
return ptrs that differ by 1 byte, right? (I'm asking because
the Std is a bit unclear there, IMHO.)

 

[CBFalconer]

I'm not criticizing this, there're good reasons to do it like that,
but the Standard does not require the allocated object to have
alignment more than its size, right? Subsequent `malloc(1)'-s
could return ptrs that differ by 1 byte, right? (I'm asking
because the Std is a bit unclear there, IMHO.)

No, that can't happen. Any non NULL return from
malloc/calloc/realloc must be suitably aligned for any object
whatsoever.

--
Some informative links:
http://www.geocities.com/nnqweb/
http://www.catb.org/~esr/faqs/smart-questions.html
http://www.caliburn.nl/topposting.html
http://www.netmeister.org/news/learn2quote.html

 

[Robert Gamble]

I was referring specifically to the behavior of realloc
deallocating the original object which is well defined.


NULL is always a sign of failure when the size provided is
non-zero, if you want to provide a size of zero is is easy
enough to handle the return

What evidence do you have that a non-NULL value returned by
realloc, when provided a size of zero, cannot safely be passed
to free() or realloc()?

As I read the following from N869:

7.20.3 Memory management functions

[#1] The order and contiguity of storage allocated by
successive calls to the calloc, malloc, and realloc
functions is unspecified. The pointer returned if the
allocation succeeds is suitably aligned so that it may be
assigned to a pointer to any type of object and then used to
access such an object or an array of such objects in the
space allocated (until the space is explicitly freed or
reallocated). Each such allocation shall yield a pointer to
an object disjoint from any other object. The pointer
returned points to the start (lowest byte address) of the
allocated space. If the space cannot be allocated, a null
pointer is returned. If the size of the space requested is
zero, the behavior is implementation-defined: either a null

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
pointer is returned, or the behavior is as if the size were
some nonzero value, except that the returned pointer shall
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
not be used to access an object. The value of a pointer
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
that refers to freed space is indeterminate.

Is freeing NULL so generated accessing an object?
No.

Is freeing non-NULL so generated accessing an object?
No.

When things are buried in layers of code we can easily guarantee
that the pointer was created by malloc/realloc, and we can even
guard it by making it NULL after passing to free, but that guard won't
work if realloc can return a non-freeable pointer.

The pointer returned by calloc/malloc/realloc in any scenerio can always
be passed to free() at least once (only once for non-NULL).

From 7.20.3.2p2, free description:

"The free function causes the space pointed to by ptr to be deallocated,
that is, made available for further allocation. If ptr is a null pointer,
no action occurs. Otherwise, if the argument does not match a pointer
earlier returned by the calloc, malloc, or realloc function, or if the
space has been deallocated by a call to free or realloc, the behavior is
undefined."

According to the description, any pointer returned by calloc, malloc or
realloc that has not already been freed may be passed to free() without
invoking undefined behavior. No special provisions are made for the case
where zero size was passed to the allocation function to generate the
pointer in question.

To further support this position, section 7.20.3 of the Rationale
v5.10 provides the following example:

"The treatment of null pointers and zero-length allocation requests in the
definition of these functions was in part guided by a desire to support
this paradigm:

OBJ * p; // pointer to a variable list of OBJs
/* initial allocation */
p = (OBJ *) calloc(0, sizeof(OBJ));
/* ... */
/* reallocations until size settles */
while(1) {
p = (OBJ *) realloc((void *)p, c * sizeof(OBJ));
/* change value of c or break out of loop */
}
"

In this example, p is assigned the return value of calloc which, since one
of the arguments is zero, according to Standard behaves the same as if
malloc were passed a size of zero. p is now either NULL, or non-NULL
depending on the implementation (or possibly NULL if an error occured).
The value of p is then passed to realloc to increase the size, initially
from zero. If p was NULL from the calloc call, realloc will behave like
malloc. If calloc returned non-NULL, that value is now being passed to
realloc which deallocates the original zero-length object (as free would
do) and allocates more memory, etc.

In short, if an implmentation can "allocate" zero-length objects, it can
also free/realloc them. The effort needed to handle the implementation
defined part seems minimal.

Rob Gamble

 

[Michael Mair]

No, that can't happen. Any non NULL return from
malloc/calloc/realloc must be suitably aligned for any object
whatsoever.

Can you provide chapter and verse, please?
This seems to be an unnecessary restriction -- why should malloc()
return 16-byte aligned memory for a 3-byte request if the
implementation "knows" that this implies a necessity for, say,
2-byte alignment _at_most_?

Cheers
Michael

 

[Russell Shaw]

No, that can't happen. Any non NULL return from
malloc/calloc/realloc must be suitably aligned for any object
whatsoever.

I'm writing a malloc thing and compiling it in gcc on a pc.
How do you know what the alignment should be?

 

[Robert Gamble]

Can you provide chapter and verse, please?

7.20.3p1 sentence 2:

"The pointer returned if the allocation succeeds is suitably aligned so
that it may be assigned to a pointer to any type of object and then used
to access such an object or an array of such objects in the space
allocated (until the space is explicitly deallocated)."

This seems to be an unnecessary restriction -- why should malloc()
return 16-byte aligned memory for a 3-byte request if the implementation
"knows" that this implies a necessity for, say, 2-byte alignment
_at_most_?

I'm not sure why, the Rationale doesn't give any hints here. It doesn't
particularly make sense that the allocated space must be aligned to
store an object that could not fit into the space, perhaps there is
another reason we are overlooking or maybe I'm misinterpreting the above
quote.

Cheers
Michael

Rob Gamble

 

[S.Tobias]

7.20.3p1 sentence 2:

"The pointer returned if the allocation succeeds is suitably aligned so
that it may be assigned to a pointer to any type of object and then used
to access such an object or an array of such objects in the space
allocated (until the space is explicitly deallocated)."

My interpretation of this is that the word "object" refers to
the allocated object, and "any type of object" means any type
that *this* object can have, although I think it is very unclear.
Without such interpretation the second part of the sentence
would not make sense (unless we accept that malloc(1) returns
an object suitable for any type - SIZE_MAX bytes at least).

 

[Dave Vandervies]

I'm writing a malloc thing and compiling it in gcc on a pc.
How do you know what the alignment should be?

Compiler magic.

(This is something that the implementation needs to know (not just
for malloc; it will be needed anywhere it creates an object that may
have alignment restrictions), but it's not exposed to the programmer,
at least not without going beyond what the standard defines.)


dave

 

[Chris Torek]

Can you provide chapter and verse, please?
This seems to be an unnecessary restriction -- why should malloc()
return 16-byte aligned memory for a 3-byte request if the
implementation "knows" that this implies a necessity for, say,
2-byte alignment _at_most_?

Indeed, it does seem unnecessary. It also seems to be what the
Standard requires. I suspect, however, that an implementor might
be able to get away with the "as-if" rule and align to just 2 bytes
in this case, provided the following code fragment still works:

void *vp;
char *cp;
double *dp;
/* add other types if desired */

cp = malloc(3);
if (cp == NULL) ... handle error ...
vp = cp;
dp = vp;
assert(vp == cp);
assert(dp == vp);
cp = (char *)dp;
assert(cp == vp);
puts("all OK");

In other words, if we have a system on which we can detect "failure
to align for any object whatsoever" by just using ordinary pointer
assignments, *then* that system will have to do 16-byte (or whatever)
alignment for a 3-byte malloc; but if the system handles "misaligned"
pointers without trouble, as long as they are not used to access
their misaligned objects, then -- because there will be no way for
the user to tell -- the as-if rule will allow us to implement
malloc(3) with just two-byte alignment.