Freeing memory - will it be available immediately

[Jack Klein]

[snips]

I think you are misunderstanding what freeing immediately actually
means. In fact the standard has nothing to say about when it is freed
and it could mean many things in many systems. I would guess that free
would remove the memory from the allocated list and add it to the free
list, at a minimum. This has to be done. Whether it also returns the
memory to the OS, i.e., shrinks the data segment of the process, is
totally implementation specific.

Is it? I see nothing whatsoever in the definition of free which allows
for any implementation-defined behaviour; its behaviour, quite the
contrary, is absolutely and clearly defined, and *does not permit*
handing the memory back to the OS.

Chapter and verse, please, I insist. What wording in the C standard
prohibits free() from returning memory to the operating system?

If you can find "implementation defined" in the definition of free, in a
manner which would allow the behaviour you describe, please quote it; I
cannot find it anywhere.

It is not implementation-defined, it is indeed unspecified. The one
and only statement in the standard that speaks about memory
deallocated by free() is (in C99) the first sentence of paragraph 2 of
7.20.3.2: "The free function causes the space pointed to by ptr to be
deallocated, that is, made available for further allocation."

What, in this wording, says that the space must be available for
further allocation by the program that free'd it?

What, in this wording, prohibits the implementation, which includes
the host OS in a hosted environment, from making that memory space
available to another process/program/component?

Since the standard does not define or constrain "further allocation",
how can you claim it forbids the second example?

--
Jack Klein
Home: http://JK-Technology.Com
FAQs for
comp.lang.c http://c-faq.com/
comp.lang.c++ http://www.parashift.com/c++-faq-lite/
alt.comp.lang.learn.c-c++
http://www.club.cc.cmu.edu/~ajo/docs/FAQ-acllc.html

 

[Jack Klein]

A literal reading of the Standard's text supports Kelsey's
interpretation, but in practise I'd expect significantly sized
deallocations to be handed back to the OS, if there was a means to do
so.

[snip]

A literal reading of the standard's text says that allocated memory
properly free'd is made available for "further allocation". It most
certainly say "further allocation by the same program". Since the
term "further allocation" is not defined or constrained by the
standard in any way, where exactly does it forbid that further
allocation from being to another executable, process, device driver,
etc.?

As already said, Kelsey's interpretation is reasonable, but by no
means either guaranteed or required by the actual wording of the
standard.

--
Jack Klein
Home: http://JK-Technology.Com
FAQs for
comp.lang.c http://c-faq.com/
comp.lang.c++ http://www.parashift.com/c++-faq-lite/
alt.comp.lang.learn.c-c++
http://www.club.cc.cmu.edu/~ajo/docs/FAQ-acllc.html

 

[Richard Bos]

Is it? I see nothing whatsoever in the definition of free which allows
for any implementation-defined behaviour; its behaviour, quite the
contrary, is absolutely and clearly defined, and *does not permit*
handing the memory back to the OS.

You keep opining that, and you continue to be wrong.

Richard

 

[Kelsey Bjarnason]

[snips]

Chapter and verse, please, I insist. What wording in the C standard
prohibits free() from returning memory to the operating system?

One needs to consider context. The C standard defines the behaviour of a
C program, which in turn means it also defines how the standard library
functions define _as relates to that C program_.

The definition of free says that the memory freed is made available for
further allocation.

Thus, in context, it defines, _for the C program_ that the memory remains
available; it cannot, therefore, hand it back to the OS, as the memory
would then _not_ be available for further allocation.

It is not implementation-defined, it is indeed unspecified.

Actually, it is quite clearly defined, absolutely and unequivocally.

The one and
only statement in the standard that speaks about memory deallocated by
free() is (in C99) the first sentence of paragraph 2 of 7.20.3.2: "The
free function causes the space pointed to by ptr to be deallocated, that
is, made available for further allocation."

What, in this wording, says that the space must be available for further
allocation by the program that free'd it?

Context. The standard is not defining the operation of toasters or
televisions, but of C programs and the library functions on which they
rely. It defines exactly such a behaviour for free: the memory is made
available for further allocation. By free, to the C program, the very
things whose behaviour is being defined.

What, in this wording, prohibits the implementation, which includes the
host OS in a hosted environment, from making that memory space available
to another process/program/component?

The fact that this violates the defined behaviour of free, which makes
the memory available for reallocation, a behaviour defined in a document
whose sole reason for existence is to define behaviours of C programs.

Since the standard does not define or constrain "further allocation",
how can you claim it forbids the second example?

By noting the context in which the behaviour is defined, and a complete
absence of any clause or allowance for any behaviour *other* than what I
described.

 

[Kelsey Bjarnason]

You keep opining that, and you continue to be wrong.

If I'm wrong, please point out the clause which allows the defined
behaviour to work other than as the standard defines it - you know, the
part where it says "implementation defined" or "undefined" or even
"unspecified" in the definition of free.

You can't; it's not there. What is there is a clear and absolute
definition of how free behaves in the context of a C program, the very
thing the standard exists to define. What is that behaviour? Oh, yes,
the memory is made available for further allocation. In what context?
Oh, yes, in the context of defining the behaviour of the C program.

It's pretty clear, there is, as far as I can tell, simply no other way to
read the definition of free, *in context*, and come to any other
conclusion.

You say I'm wrong? Fine; show me. Show me where the definition of free
gives it *any* allowance, via undefined, unspecified or implementation
defined clauses, to behave other than as the standard says it does.

 

[Kelsey Bjarnason]

According to your interpretation of the somewhat ambiguous phrase "made
available for further allocation".

The only ambiguity in the phrase is *when* the memory becomes available
for subsequent allocation. It could be immediately or six months from
now; the standard doesn't say.

What it does say is that the defined behaviour, in a document whose sole
reason to exist is to define how a C program is expected to work, is that
the memory is made available for further allocation. You know, to the C
program, the very thing whose behaviour is being defined.

If you can find a clause in the definition of free which allows for some
other behaviour, such as handing the memory back to the OS, please trot
it out; I can't find it anywhere.

 

[Willem]

Kelsey wrote:
) One needs to consider context. The C standard defines the behaviour of a
) C program, which in turn means it also defines how the standard library
) functions define _as relates to that C program_.

I believe this is the point of contention.


SaSW, Willem
--
Disclaimer: I am in no way responsible for any of the statements
made in the above text. For all I know I might be
drugged or something..
No I'm not paranoid. You all think I'm paranoid, don't you !
#EOT

 

[Kelsey Bjarnason]

[snips]

A literal reading of the Standard's text supports Kelsey's
interpretation, but in practise I'd expect significantly sized
deallocations to be handed back to the OS, if there was a means to do
so.

[snip]

A literal reading of the standard's text says that allocated memory
properly free'd is made available for "further allocation". It most
certainly say "further allocation by the same program".

I assume you meant "does not say" there.

Since the term
"further allocation" is not defined or constrained by the standard in
any way, where exactly does it forbid that further allocation from being
to another executable, process, device driver, etc.?

The C standard defines the expected behaviour of a C program.

The C standard, in order to do this, also defines the expected behaviour
of the standard library functions, behaviours on which that C program can
rely.

The C standard defines free as releasing the memory to be made available
for further allocation.

That's the defined operation _in the context of defining the operation of
a C program and the standard library on which it relies_. There is
nothing in that definition which allows for any other behaviour, such as
handing the memory back to the OS.

Nor, as noted elsewhere, does the "as-if" rule give us an out; failure to
make the memory available for further allocation (eg by giving it back to
the OS) is a direct violation of the text of the standard and cannot come
under the as-if rule; the as-if rule allows flexibility in conforming
behaviour, it does not allow non-conforming behaviour to take the place
of conforming.

Let's turn this on its head. You say "It most certainly does not say
'further allocation by the same program'". Very good. Explain, then,
exactly what the standard is defining the behaviour of, *if not the very
program under discussion*?

Of course it's defining the behaviour of the program and only the
program; it has no ability to define behaviours beyond that. It cannot
mandate that the system actually use a hard disk, or a monitor, it cannot
mandate that "text mode" universally means using "\r\n" so any other
applications or OSen better wake up; it defines one thing and only one
thing, the behaviour of the C program written against that standard.

In order to do so, in order to define the behaviour of the C program, it
*also* has to define the behaviours of the standard library, on which the
C program may rely. It does so, in the case of free: the memory is made
available for further allocation.

Not to the OS, not to your toaster oven, not to some other program, the C
standard has nothing to say on these matters. No, the only thing under
examination, the only candidate which the standard *can* define such
behaviours for, is the C program. It defines just such a behaviour, for
that program, and that definition absolutely precludes the notion of
handing the memory back to the OS; doing so violates the defined
behaviour of free.

 

[Keith Thompson]

The only ambiguity in the phrase is *when* the memory becomes available
for subsequent allocation. It could be immediately or six months from
now; the standard doesn't say.

What it does say is that the defined behaviour, in a document whose sole
reason to exist is to define how a C program is expected to work, is that
the memory is made available for further allocation. You know, to the C
program, the very thing whose behaviour is being defined.

If you can find a clause in the definition of free which allows for some
other behaviour, such as handing the memory back to the OS, please trot
it out; I can't find it anywhere.

Show us where the standard says that an implementation in which
storage deallocated by free() is made available for further allocation
by another program invoked via the standard system() function would be
non-conforming.

(For extra credit, construct a syntax diagram of that sentence.)

 

[Kelsey Bjarnason]

[snips]

A literal reading of the Standard's text supports Kelsey's
interpretation, but in practise I'd expect significantly sized
deallocations to be handed back to the OS, if there was a means to do
so. *I* wouldn't want to use any implementation that fails to return
deallocated memory of significant size (say 100 Mb) back to the OS, if
at all possible. It's poor QoI and selfish behaviour in a timesharing
environment.

Ooh, no, I would *not* want such a setup either. If I free memory, I
would much rather have it go back into the pool for all applications to
use; this strikes me as the most reasonable possible behaviour. It's
simply not a behaviour allowed by the standard.

 

[Walter Roberson]

The C standard defines free as releasing the memory to be made available
for further allocation.

That's the defined operation _in the context of defining the operation of
a C program and the standard library on which it relies_. There is
nothing in that definition which allows for any other behaviour, such as
handing the memory back to the OS.

Nor, as noted elsewhere, does the "as-if" rule give us an out; failure to
make the memory available for further allocation (eg by giving it back to
the OS) is a direct violation of the text of the standard and cannot come
under the as-if rule; the as-if rule allows flexibility in conforming
behaviour, it does not allow non-conforming behaviour to take the place
of conforming.

Why do you say that giving the memory back to the operating system
is a "failure to make the memory available for further allocation" ?

You seem to be making at least two assumptions:

1) That it is not the operating system that is responsible for the
allocation of memory within a running program; and

2) That memory given back to the operating system will not be available
to the program when the program requests more memory.


It turn, (2) breaks down into several cases:

(2a) that the -exact same- physical memory gets re-allocated by the
operating system in the -exact same- virtual address
(2b) that the -exact same- physical memory gets re-allocated by the
operating system in a -different- virtual address
(2c) that -different- physical memory gets allocated by the operating
system into the -exact same- virtual address as the original allocation
(2d) that -different- physical memory gets allocated by the operating
system into a -different- virtual address

Is there any conformant program that can usefully distinguish
these cases? Especially if the same virtual address is used, would
those subcases not be covered by the "as if" rules?

IMHO, the only situation in which it makes a difference as to whether
memory is given back to the operating system or not, is the case
in which the operating system is unable to return to the
program as much memory as was free()'d (e.g., if resources ran low
and there was no longer enough for the program to get back what it
once had.) I could see that as a potential violation of the
wording of the standard, but if the memory is available again from
the operating system for the asking, I see no violation.
[Note: I would be willing to entertain arguments having to do with
hypothetical memory fragmentation issues.]

 

[Willem]

Kelsey wrote:
) Ooh, no, I would *not* want such a setup either. If I free memory, I
) would much rather have it go back into the pool for all applications to
) use; this strikes me as the most reasonable possible behaviour. It's
) simply not a behaviour allowed by the standard.

So all you're really saying is that the wording used in the standard
is not quite what it should be.


SaSW, Willem
--
Disclaimer: I am in no way responsible for any of the statements
made in the above text. For all I know I might be
drugged or something..
No I'm not paranoid. You all think I'm paranoid, don't you !
#EOT

 

[Herbert Rosenau]

Will 'free' return the memory Immediately to the OS ?

There is nothing that says that an block given back fpr forther
allocation may given back to the OS. It belongs onnly to the
application if free() will
- reserve the memor< freed only for another malloc() of the same
program
- given back immediately to the OS
- only given back to the OS when sone other condition(s) are fullifyed

--
Tschau/Bye
Herbert

Visit http://www.ecomstation.de the home of german eComStation
eComStation 1.2R Deutsch ist da!

 

[Herbert Rosenau]

p = malloc(sizeof(*p));
free(p);
p = malloc(sizeof(*p));

then malloc() will return the same value both times.

There is no guarantee on. malloc() may return the same block or may
return another one leaving the last returned alone for later
allocation or may have given that block already away to another
request or may have given that block back to the OS and serve you
another newly aquired other one.

Always returning memory to the OS is likely to be a very poor
strategy, because the overhead of calling the OS is usually very high
compared with having the program manage memory itself. The down side
is that the memory isn't available to other programs, but (a) in
practice, few programs allocate a large amount of memory, free it, and
don't allocate similar quantities later, and (b) on a system with
virtual memory it won't affect the availability of real memory, at
least for chunks of memory large compared with a page.

There is nothing in the standard that describes how malloc should
aquire a memory block for the app. There is nothing in the standard
too that describes how an implementation should maintain memory. So it
is up to the implementation und which internally known conditions it
requests more memory from the OS, or gives memory back to it or holds
it for internal usage or gives it to the application, except:

- a pointer given to the program is aligned to allow to access an
object any data type the implementation allows the application to
handle.

- free will give the block freed by free() to another call of
malloc,realloc or calloc when that requests one. Nothing says that
this is immediately on the next request.

--
Tschau/Bye
Herbert

Visit http://www.ecomstation.de the home of german eComStation
eComStation 1.2R Deutsch ist da!

 

[Herbert Rosenau]

[snips]

I think you are misunderstanding what freeing immediately actually
means. In fact the standard has nothing to say about when it is freed
and it could mean many things in many systems. I would guess that free
would remove the memory from the allocated list and add it to the free
list, at a minimum. This has to be done. Whether it also returns the
memory to the OS, i.e., shrinks the data segment of the process, is
totally implementation specific.

Is it? I see nothing whatsoever in the definition of free which allows
for any implementation-defined behaviour; its behaviour, quite the
contrary, is absolutely and clearly defined, and *does not permit*
handing the memory back to the OS.

If you can find "implementation defined" in the definition of free, in a
manner which would allow the behaviour you describe, please quote it; I
cannot find it anywhere.

There is not a single word about what free() has to do in
multithreaded multiprocess environment with memory at all.

The only the stasnderd says that free() will hold memory gotten back
for another allocation request - and as there is absolutely no
specification who can request that memory it may be requested by
- kernel for kernel internal usage
- driver for driver usage
- another prograsm for its usage
- another thread of the same program
- some other instance of that environment fir its internal usage
- give for some other memory request from soemwhat that requests
memory for sharing something with something else

The standard does nothing than to require that memory given back by
free() can be reused and gets not unuseable.

This allows to maintain memory the most successfull usage for the
implementation at whole without knowledge of how many (if even a singe
one) prgrams are able to run in parallel.

--
Tschau/Bye
Herbert

Visit http://www.ecomstation.de the home of german eComStation
eComStation 1.2R Deutsch ist da!

 

[Herbert Rosenau]

If I'm wrong, please point out the clause which allows the defined
behaviour to work other than as the standard defines it - you know, the
part where it says "implementation defined" or "undefined" or even
"unspecified" in the definition of free.

So the standard proklamates that on an implementation is only one
single program exists, as it says nothing about what a program is, how
a program can be started or ended. It says nothing about interprocess
communication, files, printers and even devices.

At least a single program in context of standard means that there is
no operating system at all. So free() has to make free'd memory ready
to get allocated again and not blocked from allocation further. It
does never forbid to give memory< back to the OS as there is no OS
known by the standard. It does explicity require that that block of
memory can be allocated later (by anyone who can allocate it (even
other programs - because the standard does not even know that other
program can exist.

The standard is describing an abstract mashine that knows nothing
about
- memory
- CPU
- OS
- keyboard
- mouse
- printer
- screen
- any other real existent device
- programs
- DLLs
- libraries
- linker
- compiler
- debugger

So, come on and reconnoite how the standard will allow or dedy
something about something it knows not even that it (can) exist.

So the standard lefts still open what it measn by "another allocation"
other than the next request of malloc(), realloc() or calloc() should
get the now freed block again when it requests a region of memory. As
it can't differ between processes or threads because it knows not that
they amy (or even my not) exist on an real implementation it lefts
this open to the implementation give the freed memory to the next
instance that requests it.

The standard is menticulous exact in describing the language - but
extremly lax outside that to allow an implementationb to define what
it needs. So free()ing memory means give that memory to the next who
requests it, not definitely the same program, the same thread but
simply the next call that needs it. And not even the next in sequence.

--
Tschau/Bye
Herbert

Visit http://www.ecomstation.de the home of german eComStation
eComStation 1.2R Deutsch ist da!

 

[Herbert Rosenau]

The proper answer is that, according to the text of the standard, it
*cannot* return the memory to the OS; the definition of free simply does
not allow this.

That said, there are a lot of badly broken implementations out there
which do, in fact, return the memory to the OS, despite this making them
completely non-conforming. As to how _soon_ they do this, only the
implementer knows for certain.

Again, where is the stadard forbidding free() to release memory for
further allocation()? Chapter and verse, please.

How will the standard forbid another malloc() to request a block of
free space?

Remeber that the standard simply knows not that on the mashine
something like processes exists?

You likes here to say that malloc() should never reuse free'd memory
only because it was some time ago used. The standard does nowhere
speak about a process and usage of memory.

--
Tschau/Bye
Herbert

Visit http://www.ecomstation.de the home of german eComStation
eComStation 1.2R Deutsch ist da!

 

[Herbert Rosenau]

[snips]

A literal reading of the Standard's text supports Kelsey's
interpretation, but in practise I'd expect significantly sized
deallocations to be handed back to the OS, if there was a means to do
so.

[snip]

A literal reading of the standard's text says that allocated memory
properly free'd is made available for "further allocation". It most
certainly say "further allocation by the same program".

I assume you meant "does not say" there.

Since the term
"further allocation" is not defined or constrained by the standard in
any way, where exactly does it forbid that further allocation from being
to another executable, process, device driver, etc.?

The C standard defines the expected behaviour of a C program.

The C standard, in order to do this, also defines the expected behaviour
of the standard library functions, behaviours on which that C program can
rely.

The C standard defines free as releasing the memory to be made available
for further allocation.

That's the defined operation _in the context of defining the operation of
a C program and the standard library on which it relies_. There is
nothing in that definition which allows for any other behaviour, such as
handing the memory back to the OS.

Nor, as noted elsewhere, does the "as-if" rule give us an out; failure to
make the memory available for further allocation (eg by giving it back to
the OS) is a direct violation of the text of the standard and cannot come
under the as-if rule; the as-if rule allows flexibility in conforming
behaviour, it does not allow non-conforming behaviour to take the place
of conforming.

Let's turn this on its head. You say "It most certainly does not say
'further allocation by the same program'". Very good. Explain, then,
exactly what the standard is defining the behaviour of, *if not the very
program under discussion*?

Of course it's defining the behaviour of the program and only the
program; it has no ability to define behaviours beyond that. It cannot
mandate that the system actually use a hard disk, or a monitor, it cannot
mandate that "text mode" universally means using "\r\n" so any other
applications or OSen better wake up; it defines one thing and only one
thing, the behaviour of the C program written against that standard.

In order to do so, in order to define the behaviour of the C program, it
*also* has to define the behaviours of the standard library, on which the
C program may rely. It does so, in the case of free: the memory is made
available for further allocation.

Not to the OS, not to your toaster oven, not to some other program, the C
standard has nothing to say on these matters. No, the only thing under
examination, the only candidate which the standard *can* define such
behaviours for, is the C program. It defines just such a behaviour, for
that program, and that definition absolutely precludes the notion of
handing the memory back to the OS; doing so violates the defined
behaviour of free.

So, look on:

1 mashine will run only one single program. This single program
(commonly known as operation system) loads many different extensions -
you sees them as differen programs, but they are nothing else than
plugins like modern browsers use to extend theyr abilities.

So free() makes memory available for further allocation for what
extension it requests.

--
Tschau/Bye
Herbert

Visit http://www.ecomstation.de the home of german eComStation
eComStation 1.2R Deutsch ist da!

 

[CBFalconer]

.... snip ...

If I'm wrong, please point out the clause which allows the defined
behaviour to work other than as the standard defines it - you know,
the part where it says "implementation defined" or "undefined" or
even "unspecified" in the definition of free.

You can't; it's not there. What is there is a clear and absolute
definition of how free behaves in the context of a C program, the
very thing the standard exists to define. What is that behaviour?
Oh, yes, the memory is made available for further allocation. In
what context? Oh, yes, in the context of defining the behaviour of
the C program.

However, according to the standard, there is no other C program
competing. Thus the problem cannot arise.

 

[Paul Hsieh]

Show us where the standard says that an implementation in which
storage deallocated by free() is made available for further allocation
by another program invoked via the standard system() function would be
non-conforming.

system() does not need to invoke a conforming *C* program. So the
program it invokes could itself launch another program as a separate
concurrent process which could then allocate the memory freed to the
OS, then the first spawned program just returns. If the launched
process simply persists, then the memory will not be freed, and thus
the OS cannot soundly hand it back to the original program, in which
case Kelsey's reading indicates a non-conforming behavior.

Ok, the stupidity of the C language committee/standard aside, the real
world difficulty with this, is that handing memory back to the OS may
be slow, or simply be a step along the way to memory thrashing. If
the hand back is by simple rule or pattern of frees, then there will
usually will be easily constructed worst cases which cause an
arbitrarily high amount of thrashing. If you only handed back memory
after a exponentially increasing time intervals then that might be ok
-- but this will just keep the memory in the C program for
unnecessarily long amounts of time anyways.

I think the best alternative is that the program might "self-
benchmark" and just try to make sure the "return memory to OS" time
taken is no more than, say, 1% asymptotically.