How to free ALL heap memory?

[Filippo Venturi]

Thanks for your attention..

I've wrote a long statistical procedure in C language, involving tre
creation and the manipulation of big matrix of double float (about 64
per 500 cells per matrix.

These matrixes are created via malloc function since their size
depends on the input data file.

The function works fine if used once in a main file.

The problem is, if i call it 4 o more times in a FOR structure in the
main file,
the executable crashes in the fourth iteration since it can't
dinamically allocate the requested memory.

The function frees the used heap(?) space requested at the end of each
call, so i expect than if there is enough memory to run the procedure
once, and *if* *all* of the memory allocated is then freed, there
should be enough memory to run the procedure other times.

So I guess there is some memory that is still used by the previous
iteration.

Is there a way (a simple way would be even better ) (going into
debugging is not easy to me) to prove my guess is right or wrong?
maybe using function from stdlib (stackavail, memavl , memmax..)? And
finally is there a way to clear the heap (some kind of free(ALL) ) and
does it have any particular side-effect?

I'm compiling with Digital Mars Compiler via command line under
Windows XP with -mn memory model options

Thanks a lot in advance, and sorry if I've been too long or vague

Filippo

 

[Eric Sosman]

Thanks for your attention..

I've wrote a long statistical procedure in C language, involving tre
creation and the manipulation of big matrix of double float (about 64
per 500 cells per matrix.

These matrixes are created via malloc function since their size
depends on the input data file.

The function works fine if used once in a main file.

The problem is, if i call it 4 o more times in a FOR structure in the
main file,
the executable crashes in the fourth iteration since it can't
dinamically allocate the requested memory.

The function frees the used heap(?) space requested at the end of each
call, so i expect than if there is enough memory to run the procedure
once, and *if* *all* of the memory allocated is then freed, there
should be enough memory to run the procedure other times.

So I guess there is some memory that is still used by the previous
iteration.

Is there a way (a simple way would be even better ) (going into
debugging is not easy to me) to prove my guess is right or wrong?
maybe using function from stdlib (stackavail, memavl , memmax..)? And
finally is there a way to clear the heap (some kind of free(ALL) ) and
does it have any particular side-effect?

I'm compiling with Digital Mars Compiler via command line under
Windows XP with -mn memory model options

Thanks a lot in advance, and sorry if I've been too long or vague

Filippo

Part of your problem is Question 18.2 in the comp.lang.c
Frequently Asked Questions (FAQ) list

http://www.eskimo.com/~scs/C-faq/top.html

As for free(ALL), a little thought should convince you that
it would be a truly terrible idea, and would make dynamic memory
completely unreliable and hence completely unusable.

 

[Morris Dovey]

Thanks for your attention..

I've wrote a long statistical procedure in C language, involving tre
creation and the manipulation of big matrix of double float (about 64
per 500 cells per matrix.

These matrixes are created via malloc function since their size
depends on the input data file.

The function works fine if used once in a main file.

The problem is, if i call it 4 o more times in a FOR structure in the
main file,
the executable crashes in the fourth iteration since it can't
dinamically allocate the requested memory.

The function frees the used heap(?) space requested at the end of each
call, so i expect than if there is enough memory to run the procedure
once, and *if* *all* of the memory allocated is then freed, there
should be enough memory to run the procedure other times.

So I guess there is some memory that is still used by the previous
iteration.

Is there a way (a simple way would be even better ) (going into
debugging is not easy to me) to prove my guess is right or wrong?
maybe using function from stdlib (stackavail, memavl , memmax..)? And
finally is there a way to clear the heap (some kind of free(ALL) ) and
does it have any particular side-effect?

Filippo...

No help from the standard library, I'm afraid.

All memory will be recovered following a call to exit() or a
return from main(). Have you considered using system() to invoke
another program that performs a single iteration and exits,
leaving its results in a file?

It might be quite a bit slower; but it's guaranteed to be faster
than crashing.

 

[Derk Gwen]

# > Thanks for your attention..
# >
# > I've wrote a long statistical procedure in C language, involving tre
# > creation and the manipulation of big matrix of double float (about 64
# > per 500 cells per matrix.
# >
# > These matrixes are created via malloc function since their size
# > depends on the input data file.
# >
# > The function works fine if used once in a main file.
# >
# > The problem is, if i call it 4 o more times in a FOR structure in the
# > main file,
# > the executable crashes in the fourth iteration since it can't
# > dinamically allocate the requested memory.

A sneaky trick used by some program is to execute part of the problem with a child
process (perhaps using system(...)), and when the child exits the system recovers
all of its memory automatically. Keep generating children for parts of the program
until all parts have been executed.

 

[Gordon Burditt]

The function frees the used heap(?) space requested at the end of each

call, so i expect than if there is enough memory to run the procedure
once, and *if* *all* of the memory allocated is then freed, there
should be enough memory to run the procedure other times.

So I guess there is some memory that is still used by the previous
iteration.

Is there a way (a simple way would be even better ) (going into
debugging is not easy to me) to prove my guess is right or wrong?
maybe using function from stdlib (stackavail, memavl , memmax..)? And
finally is there a way to clear the heap (some kind of free(ALL) ) and
does it have any particular side-effect?

A function that did free(ALL) would free memory not only that YOU
allocated, but that the C library allocated on your behalf. This
might include, for example, buffers for stdout or open files. Free
those, and you're invoking the wrath of undefined behavior. There
is no requirement that the C library document its use of malloc()ed
memory, but a function that truly freed all malloc()ed memory would
be unusable without some assurance that you weren't pulling the rug
out from under yourself in the process. (There is no guarantee,
for example, that "the stack" (if your implementation has one) isn't
initially created by a call to malloc(). I've never seen an
implementaion that did that, but it's not prohibited.)

Gordon L. Burditt

 

[Filippo Venturi]

Thanks for your attention..

I've wrote a long statistical procedure in C language, involving tre
creation and the manipulation of big matrix of double float (about 64
per 500 cells per matrix.

These matrixes are created via malloc function since their size
depends on the input data file.

The function works fine if used once in a main file.

The problem is, if i call it 4 o more times in a FOR structure in the
main file,
the executable crashes in the fourth iteration since it can't
dinamically allocate the requested memory.

Derk Gwen wrote:

A sneaky trick used by some program is to execute part of the problem
with a child
process (perhaps using system(...)), and when the child exits the
system recovers
all of its memory automatically. Keep generating children for parts of
the program
until all parts have been executed.
--


(e-mail address removed) (Gordon Burditt)
There

is no requirement that the C library document its use of malloc()ed
memory, but a function that truly freed all malloc()ed memory would
be unusable without some assurance that you weren't pulling the rug
out from under yourself in the process. (There is no guarantee,
for example, that "the stack" (if your implementation has one) isn't
initially created by a call to malloc(). I've never seen an
implementaion that did that, but it's not prohibited.)

Gordon L. Burditt

Thanks for all the replies.

I modified the software, changing the function calls in system() call;
so i've overcome most of the difficulties i met before.
In some case, however, even the system() call crashes, during one of
the last malloc call.
The malloc does not return any message or NULL pointer, it just
freezes the shell.
I believe it's some sort of bug of my compiler or with the operating
system, because it crashes during calls which needs much less memory
than other succesful calls (i.e. elaborating fewer items);
my enviroment is the following:

Windows XP Professional, with all the updates delivered by Microsoft
except Service Pack 1 (but i'm installing it in a few days)

Digital Mars Compiler, latest version (8.36 i believe), win32 memory
model (-mn)

P4mobile 1.8 Ghz

224 Mb Ram


The same source code works fine under

Windows XP Pro with SP1
Same compiler
512 Mb Ram

but crashes in the same way (i.e. in the same instruction if using the
same arguments) under:

Windows XP Embedded Evaluation Build with Servce Pack 1
same compiler
248 Mb

except that here an error windows appear, telling me that a particular
address could not be written.

Any Clue?


Thanks


Filippo Venturi

 

[Gordon Burditt]

Thanks for all the replies.

I modified the software, changing the function calls in system() call;
so i've overcome most of the difficulties i met before.
In some case, however, even the system() call crashes, during one of
the last malloc call.
The malloc does not return any message or NULL pointer, it just
freezes the shell.

In that case, you most likely have a bug in your program. This
problem is usually caused by using more memory than you allocated,
(DID YOU LEAVE SPACE FOR THE '\0' AT THE END OF THAT STRING?)
array subscripts out of range, or dereferencing uninitialized
pointers. The point of the freezeup need not be anywhere near the
point of the bug.

I believe it's some sort of bug of my compiler or with the operating
system, because it crashes during calls which needs much less memory
than other succesful calls (i.e. elaborating fewer items);

As bad as Microsoft software is, I don't think they got this part
wrong. If you trash the malloc() arena, free() can crash, too,
even when you pass it a perfectly legitimate pointer.

my enviroment is the following:

Windows XP Professional, with all the updates delivered by Microsoft
except Service Pack 1 (but i'm installing it in a few days)

Please be sure that the versions of third-party viruses you are
running are compatible with the versions of the core XP viruses in
XP. Things get particularly nasty if you try to violate the
Digital Rights Management rules of Blaster.

Digital Mars Compiler, latest version (8.36 i believe), win32 memory
model (-mn)

P4mobile 1.8 Ghz

224 Mb Ram


The same source code works fine under

Windows XP Pro with SP1
Same compiler
512 Mb Ram

but crashes in the same way (i.e. in the same instruction if using the
same arguments) under:

Windows XP Embedded Evaluation Build with Servce Pack 1
same compiler
248 Mb

except that here an error windows appear, telling me that a particular
address could not be written.

Any Clue?

Your program is buggy. One of the characteristics of bugs is that
they don't have to behave consistently as far as observable problems
are concerned. The contents of memory you don't own and the use
of it and how much it causes problems when you scribble on it can
vary considerably between systems or even apparently-identical
executions of the same program on the same machine.

Gordon L. Burditt