A solution for the allocation failures problem

K

Kelsey Bjarnason

1:
It is not possible to check EVERY malloc result within complex software.

Says who? There are a limited number of places memory is allocated, thus
a limited number of places checks need to be put in.
3:
A solution like the one proposed by Mr McLean (aborting) is not possible
for software quality reasons. The program must decide if it is possible
to just abort() or not.

Oh, this solution is always *possible*, but I'd be very much annoyed with
any developer stupid enough to adopt this as a default strategy in any
application I actually use.

Solution:

1) At program start, allocate a big buffer that is not used elsewhere in
the program.

Note that in at least one app, I do exactly this - and I *expect* that
the allocation will fail on a pretty regular basis. Of course, if I'm
stupid enough to be using something like Malcolm's xmalloc, it's too
late, the app has already bombed by the time I would normally just detect
the NULL and reduce the size of the request.

'Sides, it's a messy approach, one which ties up resources you don't
really need, resources best left alone for other things. Just because it
makes *your* app easier to write doesn't mean it's a good idea - ponder a
word processor grabbing 128MB for document space, with no documents
loaded, on an otherwise busy machine.
 
J

jacob navia

1:
It is not possible to check EVERY malloc result within complex software.

2:
The reasonable solution (use a garbage collector) is not possible for
whatever reasons.

3:
A solution like the one proposed by Mr McLean (aborting) is not
possible for software quality reasons. The program must decide
if it is possible to just abort() or not.

Solution:

1) At program start, allocate a big buffer that is not used
elsewhere in the program. This big buffer will be freed when
a memory exhaustion situation arises, to give enough memory
to the error reporting routines to close files, or otherwise
do housekeeping chores.

2) xmalloc()

static int (*mallocfailedHandler)(int);
void *xmalloc(size_t nbytes)
{
restart:
void *r = malloc(nbytes);
if (r)
return r;
// Memory exhaustion situation.
// Release some memory to the malloc/free system.
if (BigUnusedBuffer)
free(BigUnusedBuffer);
BigUnusedBuffer = NULL;
if (mallocfailedHandler == NULL) {
// The handler has not been set. This means
// this application does not care about this
// situation. We exit.
fprintf(stderr,
"Allocation failure of %u bytes\n",
nbytes);
fprintf(stderr,"Program exit\n");
exit(EXIT_FAILURE);
}
// The malloc handler has been set. Call it.
if (mallocfailedHandler(nbytes)) {
goto restart;
}
// The handler failed to solve the problem.
// Exit without any messages.
exit(EXIT_FAILURE);
}

4:
Using the above solution the application can abort if needed, or
make a long jump to a recovery point, where the program can continue.

The recovery handler is supposed to free memory, and reallocate the
BigUnusedBuffer, that has been set to NULL;
 
R

Richard Tobin

It is not possible to check EVERY malloc result within complex software.

I assume you're just setting up a scenario here rather than making
this claim yourself.
1) At program start, allocate a big buffer that is not used
elsewhere in the program. This big buffer will be freed when
a memory exhaustion situation arises, to give enough memory
to the error reporting routines to close files, or otherwise
do housekeeping chores.

This is not guaranteed to help with all malloc() implementations.
Some use different areas of memory for different sized blocks, and may
not have code to re-purpose the freed block. Or they may free the
memory back to the OS, in which case it may get gobbled up by
something else before they have the chance to re-use it.

-- Richard
 
V

vippstar

1:
It is not possible to check EVERY malloc result within complex software.
It is, with proper design it becomes easier too.
3:
A solution like the one proposed by Mr McLean (aborting) is not
possible for software quality reasons. The program must decide
if it is possible to just abort() or not.
That was not a solution.
Solution:

1) At program start, allocate a big buffer that is not used
elsewhere in the program. This big buffer will be freed when
a memory exhaustion situation arises, to give enough memory
to the error reporting routines to close files, or otherwise
do housekeeping chores.
That does not guarantee anything, it depends on the OS, and the
implementation of free/malloc.
Also it makes your program slower, and what if the big buffer
allocation fails?
Would you exit because your program failed to allocate resources it
would not use?
What if no allocation fails? The buffer is only a waste of resources.
2) xmalloc()

static int (*mallocfailedHandler)(int);
void *xmalloc(size_t nbytes)
{
restart:
void *r = malloc(nbytes);
if (r)
return r;
// Memory exhaustion situation.
// Release some memory to the malloc/free system.
if (BigUnusedBuffer)
free(BigUnusedBuffer);
free it anyway, free(NULL); does nothing.
BigUnusedBuffer = NULL;
if (mallocfailedHandler == NULL) {
// The handler has not been set. This means
// this application does not care about this
// situation. We exit.
fprintf(stderr,
"Allocation failure of %u bytes\n",
nbytes);
Undefined behavior, you pass size_t where a variadic function expects
unsigned int.
fprintf(stderr,"Program exit\n");
exit(EXIT_FAILURE);
}
What about previous allocations that have been done with xmalloc?
memory leak.
// The malloc handler has been set. Call it.
if (mallocfailedHandler(nbytes)) {
goto restart;
}
// The handler failed to solve the problem.
// Exit without any messages.
exit(EXIT_FAILURE);
Memory leak because you don't give the user a chance to free previous
allocations
The recovery handler is supposed to free memory, and reallocate the
BigUnusedBuffer, that has been set to NULL;
So if the allocation for BigUnusedBuffer succeeds but the allocation
after the callback fails, we will enter a loop of free-ing/allocating
the big buffer, great.

mr Jacob, I suggest you read a book or two on program structure &
designing.
 
C

CJ

1:
It is not possible to check EVERY malloc result within complex software.

2:
The reasonable solution (use a garbage collector) is not possible for
whatever reasons.

3:
A solution like the one proposed by Mr McLean (aborting) is not
possible for software quality reasons. The program must decide
if it is possible to just abort() or not.

Solution:

1) At program start, allocate a big buffer that is not used
elsewhere in the program. This big buffer will be freed when
a memory exhaustion situation arises, to give enough memory
to the error reporting routines to close files, or otherwise
do housekeeping chores.

2) xmalloc()

Why xmalloc? There's already a mechanism for dealing with such problems,
viz signals.

#include <signal.h>

void alloc_failure_handle(int x)
{
fputs("Dis baby gon crash and burn!\n", stderr);
exit(EXIT_FAILURE);
/* or insert your free-buffer-and-clean-up stuff if you prefer */
}

main()
{
signal(SIGSEGV, alloc_failure_handle);
/* ... */
}
 
C

cr88192

jacob navia said:
1:
It is not possible to check EVERY malloc result within complex software.

4:
Using the above solution the application can abort if needed, or
make a long jump to a recovery point, where the program can continue.

The recovery handler is supposed to free memory, and reallocate the
BigUnusedBuffer, that has been set to NULL;

5:
add exception handling (ie, as a library feature, or, possibly as a compiler
extension).

this way, we can throw, and see if the app has any nifty ideas...
I propose implementing something combining both signal handling and
unwinding features.

sza=(1<<31)-1;
while(1)
{
i=exBegin();
if(!i)
{
//try...
pa=malloc_throw(sza);
}else if(i==EX_NO_MEM)
{
//catch...
sza=(sza/3)*2;
continue;
}else
{
//cleanup...
exRelay(i); //relay to next handler
}
break;
}


or something...


(ok, nevermind that this idea is stupid, as directly handling the NULL
pointer is simpler).

sza=(1<<31)-1;
pa=malloc(sza);
while(!pa)
{
sza=(sza/3)*2;
pa=malloc(sza);
}
 
V

vippstar

mr Jacob, I suggest you read a book or two on program structure &
designing.

Here's a solution I propose:

Have a linked list for each allocation done and three functions,
--
xmalloc_start(); /* initializes a global linked list for future
allocations */
if(atexit(xmalloc_end)) /* handle error */
/* ... */
p = xmalloc(123, 0); /* false = 0, if the allocation fails, xmalloc
exits. */
s = xmalloc(123, 1); /* true = !0, if the allocation fails, xmalloc
returns NULL */
--
 
R

Richard Tobin

CJ said:
Why xmalloc? There's already a mechanism for dealing with such problems,
viz signals.
signal(SIGSEGV, alloc_failure_handle);

An ignored malloc() failure is not guaranteed to lead to a SIGSEGV.
It may just result in memory corruption.

-- Richard
 
S

santosh

CJ said:
Why xmalloc? There's already a mechanism for dealing with such
problems, viz signals.

#include <signal.h>

void alloc_failure_handle(int x)
{
fputs("Dis baby gon crash and burn!\n", stderr);

The Standard library functions are not guaranteed to be reentrant and
you cannot safely call them from a signal handler.
exit(EXIT_FAILURE);
/* or insert your free-buffer-and-clean-up stuff if you prefer */
}

main()
{
signal(SIGSEGV, alloc_failure_handle);
/* ... */
}

The state of a program after it has handled a SIGSEGV is undefined as
per the Standard.

IMHO, it's far better to handle out-of-memory conditions within normal
code rather than invoking all the complexity and fragility of signals.
 
S

santosh

Richard said:
I assume you're just setting up a scenario here rather than making
this claim yourself.


This is not guaranteed to help with all malloc() implementations.
Some use different areas of memory for different sized blocks, and may
not have code to re-purpose the freed block. Or they may free the
memory back to the OS, in which case it may get gobbled up by
something else before they have the chance to re-use it.

Yes. A better strategy is to use this buffer directly. No need to invoke
all the uncertainty involved in freeing and allocating again.
 
M

Malcolm McLean

jacob navia said:
3:
A solution like the one proposed by Mr McLean (aborting) is not
possible for software quality reasons. The program must decide
if it is possible to just abort() or not.
xmalloc() calls a caller-defined emergency function. It can do several
things, one of which is to simply exit. Don't confuse the requirement to
return the memory requested or (inherently) abort with the requirement to
abort.
 
B

Bartc

jacob said:
A solution for the allocation failures problem

You may have noticed this is somewhat controversial here. Doubt you will get
much agreement. It's one of those things where everyone has their own ideas.
 
E

Eric Sosman

jacob said:
1:
It is not possible to check EVERY malloc result within complex software.

Why not? It's no harder than checking every fopen() for
success -- or is that another impossibility?
2:
The reasonable solution (use a garbage collector) is not possible for
whatever reasons.

Garbage collection (when it works) eases the memory management
problem by relieving the programmer of the need to call free().
But collecting all the garbage does not imply that every malloc()
will succeed! In fact, a collector that cannot relocate the non-
garbage is likely to increase fragmentation and produce allocation
failures when a free()-as-soon-as-possible strategy would not.
3:
A solution like the one proposed by Mr McLean (aborting) is not
possible for software quality reasons. The program must decide
if it is possible to just abort() or not.

Solution:

1) At program start, allocate a big buffer that is not used
elsewhere in the program. This big buffer will be freed when
a memory exhaustion situation arises, to give enough memory
to the error reporting routines to close files, or otherwise
do housekeeping chores.

This is a reasonable thing to try, and has been tried
often. The hard part is choosing a value for "big:" too little
and there's not enough for the cleanup activity, too much and
you provoke allocation failures that don't need to happen.
4:
Using the above solution the application can abort if needed, or
make a long jump to a recovery point, where the program can continue.

The problems of longjmp() are well understood. It's possible
to use it effectively, but the programmers who write functions
"between" the setjmp() and the longjmp() must be constantly aware
that they might not get a chance to clean up:

char *this = dup(getenv("THIS"));
char *that = dup(getenv("THAT"));
printf ("THIS = %s, THAT = %s\n", this, that);
free (this);
free (that);

See the memory leak? If the second dup() eventually calls
longjmp() and returns to an ancestor of this code, the
memory allocated to `this' is never freed.

It's possible to work around this problem: various people
have put together macro packages that imitate a try/finally
discipline, for example. But the language itself gives little
help, and the compilers won't warn if somebody forgets (for
example, when a function that originally didn't need cleanup
acquires such a need during maintenance). Error recovery
based on longjmp() is do-able, but difficult.
 
S

Serve Laurijssen

Richard Bos said:
Bollocks from the start. Well done.

Richard

Think he meant it's not possible to HANDLE every malloc failure. Indeed its
pretty easy putting an if in there, but not so easy to actually gracefully
handle it.
 
K

Keith Thompson

Serve Laurijssen said:
Think he meant it's not possible to HANDLE every malloc
failure. Indeed its pretty easy putting an if in there, but not so
easy to actually gracefully handle it.

Yeah, programming is hard.
 
W

William Ahern

Think he meant it's not possible to HANDLE every malloc failure. Indeed its
pretty easy putting an if in there, but not so easy to actually gracefully
handle it.

If you write software not intending to recover from a malloc failure, then,
yes, upon inspection your code will prove to be very difficult to modify to
handle such recovery.

But, that's beside the point. If at outset you write your code intending to
handle recovery, its not difficult at all. I don't remember (and granted I
can't remember what I wrote when I first began programming in C) ever being
in a situation where I found it difficult to recover or unwind from a path
because of a failed malloc call. Of course, I have developed a very
structured, almost rote method for writing software which suits me. But I
did so by necessity, because from very early on I never accepted the premise
that memory failure could or should be ignored.

I originally learned to program with Perl, then JavaScript (Navigator 3.0).
Subsequently, to me memory management was a _feature_. I found it much more
satisfying to have the capacity (if not perfectly realized or utilized) to
write more resilient programs.

When I don't wish to exercise that feature, and if C is otherwise not
particularly suited to a task, I use another language.

Aside from glib, I can't off-hand think of any widely used Free Software C
library which doesn't check and propogate memory allocation failures.
 
M

Malcolm McLean

William Ahern said:
But, that's beside the point. If at outset you write your code intending
to
handle recovery, its not difficult at all. I don't remember (and granted I
can't remember what I wrote when I first began programming in C) ever
being in a situation where I found it difficult to recover or unwind from
a
path because of a failed malloc call. Of course, I have developed a very
structured, almost rote method for writing software which suits me. But I
did so by necessity, because from very early on I never accepted the
premise that memory failure could or should be ignored.
I wrote xmalloc() for Baby X, which is a minimalist X windows toolkit.
The problem is that the whole IDE is strung together with function pointers,
so there is no high level to propagate to. Most of the allocations are for
tiny structures associated with windows, anyway. It is also not clear what
you should do if unable to put up a window. My solution is to nag the user
for more memory until he gives it or kills the application.
 
J

Jack Klein

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

Forum statistics

Threads
473,744
Messages
2,569,484
Members
44,903
Latest member
orderPeak8CBDGummies

Latest Threads

Top