Running out of memory gracefully

[thomas_okken]

I'm working on a rather memory-hungry application. There's not much I
can do to make the app use less memory, but I'm hoping that at least I
can find a way to handle low-memory situations better.
Currently, when memory runs out (typically as a result of a background
thread loading a huge table from a database), an OutOfMemoryError gets
thrown. There's no telling *where* this error gets thrown; if you're
lucky, it happens in the table-loading thread so table loading stops
but the rest of the app keeps working; if you're unlucky, the AWT Event
or Repaint threads die, and then you're stuck with a fatally hosed
application.

Years ago, when programming on the Macintosh, I used an OS function
called SetGrowZone() to register a callback, which the memory manager
would invoke when it was having trouble fulfilling a memory allocation
request. My approach was to allocate a big chunk of memory at
application startup time, and when my GrowZone callback got called, I
would release that chunk of memory to give the application some extra
breathing room, and simultaneously display a dialog box warning the
user that memory was getting dangerously low.

I'm trying to find a way to implement such a warning mechanism in Java,
but all I have come up with so far are "soft references". The problem
is that the Java API document only says "Virtual machine
implementations are [...] encouraged to bias against clearing
recently-created or recently-used soft references."
It would be more helpful if a reference type existed that would
guarantee that its referent is only deleted as a last resort; with such
a reference I could simply create, say, a 5-megabyte object, point such
a reference at it, and in its finalizer, pop up a warning message...
But I don't see a reliable way to achieve this. Do I really have no
choice but to wait until an OutOfMemoryError hits me in the face? (I
guess I could periodically call Runtime.gc() followed by checking
Runtime.freeMemory() + Runtime.maxMemory() - Runtime.totalMemory()...
Not very elegant!)

- Thomas

 

[abigale_carson]

Hi Thomas,

As far as I know, soft reference is your best bet. What you might do
is simply wait for a given reference to die, and then do the
freeMemory() + maxMemory() - totalMemory() trick.

Using soft references for this purpose has been discussed before, for
instance here:

http://groups.google.com/group/comp...nce+memory-management&rnum=1#8f57775430270c76

or, if that breaks,

http://linkfrog.net/dabp


or here:

http://groups.google.com/group/comp...nce+memory-management&rnum=2#f239782998c1c769

--or--

http://linkfrog.net/dabq

hope that helps, Abigale

 

[Andrew Thompson]

I'm working on a rather memory-hungry application. There's not much I
can do to make the app use less memory, but I'm hoping that at least I
can find a way to handle low-memory situations better.

....
I note you've struck upon the likely courses of action, for some
more detailed discussion along those lines, check this thread..
<http://groups.google.com/group/comp...ab2d76fc1de/fe7f2bb3eb99d626#fe7f2bb3eb99d626>

 

[thomas_okken]

OK, I guess SoftReference does deserve a closer look then. I'll just
try creating one and using a ReferenceQueue to find out when it is
cleared; if I find that the GC is overly aggressive in deleting it, I
can always fall back on the tired periodic-check approach, but if the
GC behaves like I hope it does, this should work well. (I'll also keep
the tip about pre-creating a warning dialog in mind!)
Thanks for the pointers!

- Thomas

 

[Thomas Hawtin]

I'm working on a rather memory-hungry application. There's not much I
can do to make the app use less memory, but I'm hoping that at least I
can find a way to handle low-memory situations better.
Currently, when memory runs out (typically as a result of a background
thread loading a huge table from a database), an OutOfMemoryError gets
thrown. There's no telling *where* this error gets thrown; if you're
lucky, it happens in the table-loading thread so table loading stops
but the rest of the app keeps working; if you're unlucky, the AWT Event
or Repaint threads die, and then you're stuck with a fatally hosed
application.

I suggest referencing non-essential, recreatable data through soft
references. On some condition you can determine that you have come
sufficiently close to running out of memory.

Years ago, when programming on the Macintosh, I used an OS function
called SetGrowZone() to register a callback, which the memory manager
would invoke when it was having trouble fulfilling a memory allocation
request. My approach was to allocate a big chunk of memory at
application startup time, and when my GrowZone callback got called, I
would release that chunk of memory to give the application some extra
breathing room, and simultaneously display a dialog box warning the
user that memory was getting dangerously low.

You get something like this for free. Before OutOfMemoryError is thrown,
soft and weak references will have been cleared. In any case,
OutOfMemoryError will still be thrown even while there is still quite a
lot of memory recovered. If there really was very little memory left,
the garbage collector would be running full time.

I'm trying to find a way to implement such a warning mechanism in Java,
but all I have come up with so far are "soft references". The problem
is that the Java API document only says "Virtual machine
implementations are [...] encouraged to bias against clearing
recently-created or recently-used soft references."
It would be more helpful if a reference type existed that would
guarantee that its referent is only deleted as a last resort; with such
a reference I could simply create, say, a 5-megabyte object, point such
a reference at it, and in its finalizer, pop up a warning message...
But I don't see a reliable way to achieve this. Do I really have no
choice but to wait until an OutOfMemoryError hits me in the face? (I
guess I could periodically call Runtime.gc() followed by checking
Runtime.freeMemory() + Runtime.maxMemory() - Runtime.totalMemory()...
Not very elegant!)

You could create a soft reference to a large area of memory and poll it
regularly. However, the information will be much the same as you get
from Runtme. You might as well use soft references for some useful data.

Tom Hawtin

 

[Roedy Green]

Currently, when memory runs out (typically as a result of a background
thread loading a huge table from a database), an OutOfMemoryError gets
thrown.

Can you estimate how much RAM you are going to need, then test to see
if it is possible with a giant buffer allocation you can catch and
deal with aborting the query, then free the buffer and carry on.

 

[Alun Harford]

I suggest referencing non-essential, recreatable data through soft
references. On some condition you can determine that you have come
sufficiently close to running out of memory.


You get something like this for free. Before OutOfMemoryError is thrown,
soft and weak references will have been cleared. In any case,
OutOfMemoryError will still be thrown even while there is still quite a
lot of memory recovered. If there really was very little memory left, the
garbage collector would be running full time.

While this will almost always work, and is possibly the best way to do it
(unless there's a way to use less memory), you should bear in mind that
according to the standard, if an object is only softly reachable, it can be
garbage collected. Theoretically, SoftReferences can be treated as
WeakReferences by the JVM, and using them like this could be dangerous.
Currently, however, Sun's JVM isn't this nasty - but beware if you want to
use a more obscure one.

Alun Harford

 

[steve]

I'm working on a rather memory-hungry application. There's not much I
can do to make the app use less memory, but I'm hoping that at least I
can find a way to handle low-memory situations better.
Currently, when memory runs out (typically as a result of a background
thread loading a huge table from a database), an OutOfMemoryError gets
thrown. There's no telling *where* this error gets thrown; if you're
lucky, it happens in the table-loading thread so table loading stops
but the rest of the app keeps working; if you're unlucky, the AWT Event
or Repaint threads die, and then you're stuck with a fatally hosed
application.

Years ago, when programming on the Macintosh, I used an OS function
called SetGrowZone() to register a callback, which the memory manager
would invoke when it was having trouble fulfilling a memory allocation
request. My approach was to allocate a big chunk of memory at
application startup time, and when my GrowZone callback got called, I
would release that chunk of memory to give the application some extra
breathing room, and simultaneously display a dialog box warning the
user that memory was getting dangerously low.

I'm trying to find a way to implement such a warning mechanism in Java,
but all I have come up with so far are "soft references". The problem
is that the Java API document only says "Virtual machine
implementations are [...] encouraged to bias against clearing
recently-created or recently-used soft references."
It would be more helpful if a reference type existed that would
guarantee that its referent is only deleted as a last resort; with such
a reference I could simply create, say, a 5-megabyte object, point such
a reference at it, and in its finalizer, pop up a warning message...
But I don't see a reliable way to achieve this. Do I really have no
choice but to wait until an OutOfMemoryError hits me in the face? (I
guess I could periodically call Runtime.gc() followed by checking
Runtime.freeMemory() + Runtime.maxMemory() - Runtime.totalMemory()...
Not very elegant!)

- Thomas

1. don't load the "huge" database, find another solution to access the data.
which database is it?
perhaps we can come up with a solution to "randomly access " the data you
need from the database ,as you need it.

have a quick look at:
http://www.forward.com.au/javaProgramming/javaGuiTips/errorRecovery.html

and the jedit source code.

but ultimately , the out of memory error , will hose your app.


Steve

 

[Steve W. Jackson]

I'm working on a rather memory-hungry application. There's not much I
can do to make the app use less memory, but I'm hoping that at least I
can find a way to handle low-memory situations better.
Currently, when memory runs out (typically as a result of a background
thread loading a huge table from a database), an OutOfMemoryError gets
thrown. There's no telling *where* this error gets thrown; if you're
lucky, it happens in the table-loading thread so table loading stops
but the rest of the app keeps working; if you're unlucky, the AWT Event
or Repaint threads die, and then you're stuck with a fatally hosed
application.

Years ago, when programming on the Macintosh, I used an OS function
called SetGrowZone() to register a callback, which the memory manager
would invoke when it was having trouble fulfilling a memory allocation
request. My approach was to allocate a big chunk of memory at
application startup time, and when my GrowZone callback got called, I
would release that chunk of memory to give the application some extra
breathing room, and simultaneously display a dialog box warning the
user that memory was getting dangerously low.

I'm trying to find a way to implement such a warning mechanism in Java,
but all I have come up with so far are "soft references". The problem
is that the Java API document only says "Virtual machine
implementations are [...] encouraged to bias against clearing
recently-created or recently-used soft references."
It would be more helpful if a reference type existed that would
guarantee that its referent is only deleted as a last resort; with such
a reference I could simply create, say, a 5-megabyte object, point such
a reference at it, and in its finalizer, pop up a warning message...
But I don't see a reliable way to achieve this. Do I really have no
choice but to wait until an OutOfMemoryError hits me in the face? (I
guess I could periodically call Runtime.gc() followed by checking
Runtime.freeMemory() + Runtime.maxMemory() - Runtime.totalMemory()...
Not very elegant!)

- Thomas

1. don't load the "huge" database, find another solution to access the data.
which database is it?
perhaps we can come up with a solution to "randomly access " the data you
need from the database ,as you need it.

have a quick look at:
<http://www.forward.com.au/javaProgramming/javaGuiTips/errorRecovery.html>

and the jedit source code.

but ultimately , the out of memory error , will hose your app.


Steve

The site at that link says that the OutOfMemoryError is "not
necessarily" fatal. But it's exceedingly difficult in most applications
to recover once it occurs. It's worth pursuing some of the many places
on the web where good information about JVM heap tuning can be obtained.

The OP should note that the table-loading example cited will *not* save
him from this error merely because it's in a background thread. This
error means that the entire JVM heap is full (or nearly so) and that,
after multiple attempts, sufficient memory cannot be allocated to it for
the task at hand to complete -- meaning creation of new objects called
for in his code. In order to allow that table load to completely
"contain" the error without any impact elsewhere in his app, it would
need to be offloaded to another JVM entirely. But, as suggested above,
pre-loading the entire table probably indicates that a redesign is in
order to get at the data.

The "years ago" example cited for Macintosh programming was, at best,
dubious in its day (and the new OS is entirely Unix-based and thus
doesn't suffer those problems any longer), but it doesn't translate into
Java due to the rather unique way that JVM memory is managed. I
personally would love to see Sun devise a new method of JVM memory
management that would enable it to take better advantage of the virtual
memory management schemes in use in virtually all modern operating
systems, rather than a closed heap with an artificial glass ceiling.

= Steve =

 

[Thomas Okken]

The SoftReferences approach works, but I had to refine it a bit: I
allocate a 5 megabyte chunk of memory, stick it in a SoftReference, and
then I monitor the ReferenceQueue to see when the chunk is deleted; at
that point, I enter a loop where I repeatedly check
Runtime.freeMemory() + RunTime.maxMemory() - RunTime.totalMemory() to
see whether it is safe to re-allocate the chunk (I consider it safe
when 10 megabytes is available).
Further, when the 5 MB chunk is deleted *and* there is insufficient
memory to re-allocate it immediately, the memory monitoring thread
pauses all active table-loading threads, and of course presents a
warning to the user.
Once enough free memory is available, start the whole process from the
beginning.

N.B. For reasons beyond the scope of this discussion, I have to load
entire tables at once; using more memory-efficient alternatives such as
scrollable result sets is not an option at this point. Fortunately, the
app already supports pausing and re-starting long table loads, so
suspending such loads programmatically in low-memory situations is easy
to implement and fits nicely within the existing design.

It should be noted that the JVM will clear SoftReferences even when it
doesn't have to. I noticed my 5 MB chunk getting cleared while my
program was trying to allocate 1 MB, with more than 15 MB of free
memory still being available. So, the SoftReference approach is better
than constantly polling free memory, but the fact that the reference is
cleared is *not* a reliable indication that memory is low. (Of course
as long as the reference is *not* cleared, that *is* a reliable
indication that memory is not low, and that's something.)
(All this probably depends on exactly which JVM you're using; I use
Sun's JDK 1.5.0_05 on Linux and Windows 2000 & XP).

So far my implementation of the SoftReference approach seems robust,
and I'm pretty happy with it. The Macintosh approach I mentioned in my
original post would still be preferable IMHO (but sadly Java does not
support this directly). I'm puzzled why Steve calls it "dubious"; I
would rather call it "perfect" because it provides a *reliable* way of
receiving advance warning that memory is running low -- and with zero
computational overhead. Of course it will only protect you as long as
you're trying to allocate memory in chunks that are smaller than the
"reserve" chunk, but in the case of my old Mac application, that was no
problem; I made the reserve 64kB, and in the one or two places where
memory was allocated in larger chunks, I just made sure to explicitly
check for allocation failures.

- Thomas