OutOfMemoryError: heap size, multiple files with allocated memory 1GB

A

Alessandro

I'm running a java application which processes some million record
files (average size per file about 200 MB).

JVM is started declaring -Xms32m -Xmx1024m on a server well provided
of memory and there isn't any problem with only one file.

Actually I load record by record to memory using an array list object.

I have problems with 2 files ... first is well loaded, the second one
gives me the classic error OutOfMemory: heap size

Is it sufficient to call a .clear() on the array list before loading
into memory ?
The strange issue is that the array list isn't a global variable,
loads only all records but for a single file ... and the new()
operator should create a new object (clean) with the same name,
garbage collector should erase the old one


//START
private void adjust(String fileName, String record)
{
ArrayList<String[]> recordsToAdjust=new ArrayList<String[]>();
String[] values;


//load file into memory....


}


//END

Anything worng ??

Thanks for suggestions/explanation
Best rgds,
Ale
 
J

John B. Matthews

[...]
Actually I load record by record to memory using an array list
object.

I have problems with 2 files ... first is well loaded, the second one
gives me the classic error OutOfMemory: heap size

Is it sufficient to call a .clear() on the array list before loading
into memory? The strange issue is that the array list isn't a global
variable, loads only all records but for a single file ... and the
new() operator should create a new object (clean) with the same name,
garbage collector should erase the old one.
[...]

Assuming you mean ArrayList, I have never seen clear() leak, but it's
fairly easy to (inadvertently) retain other references to list elements
after the list is cleared:

<http://mindprod.com/jgloss/packratting.html>
<http://java.sun.com/javase/6/docs/api/java/util/ArrayList.html>

The NetBeans (or similar) memory profiler is very useful for examining
such problems.
 
T

Tom Anderson

Is it sufficient to call a .clear() on the array list before loading
into memory ?
The strange issue is that the array list isn't a global variable,
loads only all records but for a single file ... and the new()
operator should create a new object (clean) with the same name,
garbage collector should erase the old one

If you're throwing the list away each time (which is a good idea), then
you don't need to clear it (and in fact doing so would be a bad idea), and
there is no possible way that it can be holding on to objects.

Thus, there are two possibilities. One is that you're not actually
throwing the list away. The other is that something other than the list is
holding on to objects.
//START
private void adjust(String fileName, String record)
{
ArrayList<String[]> recordsToAdjust=new ArrayList<String[]>();
String[] values;


//load file into memory....


}

//END

Anything worng ??

Yeah, you've left out all the code in the middle that might actually give
us a clue what's going wrong. Try again.

tom
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

Forum statistics

Threads
473,744
Messages
2,569,484
Members
44,903
Latest member
orderPeak8CBDGummies

Latest Threads

Top