How to handle huge array in java?

[wz]

Hi,

I have a java program which needs to manipulate a big 3D float array
(40x3000x3000).
By definition, it'll use 40x3000x3000x4=1.4GB memory, and my pc has only
512MB
physical RAM and I set 2GB page memory.

I tried the following steps:
1. run java with option -Xmx1440m, but it seems doing Garbage collection
when it used up physical ram,
and cpu usage drop to <7%, memory usage also drops to <100MB. It seems
hanging there forever.
2. added -Xnoclassgc (trying to disable garbage collection), but no help.
3. I thought about reducing the array size by using short instead of float,
but it's not acurate enough.

While I'm working on new algorithm to reduce the size of the array, it may
not be available soon.
Can anyone suggest me how to handle the big array with my current PC
setting?

Thanks a lot,

Wei

 

[Brad BARCLAY]

While I'm working on new algorithm to reduce the size of the array, it may
not be available soon.
Can anyone suggest me how to handle the big array with my current PC
setting?

Is every field initialized with a non-zero value? If not, you might be
better off implementing a sparse array, which only stores the non-zero
values.

Brad BARCLAY

 

[Michael Borgwardt]

2. added -Xnoclassgc (trying to disable garbage collection), but no help.

That option does not disable garbage collection in general (which you do NOT
want, it would just make your app crash with an OutOfMemoryError soon),
just the garbage collection of classes (which is rarely possible anyway).