Help:java.lang.OutOfMemoryError

C

ckumar

Hai guys,

I want to declare a huge two dimension array with
rows=99939263
cols=1606

when I tried to declare an array as follows

double arr[][];
arr=new double[rows][cols];

I am getting java.lang.OutOfMemoryError

So anyone tell me to handle this problem.

Is there any solution other than two dimension array?

Regards
ackumar
 
N

nobody

So anyone tell me to handle this problem.

It depends what the problem is that you're trying to solve. The
quickest solution to your problem, as stated, is to run the JVM with
more memory. Add the -mXms 256m option to your java run command to
increase the heap size that is used to 256M. I believe the default is 64M.

However, there may very well be a way for you to solve your problem
without using that data structure. Is it sparsely populated or are most
cells uses? Do cells get used and then outlive their use when others
come into use?

alan
 
C

ckumar

Hai,

I'll tell the need for that huge array.

I want to build a matrix, which is a very sparse one and calculate
Singular Vector Decomposition for that matrix.


Regards
ackumar
 
N

nobody

I don't know about Singular Vector Decomposition, but I do know you
shouldn't be using an array for this project if it's very sparsely
populated.

How about a HashMap? The key could be the address in the matrix and the
value is the double value (wrapped up as a Double). Wouldn't that
suffice? The hashmap need only be about twice as big as the number of
objects you intend to store in it.

alan
 
C

ckumar

Hai,

When I tried java -Xms256m <classname> I am getting the following error.

Error occurred during initialization of VM
Incompatible initial and maximum heap sizes specified


ackumar
 
P

Patricia Shanahan

ckumar said:
Hai,

I'll tell the need for that huge array.

I want to build a matrix, which is a very sparse one and calculate
Singular Vector Decomposition for that matrix.


Regards
ackumar

You should use a different data structure. Merely telling
the JVM to use more memory won't solve your problem. The
array is too large for anything other than a very large
server running a 64-bit version of Java.

If you already have a suitable algorithm using row,column
indexing (and that means, one designed for sparse data) you
can create a sparse-array class. Have get and set methods like:

void setElement(double value, int row, int column)

double getElement(int row, int column)

You could use a HashMap whose index is an object wrapping
the row and column together to store the non-zero elements.

The get method would return 0 for anything that isn't in the
Map. The set method would delete from the Map for a zero
value, but put an element in the Map for a non-zero value.

Depending on your algorithm, you may need to squish very
small values to zero, so that rounding errors don't lead to
too many unnecessary non-zero elements.

Patricia
 
T

Tilman Bohn

In message
Hai,

When I tried java -Xms256m <classname> I am getting the following error.

Error occurred during initialization of VM
Incompatible initial and maximum heap sizes specified

Your initial heap size of 256 MB is exceeding the maximum heap size,
which defaults to something smaller. Just increase it accordingly.
 
T

Tilman Bohn

In message <[email protected]>,
Tilman Bohn wrote on Fri, 25 Feb 2005 09:49:24 +0100:

[...]
Your initial heap size of 256 MB is exceeding the maximum heap size,
which defaults to something smaller. Just increase it accordingly.

PS: However, it would be much preferable to take Patricia Shanahan's
advice instead.
 
T

Thomas Weidenfeller

ckumar said:
I want to declare a huge two dimension array with
rows=99939263
cols=1606

when I tried to declare an array as follows

double arr[][];
arr=new double[rows][cols];

I am getting java.lang.OutOfMemoryError

Do that math. You are asking for at least 1.17 Terrabyte of memory. Is
your computer that large and can your VM handle that address range?
So anyone tell me to handle this problem.

Redesign your program.
Is there any solution other than two dimension array?

Since you didn't tell us at all what you want to do, all that can be
said is: maybe.

/Thomas
 
T

Thomas Weidenfeller

nobody said:
It depends what the problem is that you're trying to solve. The
quickest solution to your problem, as stated, is to run the JVM with
more memory. Add the -mXms 256m option to your java run command to
increase the heap size that is used to 256M. I believe the default is 64M.

I fail to see how this should give him the requested 1.17 Terrabyte.

/Thomas
 
A

Alun Harford

ckumar said:
Hai guys,

I want to declare a huge two dimension array with
rows=99939263
cols=1606

when I tried to declare an array as follows

double arr[][];
arr=new double[rows][cols];

I am getting java.lang.OutOfMemoryError

So anyone tell me to handle this problem.

Get a computer that with >1.17TB of memory and run the JVM
ith -Xmx1226900m
I believe the 64-bit JVM theoretically supports that flag.

Let me guess: a physicist? :)

Alun Harford
 
E

el goog

Consider representing your matrix using compressed row storage.

http://www.cs.utk.edu/~dongarra/etemplates/node373.html

Storage is now proportional to the number of non-zeros, any you
can perform traditional matrix operations (e.g., matrix-vector product)
efficiently. You still need a sparse SVD code, but at least now
you have a chance of solving a problem of this enormous size.
 
C

ckumar

Hai all,

First I thank everyone for ur suggetions.

I came across some math pacakages like JAMA,MTJ,JLAPACK for calculating
SVD. All these packages r using arrays to build the matrix and to
calculate SVD. That's Y I tried arrays.

As Patricia Shanahan told if try to use Hashset I want to look for some
suitable math packages.

I'll try to work on this if I get any solution, I'll post the solution. If
anyone know the solution post it.

Regards,
ackumar
 
C

ckumar

Hai all,

First I thank everyone for ur suggetions.

I came across some math pacakages like JAMA,MTJ,JLAPACK for calculating
SVD. All these packages r using arrays to build the matrix and to
calculate SVD. That's Y I tried arrays.

As Patricia Shanahan told if try to use Hashset I want to look for some
suitable math packages.

I'll try to work on this if I get any solution, I'll post the solution.
If
anyone know the solution post it.

Regards,
ackumar
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

Forum statistics

Threads
473,733
Messages
2,569,439
Members
44,829
Latest member
PIXThurman

Latest Threads

Top