heap size, why?

[iksrazal]

I have an RMI program that transfers files without problems. Unless
when these files are over lets say 25megs. Then all of a sudden I get
an "out of memory" error. This Sun box has over 2gigs. Increasing the
heap size to merely 128m solves the problem. Less of course the the
time profiling, etc.

Why is this neccesary?

iksrazal

 

[A Bag Of Memes]

I have an RMI program that transfers files without problems. Unless
when these files are over lets say 25megs. Then all of a sudden I get
an "out of memory" error. This Sun box has over 2gigs. Increasing the
heap size to merely 128m solves the problem. Less of course the the
time profiling, etc.

Why is this neccesary?

Because your code uses more heap than the default heap size and less than
128M. The heap is where objects live. My guess is that you are reading the
files into memory before sending them out again. You might want to consider
streaming them through (read a little, write a little, repeat) if you want
it to work for arbitrarily large files.

 

[Roedy Green]

128m solves the problem. Less of course the the
time profiling, etc.

Why is this neccesary?

It takes considerable RAM to process and store a fat RMI object. You
have already filled RAM with various classes and variables. It simply
won't fit in the default allocation, even after garbage collection.
Fat objects are doubly difficult. They cannot exploit fragmented RAM.

Perhaps your question is, why does not the heap just grow dynamically
as needed until the VM fills the entire hard disk?

Chances are if it does, the application has gone mad. You don't want
programs that have no business exploding like that taking over your
machine.

So, if you want to permit that behaviour, you must explicitly allow it
on the command line.

My other thought is, if you are sending giant amounts of data like
that, perhaps you would be better off to send them via GZIPed socket
so they never have to exist entirely in RAM at either end.

see http:///mindprod.com/fileio.html for sample code how to do that.

Leave everything RMI, just do a special socket for this bulk transfer.

 

[Will Hartung]

I have an RMI program that transfers files without problems. Unless
when these files are over lets say 25megs. Then all of a sudden I get
an "out of memory" error. This Sun box has over 2gigs. Increasing the
heap size to merely 128m solves the problem. Less of course the the
time profiling, etc.

Why is this neccesary?

Because the system memory has nothing to do with the Java Heap memory,
though ideally the system memory will be greater than Java Heap memory.

You can have 100GB of memory, but Java has a static, preset view of how much
memory it can use, and it uses the -Xms and -Xmx parameters to set this
value. Java will not grow its heap beyond these specified numbers.

Note, that it will likely consume more system memory than the heap amount,
but the VM is fixed at startup as to how much memory it is allowed to use.

Regards,

Will Hartung
([email protected])

 

[Joseph Millar]

Because the system memory has nothing to do with the Java Heap memory,
though ideally the system memory will be greater than Java Heap memory.

Not entirely true. Some JVM's will use an algorithm that
takes the physical memory size as one variable to determine
what the default min and max heapsize is in the absence of
-Xms and -Xmx. The IBM JVM 1.3.1 on Win32 was this way on
systems with lots of physical memory. The default max
heapsize was amost the entire user virtual address space
(something like 1.5gb when everything was added up). This
is especially problematic when using the JVM inside of
another application via the Invocation API's.

In any event, my point is simply that whatever a particular
JVM's heap size defaults, it's going to differ from JVM
to JVM so you can't count on it being a certain way in
every JVM.

--Joe