PermGen, Garbage Collection and CPU utilization

[Holmbrew]

I am struggling with some performance issues that seem to be related
to garbage collection. We have a sort of homegrown scheduling
application the can have a pretty good memory foot print so we have
given it 3000m of heap space and recently upped it's PermGen
allocation to 256m. Overall the PermGen, which would fill up on us
over time as we loaded and unload object, has been really good, but it
seems strangely coincidental that not long after increasing the
PermGen allocation we started to see random period of high CPU
utilization that looks like "stop the world" garbage collection
because it does not coincide with any other actions or job running in
the affected JVM.

Can increasing PermGen change the way GC is handles by the JVM?

General Java & System Specs:
---------------------------------------------
JVM version is 1.5.0_03-b07
java.vm.vendor = Sun Microsystems Inc.
java.vm.name = Java HotSpot(TM) Server VM
OS: SunOS v5.9

Thanks,
Jason

 

[Dave Miller]

I am struggling with some performance issues that seem to be related
to garbage collection. We have a sort of homegrown scheduling
application the can have a pretty good memory foot print so we have
given it 3000m of heap space and recently upped it's PermGen
allocation to 256m. Overall the PermGen, which would fill up on us
over time as we loaded and unload object, has been really good, but it
seems strangely coincidental that not long after increasing the
PermGen allocation we started to see random period of high CPU
utilization that looks like "stop the world" garbage collection
because it does not coincide with any other actions or job running in
the affected JVM.

Can increasing PermGen change the way GC is handles by the JVM?

General Java & System Specs:
---------------------------------------------
JVM version is 1.5.0_03-b07
java.vm.vendor = Sun Microsystems Inc.
java.vm.name = Java HotSpot(TM) Server VM
OS: SunOS v5.9

Thanks,
Jason

If you've not seen it, this article, while old, lays it out pretty well:

http://java.sun.com/docs/hotspot/gc1.4.2/

 

[alexandre_paterson]

PermGen is a Sun VM specificity and there
are --albeit rare-- known and documented
very weird bugs related to the PermGen.

Hibernate + Sun VM + Tomcat can exhibit, under
special circumstances, this bug and has proved
to be a real headaches to diagnose. No easy
solution besides changing one of the components
because "it's nobody's fault" of course.

(the link should work)

http://www.jroller.com/agileanswers/entry/preventing_java_s_java_lang

My point is that maybe you're not doing anything
wrong and it's really a bug out of your control,
depending on the VM and 3rd party APIs you're using.

Have you tried with a non-Sun VM to see if it
could fix your problem?

 

[Holmbrew]

Thank you for the response. That is very interesting and actually
related more to a problem I had in a prior project and I have passed
that info along to folks doing the care and feeding these days.

I learned something this morning that should have beeen obvious, but
went overlooked and that is the distinction between Java Process
Memory and Java Heap Space. In my situation, Solaris 32-bit running on
Sparc hardware, any single process cannot take and or use more that
2Gb of physical memory. So therefore I cannot actually allocate the
Xmx3000m and that really becomes 2000m (max memory per process). So,
that would seem fine, but the problems occur when Java fills all the
allocated 'heap' space with objects and leaves nothing for any of the
other tasks that need to happen to keep the JVM happy and the process
running like creating threads and garbage collection and mundane
things like that.

When you read the Java garbage collection tuning material (http://
java.sun.com/docs/hotspot/gc5.0/gc_tuning_5.html) and it tells you:

"Unless you have problems with pauses, try granting as much memory as
possible to the virtual machine."

Make sure you know that there are caveats to that statement. "As much
memory as possible" does not mean as much memory as the OS will allow
you to grant. It really means do your homework and profile your
application to learn how it is using the memory it has been given.
Learn what its upper and lower limit regions of usage are and allocate
accordingly, if necessary. Java 1.5 or greater running on a server-
class machine will automatically calculate the memory it can allocate
to the Java heap, but it is capped by the HotSpot JVM to 1Gb no matter
the amount of physical memory available. This seems to make sense as
Java probably need the other gig available to the process to do the
other fun things it needs to do to stay happy and healthy.

The moral of the story is:

Java likes memory, but there are really two "heaps" to consider.

or

It pays to know your hardware as well as you know your software.


Lost in the heap,
Jason