Garbage Collector Tuning?

[Will Hartung]

Anyone have any good tips on this? I've read the stuff on the Sun site, and
it's not the clearest in addressing the problem we're having.

The problem manifested itself when we tweaked a parameter on our Weblogic
container that boosted the number of Entity Beans we were caching.

Our first cut ran out of memory, and that's fine.

So, we doubled the memory from 512MB to 1G.

Now, what we're discovering is that when the total heap hits about 70%, the
JVM stops doing regular GC's and instead always does FULL GC's.

Here's an example:

49361.470: [GC 697385K->383772K(1038336K), 0.3217450 secs]
49718.707: [Full GC 712796K->343806K(1038336K), 9.3212136 secs]
50100.223: [GC 672830K->350047K(1038336K), 0.1127042 secs]
50602.456: [GC 679070K->355207K(1038336K), 0.1818174 secs]
51805.160: [GC 684230K->356492K(1038336K), 0.1795301 secs]
53811.129: [GC 685515K->361465K(1038336K), 0.1690929 secs]
54881.216: [GC 690489K->364970K(1038336K), 0.1771471 secs]
55140.403: [GC 693992K->375349K(1038336K), 0.3487647 secs]
55489.379: [Full GC 704372K->388449K(1038336K), 8.0318241 secs]
56094.146: [Full GC 717473K->401236K(1038336K), 7.9995729 secs]
56584.954: [Full GC 730260K->409971K(1038336K), 7.8904020 secs]
57313.878: [Full GC 738995K->375909K(1038336K), 9.7734903 secs]

Once it crossed that 700MB mark, it was always doing the Full GC's.
(Although it crossed 700MB earlier).

Any ideas which knob to turn that can prevent that/delay that? There's no
reason it should be using Full GC's at that point (that I can think off).

We understand where and how the memory is being used, it's the actual GC
behavior we are curious about.

Any thoughts appreciated.

Best regards,

Will Hartung
([email protected])

 

[Michael Borgwardt]

Anyone have any good tips on this? I've read the stuff on the Sun site, and
it's not the clearest in addressing the problem we're having.

The "Sun site" is pretty big, so maybe one of these is new and useful to you:

http://java.sun.com/docs/hotspot/gc1.4.2/
http://java.sun.com/docs/hotspot/VMOptions.html
http://java.sun.com/docs/hotspot/gc1.4.2/faq.html

 

[Jeff]

I found JRockit's documentation helpful.
http://e-docs.bea.com/wljrockit/docs142/userguide/memman.html
JRockit has some gc configuration options which I believe Sun has, but I'm
not sure.

 

[Kevin McMurtrie]

Anyone have any good tips on this? I've read the stuff on the Sun site, and
it's not the clearest in addressing the problem we're having.

The problem manifested itself when we tweaked a parameter on our Weblogic
container that boosted the number of Entity Beans we were caching.

Our first cut ran out of memory, and that's fine.

So, we doubled the memory from 512MB to 1G.

Now, what we're discovering is that when the total heap hits about 70%, the
JVM stops doing regular GC's and instead always does FULL GC's.

Here's an example:

49361.470: [GC 697385K->383772K(1038336K), 0.3217450 secs]
49718.707: [Full GC 712796K->343806K(1038336K), 9.3212136 secs]
50100.223: [GC 672830K->350047K(1038336K), 0.1127042 secs]
50602.456: [GC 679070K->355207K(1038336K), 0.1818174 secs]
51805.160: [GC 684230K->356492K(1038336K), 0.1795301 secs]
53811.129: [GC 685515K->361465K(1038336K), 0.1690929 secs]
54881.216: [GC 690489K->364970K(1038336K), 0.1771471 secs]
55140.403: [GC 693992K->375349K(1038336K), 0.3487647 secs]
55489.379: [Full GC 704372K->388449K(1038336K), 8.0318241 secs]
56094.146: [Full GC 717473K->401236K(1038336K), 7.9995729 secs]
56584.954: [Full GC 730260K->409971K(1038336K), 7.8904020 secs]
57313.878: [Full GC 738995K->375909K(1038336K), 9.7734903 secs]

Once it crossed that 700MB mark, it was always doing the Full GC's.
(Although it crossed 700MB earlier).

Any ideas which knob to turn that can prevent that/delay that? There's no
reason it should be using Full GC's at that point (that I can think off).

We understand where and how the memory is being used, it's the actual GC
behavior we are curious about.

Any thoughts appreciated.

Best regards,

Will Hartung
([email protected])

Some of the garbage collectors degrade over time due to incorrect
self-tuning.

The incremental collector is completely broken. At some point it will
fail because it stops adjusting heap sizes or the tenuring threshold
drops to zero. Either way it does full GCs and slows to a halt. This
collector has been removed from Java 1.5.

The CMS collector (-XX:+UseConcMarkSweepGC -XX:+UseParNewGC) in Java
1.4.2_05 and later works well long-term. Just don't drive the free
space to zero or you'll get a deadlock. The tenuring threshold degrades
to zero from poor self-tuning but that's not a major problem with this
collector. Self-tuning otherwise works well. The collection threshold
(-XX:CMSInitiatingOccupancyFraction) should be adjusted to make use of
large heaps.

I haven't done official long-term stress testing on the other
collectors, though I think I've seen the default collector go bad while
running Eclipse. CMS is probably the most appropriate for a web server.
Turn on the other GC debug options to see more about what's happening:
verbose:gc
-XX:+PrintGCTimeStamps
-XX:+PrintGCDetails
-XX:+PrintTenuringDistribution