GC performance

[ghostwhoowalks]

Hi All

We are trying to profile our software on a single server in order to
determine how many transactions per second it can handle. We were
trying to figure out what extent we can push it to. What we are
observing is that things run fine on the server for like an hour and
then everything goes haywire. We tried to dump the GC details and
figured that the full GC times are on an average 15 secs (snapshot
shown below). I am using the following VM options
-Xms1024M -Xmx4096M -XX:+UseConcMarkSweepGC -
XX:CMSInitiatingOccupancyFraction=50. How do I go about tuning this? I
am sure one of you might have more insight and point in the right
direction as to what the next steps should be for me. Please advice


GC details snapshot:

[Full GC [Tenured: 3478911K->3478911K(3478912K), 14.9602090 secs]
5044415K->5044415K(5044416K), [Perm : 14286K->14286K(21248K)],
14.9604460 secs]

[Full GC [Tenured: 3478911K->3478911K(3478912K), 14.9600190 secs]
5044415K->5044415K(5044416K), [Perm : 14286K->14286K(21248K)],
14.9602830 secs]

[Full GC [Tenured: 3478911K->3478911K(3478912K), 14.9943380 secs]
5044415K->5044415K(5044416K), [Perm : 14286K->14286K(21248K)],
14.9946070 secs]

[Full GC [Tenured: 3478911K->3478911K(3478912K), 15.0132030 secs]
5044415K->5044415K(5044416K), [Perm : 14286K->14286K(21248K)],
15.0135150 secs]

[Full GC [Tenured: 3478911K->3478911K(3478912K), 15.0087010 secs]
5044415K->5044415K(5044416K), [Perm : 14286K->14286K(21248K)],
15.0089700 secs]

[Full GC [Tenured: 3478911K->3478911K(3478912K), 15.0059100 secs]
5044415K->5044282K(5044416K), [Perm : 14286K->14286K(21248K)],
15.0062270 secs]

Exception in thread "ROW-MUTATION:7" java.lang.OutOfMemoryError: Java
heap space

[Full GC [Tenured: 3478911K->3478911K(3478912K), 15.0145700 secs]
5044415K->5044308K(5044416K), [Perm : 14286K->14286K(21248K)],
15.0148770 secs]

[Full GC [Tenured: 3478911K->3478911K(3478912K), 15.0113510 secs]
5044415K->5044330K(5044416K), [Perm : 14286K->14286K(21248K)],
15.0116270 secs]

[Full GC [Tenured: 3478911K->3478911K(3478912K), 15.0235560 secs]
5044415K->5044354K(5044416K), [Perm : 14286K->14286K(21248K)],
15.0238610 secs]

[Full GC [Tenured: 3478911K->3478911K(3478912K), 18.5820490 secs]
5044415K->5044372K(5044416K), [Perm : 14286K->14286K(21248K)],
18.5823870 secs]

[Full GC [Tenured: 3478911K->3478911K(3478912K), 14.9996820 secs]
5044415K->5044393K(5044416K), [Perm : 14286K->14286K(21248K)],
14.9999700 secs]


Thanks
A

 

[Daniel Pitts]

Hi All

We are trying to profile our software on a single server in order to
determine how many transactions per second it can handle. We were
trying to figure out what extent we can push it to. What we are
observing is that things run fine on the server for like an hour and
then everything goes haywire. We tried to dump the GC details and
figured that the full GC times are on an average 15 secs (snapshot
shown below). I am using the following VM options
-Xms1024M -Xmx4096M -XX:+UseConcMarkSweepGC -
XX:CMSInitiatingOccupancyFraction=50. How do I go about tuning this? I
am sure one of you might have more insight and point in the right
direction as to what the next steps should be for me. Please advice


GC details snapshot:

[Full GC [Tenured: 3478911K->3478911K(3478912K), 14.9602090 secs]
5044415K->5044415K(5044416K), [Perm : 14286K->14286K(21248K)],
14.9604460 secs]
[snip]
Exception in thread "ROW-MUTATION:7" java.lang.OutOfMemoryError: Java
heap space

[Full GC [Tenured: 3478911K->3478911K(3478912K), 15.0145700 secs]
5044415K->5044308K(5044416K), [Perm : 14286K->14286K(21248K)],
15.0148770 secs]

[Full GC [Tenured: 3478911K->3478911K(3478912K), 15.0113510 secs]
5044415K->5044330K(5044416K), [Perm : 14286K->14286K(21248K)],
15.0116270 secs]

[snip]

Sounds a lot like a memory leak to me. If you have any object that
lives for a long time, make sure that it isn't holding any reference to
any objects for a long time. Garbage collection can only reclaim an
object if there is no path to it.

Baring that, there are tools out there that will produce and analyze a
heap dump for you. I don't know any of the top of my head, but google
for Java heap dump, and I bet you'll find one.

Hope this helps,
Daniel.