jvm memory usage on Linux

[tlo]

Hi,

I have a program where the main program runs a TimerTask every 10 sec. I have
observed that the memory usage when running the commands " top" and "ps -aux"
on the Linux box slowly increases. Over the time span of 3 hours Linux reports
that the java process has grown approx 30kb. I have started the java process
with the options -Xmx32M and -Xms32M (as it is running on a small hardware
platfrom) and I can see that by using the Runtime.getRuntime() the jvm's memory
usage looks fine:

Total : 33357824
Max : 33357824
Free : 24832448
Used : 8525376

When looking at the top command following is displayed:
PID USER STATUS RSS PPID %CPU %MEM COMMAND
1544 root S 38180 1 0.0 32.2 java

And the ps command:
1544 root 38180 S java -Xmx32m -Xms32m main

There are a couple of things that I don't understand:

1. Why the Java process seems to increase in memory usage and it doesn't ever
decrease again. The program needs to run 24/7 and I'm not sure if it will do
this at the moment. Could this have something to do with Java program or could
it be related to the JVM. I have tried it with Java 1.5 and Java 1.6 on Debain
and another Linux distro with the same result.

2. Why Linux reports the java process uses 38180kb, when I have limited the
java heap to 32Mb.

3. Why it increaes the memory when Java clealy have enough, is this maybe
related to os resoures like files, sockets, etc?

Any Help would be greatly appriciated.

Thanks

TLO

 

[Knute Johnson]

Hi,

I have a program where the main program runs a TimerTask every 10 sec. I have
observed that the memory usage when running the commands " top" and "ps -aux"
on the Linux box slowly increases. Over the time span of 3 hours Linux reports
that the java process has grown approx 30kb. I have started the java process
with the options -Xmx32M and -Xms32M (as it is running on a small hardware
platfrom) and I can see that by using the Runtime.getRuntime() the jvm's memory
usage looks fine:

Total : 33357824
Max : 33357824
Free : 24832448
Used : 8525376

When looking at the top command following is displayed:
PID USER STATUS RSS PPID %CPU %MEM COMMAND
1544 root S 38180 1 0.0 32.2 java

And the ps command:
1544 root 38180 S java -Xmx32m -Xms32m main

There are a couple of things that I don't understand:

1. Why the Java process seems to increase in memory usage and it doesn't ever
decrease again. The program needs to run 24/7 and I'm not sure if it will do
this at the moment. Could this have something to do with Java program or could
it be related to the JVM. I have tried it with Java 1.5 and Java 1.6 on Debain
and another Linux distro with the same result.

2. Why Linux reports the java process uses 38180kb, when I have limited the
java heap to 32Mb.

3. Why it increaes the memory when Java clealy have enough, is this maybe
related to os resoures like files, sockets, etc?

Any Help would be greatly appriciated.

Thanks

TLO

30 kilobytes isn't very much memory. The garbage collector will run
when it needs to. You could set your JVM to use even less memory but
enough so that your program will run and then you will most likely see
the garbage collector run. Now of course this all assumes that you
don't have some sort of memory leak . So I wouldn't worry about it
until you try running it for a few days.

 

[Lew]

Because -Xmx only affects the heap, not the rest of the RAM used by the java
process.

 

[tlo]

30 kilobytes isn't very much memory. The garbage collector will run
when it needs to. You could set your JVM to use even less memory but
enough so that your program will run and then you will most likely see
the garbage collector run. Now of course this all assumes that you
don't have some sort of memory leak . So I wouldn't worry about it
until you try running it for a few days.

Thanks for your advise. I didn't mention that I also have tried using a
profiling tool and i could see that the garbage collector did run and that the
memory within the jvm got garbage collected. However, I'm still not sure about
the underlying operating system.

I have also tried running a perl script on the /proc/<pid>/maps within linux,
which lists the memory usage for the java process (found here
http://tree.celinuxforum.org/CelfPubWiki/RuntimeMemoryMeasurement). When the
memory usage increases, the following output from the perl script stays the
same, which also could indicate that maybe meassuring the memory using top or
ps isn't the bast way in linux. But I couldn't say that for sure.

Backed by file:
Executable r-x 15136
Write/Exec (jump tables) rwx 9208
RO data r-- 0
Data rw- 0
Unreadable --- 0
Unknown 0
Anonymous:
Writable code (stack) rwx 159612
Data (malloc, mmap) rw- 0
RO data r-- 0
Unreadable --- 772
Unknown 8

Do you have any hint to where I should consider looking for the memory leak?

Thanks

TLO

 

[tlo]

Because -Xmx only affects the heap, not the rest of the RAM used by the java
process.

Hi Lew,

Ok, that makes sense then. Also I assume that the jvm won't physically allocate
the entire 32mb when it is initially started, for performance reasons, is that
correct? Could this then have something to do with the constant increase, that
the heap is less then 32 mb and when it grows new memory is allocated from the
OS?

Thanks

TLO

 

[Lew]

Hi Lew,

Ok, that makes sense then. Also I assume that the jvm won't physically allocate
the entire 32mb when it is initially started, for performance reasons, is that correct?
No.

Could this then have something to do with the constant increase, that
the heap is less then 32 mb and when it grows new memory is allocated from the
OS?

You indicated that you're using java with the -Xms32M switch. That means that
you start with 32MB of heap. It's supposed to allocate that entire amount
right at the beginning.

 

[SadRed]

Hi,

I have a program where the main program runs a TimerTask every 10 sec. I have
observed that the memory usage when running the commands " top" and "ps -aux"
on the Linux box slowly increases. Over the time span of 3 hours Linux reports
that the java process has grown approx 30kb. I have started the java process
with the options -Xmx32M and -Xms32M (as it is running on a small hardware
platfrom) and I can see that by using the Runtime.getRuntime() the jvm's memory
usage looks fine:

Total : 33357824
Max : 33357824
Free : 24832448
Used : 8525376

When looking at the top command following is displayed:
PID USER STATUS RSS PPID %CPU %MEM COMMAND
1544 root S 38180 1 0.0 32.2 java

And the ps command:
1544 root 38180 S java -Xmx32m -Xms32m main

There are a couple of things that I don't understand:

1. Why the Java process seems to increase in memory usage and it doesn't ever
decrease again. The program needs to run 24/7 and I'm not sure if it will do
this at the moment. Could this have something to do with Java program or could
it be related to the JVM. I have tried it with Java 1.5 and Java 1.6 on Debain
and another Linux distro with the same result.

2. Why Linux reports the java process uses 38180kb, when I have limited the
java heap to 32Mb.

3. Why it increaes the memory when Java clealy have enough, is this maybe
related to os resoures like files, sockets, etc?

Any Help would be greatly appriciated.

Thanks

TLO

See http://mindprod.com/jgloss/sscce.html

 

[=?ISO-8859-1?Q?Bernhard_M=FCller?=]

1. Why the Java process seems to increase in memory usage and it doesn't ever
decrease again. The program needs to run 24/7 and I'm not sure if it will do
this at the moment. Could this have something to do with Java program or could
it be related to the JVM. I have tried it with Java 1.5 and Java 1.6 on Debain
and another Linux distro with the same result.

seems unlikely the VM or the OS are to blame, i'd say;

rather check lists, arrays, race conditions, native-interface calls and
the like in your application. Probably some references never get
released anymore, mem leaks are easier to avoid in J but not impossible.

There should be some useful java profilers around, you can use them to
check your heap.

bm

 

[wimpunk]

seems unlikely the VM or the OS are to blame, i'd say;

rather check lists, arrays, race conditions, native-interface calls and
the like in your application. Probably some references never get
released anymore, mem leaks are easier to avoid in J but not impossible.

There should be some useful java profilers around, you can use them to
check your heap.

bm

There's a way to specify to your VM how much memory it may use. What
you see is a quiet normal behaviour.