Java at multi application servers

Discussion in 'Java' started by moin, Oct 14, 2005.

  1. moin

    moin Guest

    Hi,

    we develop a server application, that has the request for very flexible
    memory scalabilty: Normaly it uses only 200MiByte and sometimes up to
    3GiByte memory. The server (RS6000 / AIX5.1L / JDK1.4.2 (32Bit)) runs
    many other applications. Now the problem:

    After GC the up to 2,5GiByte free (Java-)heap memory is _not_ returned
    to the OS.

    We write a small test class to check this issue (see below). It showes
    (eg. svmon), that the physical memory is not released. We played around
    with the JVM options described in 'IBM JVM Garbage Collection and
    Storage Allocation techniques' without any success. We cross checked
    our test with the SUN JVM (Linux and Windows 2000) and it works as
    expected.

    The question:

    Does someone knows the correct parameter for the IBM JVM that enables
    the shrinkage of the native heap?
    Have we have to modify the AIX environment in a special way (eg
    MALLOCTYPE)?

    Bye Thomas

    -------------- Test class -----------------

    public class MemTest {
    public static void main(String argc[]) {
    java.util.ArrayList arrayList = new java.util.ArrayList();
    int allocateStep;
    int character;

    if (argc.length > 0) {
    System.out.println("Allocation Step: " + argc[0] + "MB");
    allocateStep = Integer.parseInt(argc[0]);
    } else {
    System.out.println("Allocation Step: 128MB");
    allocateStep = 128;
    }

    try {
    character = System.in.read();
    while (true) {
    switch (character) {
    case 'q':
    System.exit(0);
    break;

    case 'a':
    arrayList.add(allocate(allocateStep));
    printStorage();
    break;


    case 'd':
    if (arrayList.size() > 0)
    arrayList.remove(0);
    else
    System.out.println("All arrays
    deallocated");
    printStorage();

    break;

    case 'g':
    collectGarbage();
    printStorage();
    break;

    case 'l':
    printStorage();
    break;
    }
    character = System.in.read();
    }
    } catch (Throwable aThrowable) {
    aThrowable.printStackTrace();
    }
    }

    static void printStorage() {
    System.out.println("\n\nSpeicher (Max): " +
    Runtime.getRuntime().maxMemory());
    System.out.println("Speicher (Total): " +
    Runtime.getRuntime().totalMemory());
    System.out.println("Speicher (Frei): " +
    Runtime.getRuntime().freeMemory());
    System.out.println("\nSpeicher (Heap): " +
    (Runtime.getRuntime().totalMemory() -
    Runtime.getRuntime().freeMemory()));
    }

    static void collectGarbage() {
    Runtime.getRuntime().runFinalization() ;
    System.gc();
    }

    static byte[] allocate(int size) {
    return new byte[size * 1024 * 1024];
    }
    }
     
    moin, Oct 14, 2005
    #1
    1. Advertising

  2. moin

    dnasmars Guest

    hello,

    can you send the command line that launches
    the server application.
    In fact just the parameters of the jvm
    moin wrote:
    > Hi,
    >
    > we develop a server application, that has the request for very flexible
    > memory scalabilty: Normaly it uses only 200MiByte and sometimes up to
    > 3GiByte memory. The server (RS6000 / AIX5.1L / JDK1.4.2 (32Bit)) runs
    > many other applications. Now the problem:
    >
    > After GC the up to 2,5GiByte free (Java-)heap memory is _not_ returned
    > to the OS.
    >
    > We write a small test class to check this issue (see below). It showes
    > (eg. svmon), that the physical memory is not released. We played around
    > with the JVM options described in 'IBM JVM Garbage Collection and
    > Storage Allocation techniques' without any success. We cross checked
    > our test with the SUN JVM (Linux and Windows 2000) and it works as
    > expected.
    >
    > The question:
    >
    > Does someone knows the correct parameter for the IBM JVM that enables
    > the shrinkage of the native heap?
    > Have we have to modify the AIX environment in a special way (eg
    > MALLOCTYPE)?
    >
    > Bye Thomas
    >
    > -------------- Test class -----------------
    >
    > public class MemTest {
    > public static void main(String argc[]) {
    > java.util.ArrayList arrayList = new java.util.ArrayList();
    > int allocateStep;
    > int character;
    >
    > if (argc.length > 0) {
    > System.out.println("Allocation Step: " + argc[0] + "MB");
    > allocateStep = Integer.parseInt(argc[0]);
    > } else {
    > System.out.println("Allocation Step: 128MB");
    > allocateStep = 128;
    > }
    >
    > try {
    > character = System.in.read();
    > while (true) {
    > switch (character) {
    > case 'q':
    > System.exit(0);
    > break;
    >
    > case 'a':
    > arrayList.add(allocate(allocateStep));
    > printStorage();
    > break;
    >
    >
    > case 'd':
    > if (arrayList.size() > 0)
    > arrayList.remove(0);
    > else
    > System.out.println("All arrays
    > deallocated");
    > printStorage();
    >
    > break;
    >
    > case 'g':
    > collectGarbage();
    > printStorage();
    > break;
    >
    > case 'l':
    > printStorage();
    > break;
    > }
    > character = System.in.read();
    > }
    > } catch (Throwable aThrowable) {
    > aThrowable.printStackTrace();
    > }
    > }
    >
    > static void printStorage() {
    > System.out.println("\n\nSpeicher (Max): " +
    > Runtime.getRuntime().maxMemory());
    > System.out.println("Speicher (Total): " +
    > Runtime.getRuntime().totalMemory());
    > System.out.println("Speicher (Frei): " +
    > Runtime.getRuntime().freeMemory());
    > System.out.println("\nSpeicher (Heap): " +
    > (Runtime.getRuntime().totalMemory() -
    > Runtime.getRuntime().freeMemory()));
    > }
    >
    > static void collectGarbage() {
    > Runtime.getRuntime().runFinalization() ;
    > System.gc();
    > }
    >
    > static byte[] allocate(int size) {
    > return new byte[size * 1024 * 1024];
    > }
    > }
    >
     
    dnasmars, Oct 17, 2005
    #2
    1. Advertising

  3. moin

    Roedy Green Guest

    On Mon, 17 Oct 2005 09:44:40 +0200, dnasmars <> wrote
    or quoted :

    >> After GC the up to 2,5GiByte free (Java-)heap memory is _not_ returned
    >> to the OS.
    >>
    >> We write a small test class to check this issue (see below). It showes
    >> (eg. svmon), that the physical memory is not released.


    In talking about such problems you have to constantly keep mentioning
    whether you mean the working set RAM or the Virtual backing store
    allocated on disk.

    1. Do you have any reason to believe returning VM to the OS is the
    expected or possible behaviour?

    2. Do you have any reason to believe that would be a good thing to do.
    VM is just disk space. You are not really tying up anything that
    someone else is in desperate need of. There is an advantage in
    holding on -- keeping the VM contiguous and avoiding the cost of
    reallocating and deallocating.

    Nobody can really help you much since almost none of us have access to
    such a splendid server.

    --
    Canadian Mind Products, Roedy Green.
    http://mindprod.com Again taking new Java programming contracts.
     
    Roedy Green, Oct 19, 2005
    #3
  4. moin

    moin Guest

    Hi,

    simply:

    jvm -Xmx3000m -jar ServerApp.jar

    should do the job. The defaults descibed in 'IBM JVM Garbage Collection
    and
    Storage Allocation techniques' should do the job.

    Bye Thomas
     
    moin, Oct 20, 2005
    #4
  5. moin

    moin Guest

    Hi,

    Roedy Green schrieb:

    > On Mon, 17 Oct 2005 09:44:40 +0200, dnasmars <> wrote
    > or quoted :
    > In talking about such problems you have to constantly keep mentioning
    > whether you mean the working set RAM or the Virtual backing store
    > allocated on disk.

    Both. As long as the application is active the OS does not swap out any
    memory or
    constantly swaps in and out while the task switching.

    >
    > 1. Do you have any reason to believe returning VM to the OS is the
    > expected or possible behaviour?

    Yes, it is an expected behaviour. The IBM manual describes exactly
    under which condition it is done.

    >
    > 2. Do you have any reason to believe that would be a good thing to do.
    > VM is just disk space. You are not really tying up anything that
    > someone else is in desperate need of. There is an advantage in
    > holding on -- keeping the VM contiguous and avoiding the cost of
    > reallocating and deallocating.

    Normaly you are absolutely right. But, it is a good thing if you can
    control this process. The IBM jvm returns the 'Java heap garbage' only,
    if you call System.gc.

    Normaly an application can allocate 2GiByte for no cost. The
    application needs/holds no more (virtual) memory at that point. Not
    until the write access to a memory page the MMU asigns accessable
    memory to the application. The IBM jvm locks these memory pages all the
    time.

    BTW: The same IBM jvm running on Linux works as expected/described ...

    >
    > Nobody can really help you much since almost none of us have access to
    > such a splendid server.

    Sounds nice :)) but the 80 concurrent users (each with a horrable,
    memory consuming application) kill it.

    Bye Thomas

    BTW: I see, that I used the wrong group. I have to have post my problem
    into 'machine' not 'programmer'. Tsts ...
     
    moin, Oct 20, 2005
    #5
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Undecided
    Replies:
    0
    Views:
    351
    Undecided
    Apr 23, 2004
  2. Charles Handy

    Fast CGI Vs Java Application Servers

    Charles Handy, Jun 29, 2003, in forum: Python
    Replies:
    13
    Views:
    6,508
    Charlton Wilbur
    Jul 1, 2003
  3. Carl Waldbieser

    Python, COM Servers, and Multi-Threading

    Carl Waldbieser, Oct 11, 2005, in forum: Python
    Replies:
    1
    Views:
    388
    Steve Holden
    Oct 11, 2005
  4. Martin
    Replies:
    2
    Views:
    331
    Martin
    Jul 25, 2006
  5. Charles Handy

    Fast CGI Vs Java Application Servers

    Charles Handy, Jun 29, 2003, in forum: Perl Misc
    Replies:
    9
    Views:
    217
    Charlton Wilbur
    Jul 1, 2003
Loading...

Share This Page