Help: Tomcat crashes on java.lang.OutOfMemoryError

Discussion in 'Java' started by Cathy Hui, May 31, 2005.

  1. Cathy  Hui

    Cathy Hui Guest

    I am running an application (i.e. Remedy Midtier) on Tomcat. Currently
    I run into the "java.lang.OutOfMemoryError" when performing some
    activities from the browser when accessing this application.
    Basically, the Tomcat crashes when the java process takes about 160MB -
    190MB memory usage (I did monitor the "top" for system resources)

    Background information:

    OS: Solaris 8
    System physical memory: 2048MB
    Tomcat version: 4.1.29
    JDK version: 1.4.1

    /etc/system

    set rlim_fd_cur = 65536
    set rlim_fd_max = 65536

    This is what I have tried so far:

    1) Double the heap size in catalina.sh to allocate to java from
    "-Xms256m -Xmx512m" to "-Xms512m -Xmx1024m"

    2) install jdk 1.4.2

    However, the result is: JAVA still fails after taking 160MB-190MB
    memory and Tomcat crashes on java.lang.OutOfMemoryError.

    Does anyone know what might be the cause of it and how to fix it?

    Thanks!
     
    Cathy Hui, May 31, 2005
    #1
    1. Advertising

  2. Cathy Hui schreef:
    > I am running an application (i.e. Remedy Midtier) on Tomcat. Currently
    > I run into the "java.lang.OutOfMemoryError" when performing some
    > activities from the browser when accessing this application.
    > Basically, the Tomcat crashes when the java process takes about 160MB -
    > 190MB memory usage (I did monitor the "top" for system resources)
    >


    Hello,

    You might try to use a profiler.
    I'm recently used JProfiler (http://www.jprofiler.com) that was quite
    easy to use.
    You can view the heap-memory and see what objects are created and/or
    garbage collected. If there's steady increase, this might indicate a
    memory-leak.

    Also, you might look into the source code for code that might use a lot
    of memory without freeing it (well pretty obvious maybe). Static objects
    (lists) might be a problem. Also, objects that are stored in sessions
    could be a problem, if there are a lot of users to the webapplication.
     
    Gerbrand van Dieijen, May 31, 2005
    #2
    1. Advertising

  3. Cathy  Hui

    Cathy Hui Guest

    More info: This is the catalina.out with the error log. Any idea of
    why it's failing?

    ====================================================

    loaded libraryException in thread "ARTID~0" java.lang.OutOfMemoryError
    Exception in thread "ARTID~0" java.lang.OutOfMemoryError
    Exception in thread "ARTID~0" java.lang.OutOfMemoryError
    Exception in thread "ARTID~0" java.lang.OutOfMemoryError
    Exception in thread "ARTID~0" java.lang.OutOfMemoryError
    java.lang.OutOfMemoryError
    Exception in thread "ARTID~0" java.lang.OutOfMemoryError
    java.lang.OutOfMemoryError
    Exception in thread "ARTID~0" java.lang.OutOfMemoryError
    Exception in thread "ARTID~0" java.lang.OutOfMemoryError
    Exception in thread "ARTID~0" java.lang.OutOfMemoryError
    Exception in thread "ARTID~0" java.lang.OutOfMemoryError
    Exception in thread "ARTID~0" java.lang.OutOfMemoryError
    Exception in thread "ARTID~0" java.lang.OutOfMemoryError
    Exception in thread "ARTID~0" java.lang.OutOfMemoryError
    Exception in thread "ARTID~0" java.lang.OutOfMemoryError
    Exception in thread "ARTID~0" java.lang.OutOfMemoryError
    Exception in thread "ARTID~0" java.lang.OutOfMemoryError
    Exception in thread "ARTID~0" java.lang.OutOfMemoryError

    Unexpected Signal : 11 occurred at PC=0xFE1A03FC
    Function=[Unknown. Nearest: JVM_Clone+0xD9DC]
    Library=/usr/j2se/jre/lib/sparc/client/libjvm.so

    Current Java thread:
    at com.remedy.arsys.api.Proxy.ARGetListActiveLinkObjects(Native
    Method)
    at
    com.remedy.arsys.api.ActiveLinkFactory.findObjects(ActiveLinkFactory.java:142)
    at
    com.remedy.arsys.cache.definition.ActiveLinkCache.getActiveLinkListFromServer(Unknown
    Source)
    at
    com.remedy.arsys.cache.definition.ActiveLinkCache.populateCacheWithDefinitions(Unknown
    Source)
    at
    com.remedy.arsys.cache.definition.ActiveLinkCache.getActiveLinkList(Unknown
    Source)
    at
    com.remedy.arsys.cache.definition.ActiveLinkCache.preLoadObjects(Unknown
    Source)
    at com.remedy.arsys.cache.Cache.preLoadObjects(Unknown Source)
    at
    com.remedy.arsys.cache.definition.SchemaCache.CreateSchemaDefinitions(Unknown
    Source)
    at
    com.remedy.arsys.cache.definition.SchemaCache.getSchemaDefinition(Unknown
    Source)
    at
    com.remedy.arsys.cache.definition.SchemaCache.getSchemaViewDescriptorMap(Unknown
    Source)
    at
    com.remedy.arsys.cache.Cache.getSchemaViewDescriptorMap(Unknown Source)
    at com.remedy.arsys.appmgr.Application.getLocaleList(Unknown
    Source)
    at com.remedy.arsys.appmgr.Application.deployFromCache(Unknown
    Source)
    - locked <f2f77248> (a com.remedy.arsys.appmgr.Application)
    at com.remedy.arsys.appmgr.AppManager.deployApp(Unknown Source)
    - locked <f2d75508> (a com.remedy.arsys.appmgr.AppManager)
    at
    com.remedy.arsys.config.ConfigServlet.deployApplications(Unknown
    Source)
    at
    com.remedy.arsys.config.ConfigServlet.deployApplication(Unknown Source)
    at com.remedy.arsys.config.ConfigServlet.doPost(Unknown Source)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:760)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:853)
    at
    org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:247)
    at
    org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193)
    at
    org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:256)
    at
    org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invokeNext(StandardPipeline.java:643)
    at
    org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:480)
    at
    org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:995)
    at
    org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
    at
    org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invokeNext(StandardPipeline.java:643)
    at
    org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:480)
    at
    org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:995)
    at
    org.apache.catalina.core.StandardContext.invoke(StandardContext.java:2417)
    at
    org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:180)
    at
    org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invokeNext(StandardPipeline.java:643)
    at
    org.apache.catalina.valves.ErrorDispatcherValve.invoke(ErrorDispatcherValve.java:171)
    at
    org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invokeNext(StandardPipeline.java:641)
    at
    org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:172)
    at
    org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invokeNext(StandardPipeline.java:641)
    at
    org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:480)
    at
    org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:995)
    at
    org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:174)
    at
    org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invokeNext(StandardPipeline.java:643)
    at
    org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:480)
    at
    org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:995)
    at
    org.apache.ajp.tomcat4.Ajp13Processor.process(Ajp13Processor.java:457)
    at
    org.apache.ajp.tomcat4.Ajp13Processor.run(Ajp13Processor.java:576)
    at java.lang.Thread.run(Thread.java:536)

    Dynamic libraries:
    0x10000 /usr/j2se/bin/java
    0xff360000 /usr/lib/libthread.so.1
    0xff3a0000 /usr/lib/libdl.so.1
    0xff200000 /usr/lib/libc.so.1
    0xff330000 /usr/platform/SUNW,Serverblade1/lib/libc_psr.so.1
    0xfe000000 /usr/j2se/jre/lib/sparc/client/libjvm.so
    0xff2e0000 /usr/lib/libCrun.so.1
    0xff1e0000 /usr/lib/libsocket.so.1
    0xff100000 /usr/lib/libnsl.so.1
    0xff0d0000 /usr/lib/libm.so.1
    0xff310000 /usr/lib/libw.so.1
    0xff0b0000 /usr/lib/libmp.so.2
    0xff080000 /usr/j2se/jre/lib/sparc/native_threads/libhpi.so
    0xff050000 /usr/j2se/jre/lib/sparc/libverify.so
    0xfe7c0000 /usr/j2se/jre/lib/sparc/libjava.so
    0xff030000 /usr/j2se/jre/lib/sparc/libzip.so
    0xf1f90000 /usr/j2se/jre/lib/sparc/libnet.so
    0xede00000
    /export/home/local/jakarta-tomcat-4.1.29/webapps/arsys/WEB-INF/lib/libarutiljni60.so
    0xefb60000 /usr/lib/libpthread.so.1
    0xef180000
    /export/home/local/jakarta-tomcat-4.1.29/webapps/arsys/WEB-INF/lib/libarjni60.so

    Local Time = Tue May 31 23:13:55 2005
    Elapsed Time = 1052
    #
    # HotSpot Virtual Machine Error : 11
    # Error ID : 4F530E43505002E6 01
    # Please report this error at
    # http://java.sun.com/cgi-bin/bugreport.cgi
    #
    # Java VM: Java HotSpot(TM) Client VM (1.4.1_02-b06 mixed mode)
    #
    # An error report file has been saved as hs_err_pid1184.log.
    # Please refer to the file for further information.
    #
     
    Cathy Hui, Jun 1, 2005
    #3
  4. Cathy  Hui

    Cathy Hui Guest

    Another piece of useful information is. While the java memory usage is
    trap in range 160-190MB, the CPU usage is going up to from very minimal
    to > 95% before Tomcat crashes.
     
    Cathy Hui, Jun 1, 2005
    #4
  5. Cathy  Hui

    . Guest

    On Tue, 31 May 2005, Cathy Hui wrote:

    > I am running an application (i.e. Remedy Midtier) on Tomcat. Currently
    > I run into the "java.lang.OutOfMemoryError" when performing some
    > activities from the browser when accessing this application.
    > Basically, the Tomcat crashes when the java process takes about 160MB -
    > 190MB memory usage (I did monitor the "top" for system resources)


    Try profiling it. JProbe from Quest Software has a profiler, memory
    debugger and coverage tool. It integrates with Tomcat 4.x. You can get a
    trial version at http://www.quest.com/jprobe/try.asp


    > Background information:
    >
    > OS: Solaris 8
    > System physical memory: 2048MB
    > Tomcat version: 4.1.29
    > JDK version: 1.4.1
    >
    > /etc/system
    >
    > set rlim_fd_cur = 65536
    > set rlim_fd_max = 65536
    >
    > This is what I have tried so far:
    >
    > 1) Double the heap size in catalina.sh to allocate to java from
    > "-Xms256m -Xmx512m" to "-Xms512m -Xmx1024m"
    >
    > 2) install jdk 1.4.2
    >
    > However, the result is: JAVA still fails after taking 160MB-190MB
    > memory and Tomcat crashes on java.lang.OutOfMemoryError.
    >
    > Does anyone know what might be the cause of it and how to fix it?
    >
    > Thanks!
    >
    >


    --
    Send e-mail to: darrell dot grainger at utoronto dot ca
     
    ., Jun 1, 2005
    #5
  6. Cathy Hui schreef:
    > Another piece of useful information is. While the java memory usage is
    > trap in range 160-190MB, the CPU usage is going up to from very minimal
    > to > 95% before Tomcat crashes.
    >


    I think the best way to go is using a profiler and examine the objects
    created as well as reviewing the code.

    Do you use Threads inside the servlets code too? These might be the
    cause of the problem as well.
     
    Gerbrand van Dieijen, Jun 2, 2005
    #6
  7. Cathy  Hui

    Wibble Guest

    Cathy Hui wrote:
    > I am running an application (i.e. Remedy Midtier) on Tomcat. Currently
    > I run into the "java.lang.OutOfMemoryError" when performing some
    > activities from the browser when accessing this application.
    > Basically, the Tomcat crashes when the java process takes about 160MB -
    > 190MB memory usage (I did monitor the "top" for system resources)
    >
    > Background information:
    >
    > OS: Solaris 8
    > System physical memory: 2048MB
    > Tomcat version: 4.1.29
    > JDK version: 1.4.1
    >
    > /etc/system
    >
    > set rlim_fd_cur = 65536
    > set rlim_fd_max = 65536
    >
    > This is what I have tried so far:
    >
    > 1) Double the heap size in catalina.sh to allocate to java from
    > "-Xms256m -Xmx512m" to "-Xms512m -Xmx1024m"
    >
    > 2) install jdk 1.4.2
    >
    > However, the result is: JAVA still fails after taking 160MB-190MB
    > memory and Tomcat crashes on java.lang.OutOfMemoryError.
    >
    > Does anyone know what might be the cause of it and how to fix it?
    >
    > Thanks!
    >


    We had these too and none of the profilers found it... it was the Perm
    memory. Use the following command line switch (you may need more or less)

    -XX:MaxPermSize=64m
     
    Wibble, Jun 2, 2005
    #7
  8. Cathy Hui wrote:

    > Another piece of useful information is. While the java memory usage is
    > trap in range 160-190MB, the CPU usage is going up to from very minimal
    > to > 95% before Tomcat crashes.


    It all sounds consistent with the VM hitting the heap size bound and
    going into GC thrashing before being forced to give up. It is possible
    that the 160-190 MB is coincidental. For instance, if your program
    happens for some reason to reliably try to allocate a 1 GB array when
    the memory consumption is at that level then raising the heap limit from
    512 MB to 1024 MB would not change the symptoms. This would also be
    consistent with the observation that a 512 MB heap limit was not enough,
    despite the fact that the observed consumption was less than 200 MB.

    --
    John Bollinger
     
    John C. Bollinger, Jun 6, 2005
    #8
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Ian Suttle
    Replies:
    2
    Views:
    2,324
    Wolfram Rittmeyer
    Aug 27, 2003
  2. Xiaopeng Xiong

    java.lang.OutOfMemoryError problem

    Xiaopeng Xiong, Nov 14, 2003, in forum: Java
    Replies:
    3
    Views:
    445
    Xiaopeng Xiong
    Nov 15, 2003
  3. Replies:
    0
    Views:
    479
  4. ckumar
    Replies:
    15
    Views:
    6,253
    ckumar
    Feb 26, 2005
  5. Goofball
    Replies:
    9
    Views:
    786
    Goofball
    Nov 8, 2006
Loading...

Share This Page