java.lang.OutOfMemoryError

Discussion in 'Java' started by Rizwan, Dec 13, 2004.

  1. Rizwan

    Rizwan Guest

    I have written a java program (an EJB) to read a file and write it to
    database w.r.t. some rules. The program works ok but when the file has
    around 6000 rows it fails and give an error message :

    2004-12-13 14:08:47,467 ERROR [org.jboss.ejb.plugins.LogInterceptor]
    Unexpected Error:
    java.lang.OutOfMemoryError

    My environment is Java 1.4.2, Eclipse, Jboss 3.2.3 and MS SqlSever 2000 on
    Windows 2000/XP


    Any reason? Thanks
    Rizwan, Dec 13, 2004
    #1
    1. Advertising

  2. On Mon, 13 Dec 2004 14:38:50 -0500, Rizwan <> wrote:

    > I have written a java program (an EJB) to read a file and write it to
    > database w.r.t. some rules. The program works ok but when the file has
    > around 6000 rows it fails and give an error message :
    >
    > 2004-12-13 14:08:47,467 ERROR [org.jboss.ejb.plugins.LogInterceptor]
    > Unexpected Error:
    > java.lang.OutOfMemoryError
    >
    > My environment is Java 1.4.2, Eclipse, Jboss 3.2.3 and MS SqlSever 2000
    > on
    > Windows 2000/XP


    Try making parts of the data eligable for Garbage Collection as soon as you
    can, you are (obviously) running out of memory.

    --

    Whom the gods wish to destroy they first call promising.
    Stefan Schulz, Dec 13, 2004
    #2
    1. Advertising

  3. Rizwan

    srh Guest

    Hi,

    I have find out where exactly it is failing but dont know how to fix
    it. My program read the first row then based on some rules it generates
    a SQL statement for this row in a string. I then add it to the batch
    using addBatch method of Statement. When I finish reading the file I
    executes the batch using executeBatch metohd of Statement. Thats where
    it is failing. Any reason? Thanks

    Statement stmt = cx.createStatement();
    FileReader fr = new FileReader( file );
    BufferedReader br = new BufferedReader( fr );
    String lineData = null;
    String insertStatementSQL = "";
    while ( (lineData = br.readLine()) != null ) {
    .....
    stmt.addBatch(insertStatementSQL);
    }

    int returnValueIntArray[] = stmt.executeBatch();

    Stefan Schulz wrote:
    > On Mon, 13 Dec 2004 14:38:50 -0500, Rizwan <>

    wrote:
    >
    > > I have written a java program (an EJB) to read a file and write it

    to
    > > database w.r.t. some rules. The program works ok but when the file

    has
    > > around 6000 rows it fails and give an error message :
    > >
    > > 2004-12-13 14:08:47,467 ERROR

    [org.jboss.ejb.plugins.LogInterceptor]
    > > Unexpected Error:
    > > java.lang.OutOfMemoryError
    > >
    > > My environment is Java 1.4.2, Eclipse, Jboss 3.2.3 and MS SqlSever

    2000
    > > on
    > > Windows 2000/XP

    >
    > Try making parts of the data eligable for Garbage Collection as soon

    as you
    > can, you are (obviously) running out of memory.
    >
    > --
    >
    > Whom the gods wish to destroy they first call promising.
    srh, Dec 13, 2004
    #3
  4. Rizwan wrote:

    > I have written a java program (an EJB) to read a file and write it to
    > database w.r.t. some rules. The program works ok but when the file has
    > around 6000 rows it fails and give an error message :
    >
    > 2004-12-13 14:08:47,467 ERROR [org.jboss.ejb.plugins.LogInterceptor]
    > Unexpected Error:
    > java.lang.OutOfMemoryError
    >
    > My environment is Java 1.4.2, Eclipse, Jboss 3.2.3 and MS SqlSever 2000 on
    > Windows 2000/XP
    >
    >
    > Any reason? Thanks


    Yes. You are running out of memory.

    This probably means you are creating huge objects or huge numbers of
    objects, retaining references to objects you no longer need, or both. I
    cannot possibly tell you which of those or where, for the dry cleaner
    lost my magic turban and no one seems to want to make me a new one.


    John Bollinger
    John C. Bollinger, Dec 13, 2004
    #4
  5. Rizwan

    Tilman Bohn Guest

    On Mon, 13 Dec 2004 12:51:30 -0800, srh wrote:

    > Hi,
    >
    > I have find out where exactly it is failing but dont know how to fix
    > it. My program read the first row then based on some rules it generates
    > a SQL statement for this row in a string. I then add it to the batch
    > using addBatch method of Statement. When I finish reading the file I
    > executes the batch using executeBatch metohd of Statement. Thats where
    > it is failing. Any reason?


    Yes. Isn't this obvious?

    Either try dispatching smaller batches at a time or give your VM
    more memory to work with. The latter option only moves the threshold
    at which your app will fail, so the former is preferred.

    Cheers, Tilman

    --
    `Boy, life takes a long time to live...' -- Steven Wright
    Tilman Bohn, Dec 13, 2004
    #5
  6. Rizwan

    srh Guest

    But I want all the SQL statements to be part of one transaction; if any
    one SQL statement fails, I want to rollback the whole transaction.
    Correct me please if I am wron, but do you want me to do it this way:

    int recordCount = 0;

    while ( (lineData = br.readLine()) != null ) {
    recordCount ++;
    .....
    if (recordCount == 2000) {
    stmt.executeBatch();
    recordCount = 1;
    }
    stmt.addBatch(insertStatementSQL);
    }

    if (recordCount > 0) {
    stmt.executeBatch();
    }

    Thanks
    srh, Dec 13, 2004
    #6
  7. Rizwan

    srh Guest

    oops... forget to add one line stmt.clearBatch(); in the (recordCount
    == 2000) block.

    It still failed. On the second 2000 records. Any input?

    16:33:47,929 INFO [STDOUT] importTimeCardFile: recordCount = 2000
    16:35:03,070 INFO [STDOUT] importTimeCardFile: stmt.executeBatch()
    successful
    16:35:03,070 INFO [STDOUT] importTimeCardFile: stmt.clearBatch()
    successful
    16:35:03,632 INFO [STDOUT] importTimeCardFile: recordCount = 2000
    16:36:14,210 INFO [STDOUT] importTimeCardFile: SQLException = SQL
    Attempt to pr
    oduce a ResultSet from executeBatch
    16:36:14,210 ERROR [STDERR] sun.jdbc.odbc.JdbcOdbcBatchUpdateException:
    SQL Atte
    mpt to produce a ResultSet from executeBatch
    16:36:14,226 ERROR [STDERR] at
    sun.jdbc.odbc.JdbcOdbcStatement.executeBatchU
    pdate(JdbcOdbcStatement.java:1018)
    16:36:14,226 ERROR [STDERR] at
    sun.jdbc.odbc.JdbcOdbcStatement.executeBatch(
    JdbcOdbcStatement.java:912)
    16:36:14,226 ERROR [STDERR] at
    org.jboss.resource.adapter.jdbc.WrappedStatem
    ent.executeBatch(WrappedStatement.java:736)
    16:36:14,226 ERROR [STDERR] at
    com.pendylum.ejb.TimeCardImportEJBBean.import
    TimeCardFile(TimeCardImportEJBBean.java:574)
    16:36:14,226 ERROR [STDERR] at
    com.pendylum.ejb.TimeCardImportEJBBean.proces
    sTimeCardImport(TimeCardImportEJBBean.java:1943)
    16:36:14,226 ERROR [STDERR] at
    sun.reflect.NativeMethodAccessorImpl.invoke0(
    Native Method)
    16:36:14,226 ERROR [STDERR] at
    sun.reflect.NativeMethodAccessorImpl.invoke(N
    ativeMethodAccessorImpl.java:39)
    16:36:14,226 ERROR [STDERR] at
    sun.reflect.DelegatingMethodAccessorImpl.invo
    ke(DelegatingMethodAccessorImpl.java:25)
    16:36:14,226 ERROR [STDERR] at
    java.lang.reflect.Method.invoke(Method.java:3
    24)
    16:36:14,226 ERROR [STDERR] at
    org.jboss.ejb.StatelessSessionContainer$Conta
    inerInterceptor.invoke(StatelessSessionContainer.java:683)
    16:36:14,226 ERROR [STDERR] at
    org.jboss.resource.connectionmanager.CachedCo
    nnectionInterceptor.invoke(CachedConnectionInterceptor.java:185)
    16:36:14,226 ERROR [STDERR] at
    org.jboss.ejb.plugins.StatelessSessionInstanc
    eInterceptor.invoke(StatelessSessionInstanceInterceptor.java:72)
    16:36:14,226 ERROR [STDERR] at
    org.jboss.ejb.plugins.AbstractTxInterceptor.i
    nvokeNext(AbstractTxInterceptor.java:84)
    16:36:14,226 ERROR [STDERR] at
    org.jboss.ejb.plugins.TxInterceptorCMT.runWit
    hTransactions(TxInterceptorCMT.java:267)
    16:36:14,226 ERROR [STDERR] at
    org.jboss.ejb.plugins.TxInterceptorCMT.invoke
    (TxInterceptorCMT.java:128)
    16:36:14,226 ERROR [STDERR] at
    org.jboss.ejb.plugins.SecurityInterceptor.inv
    oke(SecurityInterceptor.java:118)
    16:36:14,226 ERROR [STDERR] at
    org.jboss.ejb.plugins.LogInterceptor.invoke(L
    ogInterceptor.java:191)
    16:36:14,226 ERROR [STDERR] at
    org.jboss.ejb.plugins.ProxyFactoryFinderInter
    ceptor.invoke(ProxyFactoryFinderInterceptor.java:122)
    16:36:14,226 ERROR [STDERR] at
    org.jboss.ejb.StatelessSessionContainer.inter
    nalInvoke(StatelessSessionContainer.java:331)
    16:36:14,226 ERROR [STDERR] at
    org.jboss.ejb.Container.invoke(Container.java
    :700)
    16:36:14,226 ERROR [STDERR] at
    sun.reflect.GeneratedMethodAccessor63.invoke(
    Unknown Source)
    16:36:14,226 ERROR [STDERR] at
    sun.reflect.DelegatingMethodAccessorImpl.invo
    ke(DelegatingMethodAccessorImpl.java:25)
    16:36:14,226 ERROR [STDERR] at
    java.lang.reflect.Method.invoke(Method.java:3
    24)
    16:36:14,226 ERROR [STDERR] at
    org.jboss.mx.capability.ReflectedMBeanDispatc
    her.invoke(ReflectedMBeanDispatcher.java:284)
    16:36:14,226 ERROR [STDERR] at
    org.jboss.mx.server.MBeanServerImpl.invoke(MB
    eanServerImpl.java:546)
    16:36:14,226 ERROR [STDERR] at
    org.jboss.invocation.local.LocalInvoker.invok
    e(LocalInvoker.java:101)
    16:36:14,226 ERROR [STDERR] at
    org.jboss.invocation.InvokerInterceptor.invok
    e(InvokerInterceptor.java:90)
    16:36:14,226 ERROR [STDERR] at
    org.jboss.proxy.TransactionInterceptor.invoke
    (TransactionInterceptor.java:46)
    16:36:14,226 ERROR [STDERR] at
    org.jboss.proxy.SecurityInterceptor.invoke(Se
    curityInterceptor.java:45)
    16:36:14,226 ERROR [STDERR] at
    org.jboss.proxy.ejb.StatelessSessionIntercept
    or.invoke(StatelessSessionInterceptor.java:100)
    16:36:14,226 ERROR [STDERR] at
    org.jboss.proxy.ClientContainer.invoke(Client
    Container.java:85)
    16:36:14,226 ERROR [STDERR] at
    $Proxy74.processTimeCardImport(Unknown Source
    )
    16:36:14,226 ERROR [STDERR] at
    org.apache.jsp.inputprocess_jsp._jspService(U
    nknown Source)
    16:36:14,226 ERROR [STDERR] at
    org.apache.jasper.runtime.HttpJspBase.service
    (HttpJspBase.java:137)
    16:36:14,226 ERROR [STDERR] at
    javax.servlet.http.HttpServlet.service(HttpSe
    rvlet.java:853)
    16:36:14,226 ERROR [STDERR] at
    org.apache.jasper.servlet.JspServletWrapper.s
    ervice(JspServletWrapper.java:210)
    16:36:14,226 ERROR [STDERR] at
    org.apache.jasper.servlet.JspServlet.serviceJ
    spFile(JspServlet.java:295)
    16:36:14,226 ERROR [STDERR] at
    org.apache.jasper.servlet.JspServlet.service(
    JspServlet.java:241)
    16:36:14,226 ERROR [STDERR] at
    javax.servlet.http.HttpServlet.service(HttpSe
    rvlet.java:853)
    16:36:14,226 ERROR [STDERR] at
    org.apache.catalina.core.ApplicationFilterCha
    in.internalDoFilter(ApplicationFilterChain.java:247)
    16:36:14,226 ERROR [STDERR] at
    org.apache.catalina.core.ApplicationFilterCha
    in.doFilter(ApplicationFilterChain.java:193)
    16:36:14,226 ERROR [STDERR] at
    org.apache.catalina.core.StandardWrapperValve
    ..invoke(StandardWrapperValve.java:256)
    16:36:14,226 ERROR [STDERR] at
    org.apache.catalina.core.StandardPipeline$Sta
    ndardPipelineValveContext.invokeNext(StandardPipeline.java:643)
    16:36:14,226 ERROR [STDERR] at
    org.apache.catalina.core.StandardPipeline.inv
    oke(StandardPipeline.java:480)
    16:36:14,226 ERROR [STDERR] at
    org.apache.catalina.core.ContainerBase.invoke
    (ContainerBase.java:995)
    16:36:14,226 ERROR [STDERR] at
    org.apache.catalina.core.StandardContextValve
    ..invoke(StandardContextValve.java:191)
    16:36:14,226 ERROR [STDERR] at
    org.apache.catalina.core.StandardPipeline$Sta
    ndardPipelineValveContext.invokeNext(StandardPipeline.java:643)
    16:36:14,226 ERROR [STDERR] at
    org.jboss.web.tomcat.security.JBossSecurityMg
    rRealm.invoke(JBossSecurityMgrRealm.java:220)
    16:36:14,226 ERROR [STDERR] at
    org.apache.catalina.core.StandardPipeline$Sta
    ndardPipelineValveContext.invokeNext(StandardPipeline.java:641)
    16:36:14,226 ERROR [STDERR] at
    org.apache.catalina.valves.CertificatesValve.
    invoke(CertificatesValve.java:246)
    16:36:14,226 ERROR [STDERR] at
    org.apache.catalina.core.StandardPipeline$Sta
    ndardPipelineValveContext.invokeNext(StandardPipeline.java:641)
    16:36:14,241 ERROR [STDERR] at
    org.jboss.web.tomcat.tc4.statistics.Container
    StatsValve.invoke(ContainerStatsValve.java:76)
    16:36:14,241 ERROR [STDERR] at
    org.apache.catalina.core.StandardPipeline$Sta
    ndardPipelineValveContext.invokeNext(StandardPipeline.java:641)
    16:36:14,241 ERROR [STDERR] at
    org.jboss.web.tomcat.session.ClusteredSession
    Valve.invoke(ClusteredSessionValve.java:78)
    16:36:14,241 ERROR [STDERR] at
    org.apache.catalina.core.StandardPipeline$Sta
    ndardPipelineValveContext.invokeNext(StandardPipeline.java:641)
    16:36:14,241 ERROR [STDERR] at
    org.apache.catalina.core.StandardPipeline.inv
    oke(StandardPipeline.java:480)
    16:36:14,241 ERROR [STDERR] at
    org.apache.catalina.core.ContainerBase.invoke
    (ContainerBase.java:995)
    16:36:14,241 ERROR [STDERR] at
    org.apache.catalina.core.StandardContext.invo
    ke(StandardContext.java:2417)
    16:36:14,241 ERROR [STDERR] at
    org.apache.catalina.core.StandardHostValve.in
    voke(StandardHostValve.java:180)
    16:36:14,241 ERROR [STDERR] at
    org.apache.catalina.core.StandardPipeline$Sta
    ndardPipelineValveContext.invokeNext(StandardPipeline.java:643)
    16:36:14,476 ERROR [STDERR] at
    org.apache.catalina.valves.ErrorDispatcherVal
    ve.invoke(ErrorDispatcherValve.java:171)
    16:36:14,476 ERROR [STDERR] at
    org.apache.catalina.core.StandardPipeline$Sta
    ndardPipelineValveContext.invokeNext(StandardPipeline.java:641)
    16:36:14,476 ERROR [STDERR] at
    org.apache.catalina.valves.ErrorReportValve.i
    nvoke(ErrorReportValve.java:172)
    16:36:14,476 ERROR [STDERR] at
    org.apache.catalina.core.StandardPipeline$Sta
    ndardPipelineValveContext.invokeNext(StandardPipeline.java:641)
    16:36:14,476 ERROR [STDERR] at
    org.jboss.web.tomcat.security.SecurityAssocia
    tionValve.invoke(SecurityAssociationValve.java:65)
    16:36:14,476 ERROR [STDERR] at
    org.apache.catalina.core.StandardPipeline$Sta
    ndardPipelineValveContext.invokeNext(StandardPipeline.java:641)
    16:36:14,476 ERROR [STDERR] at
    org.apache.catalina.valves.AccessLogValve.inv
    oke(AccessLogValve.java:577)
    16:36:14,476 ERROR [STDERR] at
    org.apache.catalina.core.StandardPipeline$Sta
    ndardPipelineValveContext.invokeNext(StandardPipeline.java:641)
    16:36:14,476 ERROR [STDERR] at
    org.apache.catalina.core.StandardPipeline.inv
    oke(StandardPipeline.java:480)
    16:36:14,476 ERROR [STDERR] at
    org.apache.catalina.core.ContainerBase.invoke
    (ContainerBase.java:995)
    16:36:14,476 ERROR [STDERR] at
    org.apache.catalina.core.StandardEngineValve.
    invoke(StandardEngineValve.java:174)
    16:36:14,476 ERROR [STDERR] at
    org.apache.catalina.core.StandardPipeline$Sta
    ndardPipelineValveContext.invokeNext(StandardPipeline.java:643)
    16:36:14,476 ERROR [STDERR] at
    org.apache.catalina.core.StandardPipeline.inv
    oke(StandardPipeline.java:480)
    16:36:14,476 ERROR [STDERR] at
    org.apache.catalina.core.ContainerBase.invoke
    (ContainerBase.java:995)
    16:36:14,476 ERROR [STDERR] at
    org.apache.coyote.tomcat4.CoyoteAdapter.servi
    ce(CoyoteAdapter.java:197)
    16:36:14,476 ERROR [STDERR] at
    org.apache.coyote.http11.Http11Processor.proc
    ess(Http11Processor.java:781)
    16:36:14,476 ERROR [STDERR] at
    org.apache.coyote.http11.Http11Protocol$Http1
    1ConnectionHandler.processConnection(Http11Protocol.java:549)
    16:36:14,476 ERROR [STDERR] at
    org.apache.tomcat.util.net.TcpWorkerThread.ru
    nIt(PoolTcpEndpoint.java:605)
    16:36:14,476 ERROR [STDERR] at
    org.apache.tomcat.util.threads.ThreadPool$Con
    trolRunnable.run(ThreadPool.java:677)
    16:36:14,476 ERROR [STDERR] at
    java.lang.Thread.run(Thread.java:534)
    16:36:14,476 INFO [STDOUT] processTimeCardImport: closing the
    connection....


    574 line is where the stmt.executeBatch(); is
    srh, Dec 13, 2004
    #7
  8. Rizwan

    Andy Hill Guest

    "srh" <> wrote:
    >But I want all the SQL statements to be part of one transaction; if any
    >one SQL statement fails, I want to rollback the whole transaction.
    >

    So why not make the Connection non-auto-committing, issue your SQL statements
    one at a time (instead of in a batch), then do an explicit Connection.commit()
    when you know you haven't screwed the pooch? That way you don't have to keep
    more than the minimum amount of info around in memory.
    Andy Hill, Dec 13, 2004
    #8
  9. Rizwan

    Guest

    Rizwan wrote:
    > I have written a java program (an EJB) to read a file and write it to
    > database w.r.t. some rules. The program works ok but when the file

    has
    > around 6000 rows it fails and give an error message :
    >
    > 2004-12-13 14:08:47,467 ERROR [org.jboss.ejb.plugins.LogInterceptor]
    > Unexpected Error:
    > java.lang.OutOfMemoryError
    >
    > My environment is Java 1.4.2, Eclipse, Jboss 3.2.3 and MS SqlSever

    2000 on
    > Windows 2000/XP
    >
    >
    > Any reason? Thanks


    If there's a leak, that's tricker. But I'm
    betting in this case you might just need a
    bigger heap. Try java -Xmx200m (or something
    to that effect.)

    --Joe
    , Dec 15, 2004
    #9
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Ian Suttle
    Replies:
    2
    Views:
    2,290
    Wolfram Rittmeyer
    Aug 27, 2003
  2. Xiaopeng Xiong

    java.lang.OutOfMemoryError problem

    Xiaopeng Xiong, Nov 14, 2003, in forum: Java
    Replies:
    3
    Views:
    429
    Xiaopeng Xiong
    Nov 15, 2003
  3. Mit Majithia
    Replies:
    6
    Views:
    15,041
    Moazam Raja
    Feb 26, 2004
  4. H.MuthuKumaraRajan

    java.lang.OutOfMemoryError

    H.MuthuKumaraRajan, Apr 3, 2004, in forum: Java
    Replies:
    1
    Views:
    1,217
    e.Morr
    Apr 3, 2004
  5. Goofball
    Replies:
    9
    Views:
    753
    Goofball
    Nov 8, 2006
Loading...

Share This Page