OutOfMemoryException With XML DOM

Discussion in 'Java' started by Jason Cavett, Feb 21, 2008.

  1. Jason Cavett

    Jason Cavett Guest

    My project uses XML for its data files and I am using a DOM parser
    (the one native to the JDK) to parse out the files. DOM is especially
    useful because the project lends itself to the use of trees.

    Unfortunately, there tends to be a limit as to how big the XML files
    can be before the DOM parser starts chewing up memory and, if the file
    is big enough, I get an OutOfMemoryException. It's not from the
    project specifically - it's instead a result of the enormous amount of
    space DOM takes up.

    I was wondering if there's a solution to this? I have read about SAX
    a bit, and although it would fix the OOMEx. it would make it more
    difficult to manage the tree structure. I could also increase the
    amount of RAM available to the JRE, but I'd rather do that as a last
    resort.

    Does anybody have any other suggestions? Thanks.
     
    Jason Cavett, Feb 21, 2008
    #1
    1. Advertising

  2. Jason Cavett

    Arne Vajhøj Guest

    Jason Cavett wrote:
    > My project uses XML for its data files and I am using a DOM parser
    > (the one native to the JDK) to parse out the files. DOM is especially
    > useful because the project lends itself to the use of trees.
    >
    > Unfortunately, there tends to be a limit as to how big the XML files
    > can be before the DOM parser starts chewing up memory and, if the file
    > is big enough, I get an OutOfMemoryException. It's not from the
    > project specifically - it's instead a result of the enormous amount of
    > space DOM takes up.
    >
    > I was wondering if there's a solution to this? I have read about SAX
    > a bit, and although it would fix the OOMEx. it would make it more
    > difficult to manage the tree structure. I could also increase the
    > amount of RAM available to the JRE, but I'd rather do that as a last
    > resort.
    >
    > Does anybody have any other suggestions?


    No.

    -Xmx seems as the best way to go.

    Arne
     
    Arne Vajhøj, Feb 21, 2008
    #2
    1. Advertising

  3. Jason Cavett

    Boris Stumm Guest

    Jason Cavett wrote:
    > I was wondering if there's a solution to this? I have read about SAX
    > a bit, and although it would fix the OOMEx. it would make it more
    > difficult to manage the tree structure. I could also increase the
    > amount of RAM available to the JRE, but I'd rather do that as a last
    > resort.


    Maybe have a look at XML databases. I am not really into this matter,
    but I know some guys in my working group have one that is accessible with
    DOM. There should be others, too. The problem will be to find one which
    is stable enough for production use.
     
    Boris Stumm, Feb 21, 2008
    #3
  4. Jason Cavett

    Jason Cavett Guest

    On Feb 20, 9:07 pm, Arne Vajhøj <> wrote:
    > Jason Cavett wrote:
    > > My project uses XML for its data files and I am using a DOM parser
    > > (the one native to the JDK) to parse out the files. DOM is especially
    > > useful because the project lends itself to the use of trees.

    >
    > > Unfortunately, there tends to be a limit as to how big the XML files
    > > can be before the DOM parser starts chewing up memory and, if the file
    > > is big enough, I get an OutOfMemoryException. It's not from the
    > > project specifically - it's instead a result of the enormous amount of
    > > space DOM takes up.

    >
    > > I was wondering if there's a solution to this? I have read about SAX
    > > a bit, and although it would fix the OOMEx. it would make it more
    > > difficult to manage the tree structure. I could also increase the
    > > amount of RAM available to the JRE, but I'd rather do that as a last
    > > resort.

    >
    > > Does anybody have any other suggestions?

    >
    > No.
    >
    > -Xmx seems as the best way to go.
    >
    > Arne


    Haha. Alright. I was sort of hoping that wasn't the solution, but if
    that's what has to be done, that's what I'll do.

    Thanks.
     
    Jason Cavett, Feb 21, 2008
    #4
  5. Wed, 20 Feb 2008 17:19:32 -0800 (PST), /Jason Cavett/:

    > My project uses XML for its data files and I am using a DOM parser
    > (the one native to the JDK) to parse out the files. DOM is especially
    > useful because the project lends itself to the use of trees.
    >
    > Unfortunately, there tends to be a limit as to how big the XML files
    > can be before the DOM parser starts chewing up memory and, if the file
    > is big enough, I get an OutOfMemoryException. It's not from the
    > project specifically - it's instead a result of the enormous amount of
    > space DOM takes up.
    >
    > I was wondering if there's a solution to this? I have read about SAX
    > a bit, and although it would fix the OOMEx. it would make it more
    > difficult to manage the tree structure. I could also increase the
    > amount of RAM available to the JRE, but I'd rather do that as a last
    > resort.
    >
    > Does anybody have any other suggestions? Thanks.


    Great deal of the DOM is usually taken by whitespace in element
    content (used only to format the source XML text). Depending on the
    parser implementation you could supply a DTD to make the parser
    ignore [1] the whitespace in element content, or use custom
    filtering [2] as provided by the DOM Level 3 Load and Save APIs and
    implementation part of the standard Java 1.5 framework.

    The Xerces2 implementation (modified version of which is part of the
    Sun's Java 1.5 distribution) is capable of ignoring whitespace in
    element content when a suitable DTD is provided even in
    non-validating mode. One could supply a DTD for documents which
    don't have a DOCTYPE declaration setting an EntityResolver2 [3] (see
    the getExternalSubset() method) instance to the DocumentBuilder [4].

    All the above stuff is also available to Java 1.4 users simply by
    plugging the latest Xerces2 jars into the classpath.

    [1]
    <http://java.sun.com/j2se/1.5.0/docs/api/javax/xml/parsers/DocumentBuilderFactory.html#setIgnoringElementContentWhitespace(boolean)>
    [2]
    <http://java.sun.com/j2se/1.5.0/docs/api/org/w3c/dom/ls/LSParserFilter.html>
    [3]
    <http://java.sun.com/j2se/1.5.0/docs/api/org/xml/sax/ext/EntityResolver2.html>
    [4]
    <http://java.sun.com/j2se/1.5.0/docs/api/javax/xml/parsers/DocumentBuilder.html#setEntityResolver(org.xml.sax.EntityResolver)>

    --
    Stanimir
     
    Stanimir Stamenkov, Feb 21, 2008
    #5
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. =?Utf-8?B?UmlwdWwgSGFuZGE=?=

    Base Exception:System.OutOfMemoryException error message

    =?Utf-8?B?UmlwdWwgSGFuZGE=?=, Nov 17, 2003, in forum: ASP .Net
    Replies:
    1
    Views:
    999
    Natty Gur
    Nov 18, 2003
  2. =?Utf-8?B?amF6?=

    OutOfMemoryException

    =?Utf-8?B?amF6?=, Dec 10, 2003, in forum: ASP .Net
    Replies:
    2
    Views:
    352
    =?Utf-8?B?SmF6?=
    Dec 16, 2003
  3. Replies:
    0
    Views:
    559
  4. Replies:
    3
    Views:
    538
    Stefan Behnel
    Aug 3, 2007
  5. Alan
    Replies:
    6
    Views:
    1,604
Loading...

Share This Page