XML - Xpath Memory issues

Discussion in 'XML' started by suresh.csharp@gmail.com, Feb 27, 2006.

  1. Guest

    Hi,
    I am trying to import huge data into XML files it is in more than
    700Mbs, creating a index(catalog) XML file for all the imported XML
    files for searching xml files content. The idex(catalog) file will be
    more than
    100mb. When I try to load this index(catalog) file to DOM on IE, it is
    taking lot of time and when I do search on this document with Xpath and
    load the results it is taking almost like 2-4 minutes and some times it
    will give out of memory error. I have to run this entire application
    in CD's and run on clients browsers.

    I am doing lot of google on this issue.. but not much help. Can any one
    tell me.. How much size of XML file load on to DOM, it will stable and
    doing Xpath on this document. Any performance tips most welcome..

    Thanks
    Suresh
    , Feb 27, 2006
    #1
    1. Advertising

  2. wrote:
    > How much size of XML file load on to DOM, it will stable and
    > doing Xpath on this document. Any performance tips most welcome..


    Depends on the DOM implementation. (Remember, the DOM is just an API,
    and that same API may have many different kinds of data storage
    implemented behind it.)

    Simple-minded DOM implementations (one object per node) do tend to burn
    a lot of memory, especialy in Java where objects have a lot of overhead.
    That shouldn't be a huge problem in a system with decent virtual memory,
    but you may run into swapping.

    Some DOM implementations may use more compact back-end representations,
    simply exposing them through DOM proxy objects.

    Or the best answer may not be to use DOM at all. Apache Xalan uses a
    non-DOM data model known as DTM. If your task can be implemented as an
    essentially linear scan of the document, you may not even need a
    complete in-memory model and may prefer to do a SAX-based streaming
    solution.

    (See also http://www.w3.org/DOM/faq.html#SAXandDOM)

    --
    () ASCII Ribbon Campaign | Joe Kesselman
    /\ Stamp out HTML e-mail! | System architexture and kinetic poetry
    Joe Kesselman, Feb 28, 2006
    #2
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Marvin_123456

    "Memory leak" in javax.xml.xpath.XPath

    Marvin_123456, Jul 29, 2005, in forum: Java
    Replies:
    4
    Views:
    1,968
    jan V
    Jul 29, 2005
  2. Alastair Cameron
    Replies:
    1
    Views:
    7,399
    SQL Server Development Team [MSFT]
    Jul 8, 2003
  3. Anna
    Replies:
    0
    Views:
    520
  4. goog
    Replies:
    0
    Views:
    506
  5. Tjerk Wolterink

    XPath: efficiency in xpath expressions

    Tjerk Wolterink, Nov 13, 2004, in forum: XML
    Replies:
    1
    Views:
    1,624
    Richard Tobin
    Nov 13, 2004
Loading...

Share This Page