Memory Consumption in JDOM

Discussion in 'XML' started by Vijay Anandh, May 13, 2006.

  1. Vijay Anandh

    Vijay Anandh Guest

    Hi All,
    I am new to JDOM and I am currently coding a program to build xml
    files using JDOM.
    The size of the xml file which i am goin to build will be in the order
    of a few GB's( 1 to 5 GB).
    Will I be able to build such huge documents with JDOM??
    Also if there are any resources on jdom benchmarking please let me
    know the links.
    ( BTW, I am using the latest JDOM build ).

    Thanks in Advance for your help
    Vijay Anandh
     
    Vijay Anandh, May 13, 2006
    #1
    1. Advertising

  2. Vijay Anandh

    Vijay Anandh Guest

    JDOM experts........, help me please on my above question.
    I am eagerly waiting...,
     
    Vijay Anandh, May 13, 2006
    #2
    1. Advertising

  3. Vijay Anandh wrote:

    > I am new to JDOM and I am currently coding a program to build xml
    > files using JDOM.
    > The size of the xml file which i am goin to build will be in the order
    > of a few GB's( 1 to 5 GB).
    > Will I be able to build such huge documents with JDOM??


    No, probably not. In this newsgroup, we have
    explained many many times what a DOM does:
    Read the complete XML file into memory.
    This procedure needs at least so much memory
    (RAM) as the XML file takes on disk.

    So, unless you have many GB of RAM, the answer is no.
    Use Google to find all the other details that
    have been posted in this newsgroup earlier.
     
    =?ISO-8859-1?Q?J=FCrgen_Kahrs?=, May 13, 2006
    #3
  4. J├╝rgen Kahrs wrote:
    > So, unless you have many GB of RAM, the answer is no.


    Slight quibble: Most systems these days support virtual memory, so it is
    theoretically possible to build models that exceed the size of physical
    memory if you're willing to spill to disk. But that costs performance
    and Java itself may impose some memory-size limits.

    There are two possible approaches. One is to do active memory
    management. Another is to replace the JDOM (which is a rather naive
    design, I believe, consuming an object per node) or DOM with a more
    compact data model; this is part of why we invented the DTM model for Xalan.

    If you don't want to do a lot of coding to reinvent those solutions, you
    may want to investigate XML databases. Managing huge amounts of data is
    *supposed* to be what they're tuned for, and they should have applied
    these sorts of tricks. (Note that I say "should"; I can't advise you on
    which ones are good or not other than to say that I'm favorably
    impressed with IBM's new XML support in the latest version of DB2. But
    I'm an IBMer, so I may be biased.)


    --
    () ASCII Ribbon Campaign | Joe Kesselman
    /\ Stamp out HTML e-mail! | System architexture and kinetic poetry
     
    Joe Kesselman, May 13, 2006
    #4
  5. Vijay Anandh

    Vijay Anandh Guest

    Thanks for your replies..
    I have tried benchmarking my JDOM application
    on our solaris 8 server( more RAM now ).

    sizes of files to create 10M 20M 30M 40M
    increase heap size to none none 128M 128M
    ~ time taken in seconds 2 4 7 11


    And fortunately the maximum file size I will be creating is ~70M.
    So I will right now persist with JDOM ( while increasing my JVM heap
    size )
    and test its limits and then plan for XML databases

    Vijay Anandh
     
    Vijay Anandh, May 13, 2006
    #5
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Wendy S
    Replies:
    1
    Views:
    6,470
    Darren Davison
    Aug 5, 2003
  2. Bernd Oninger
    Replies:
    4
    Views:
    12,729
    GIMME
    Jun 21, 2004
  3. Tinker
    Replies:
    4
    Views:
    5,449
    Harry Bosch
    Oct 9, 2005
  4. Bernd Oninger
    Replies:
    3
    Views:
    2,928
    GIMME
    Jun 21, 2004
  5. Replies:
    5
    Views:
    936
    Dale King
    May 15, 2006
Loading...

Share This Page