Memory Consumption in JDOM

[Vijay Anandh]

Hi All,
I am new to JDOM and I am currently coding a program to build xml
files using JDOM.
The size of the xml file which i am goin to build will be in the order
of a few GB's( 1 to 5 GB).
Will I be able to build such huge documents with JDOM??
Also if there are any resources on jdom benchmarking please let me
know the links.
( BTW, I am using the latest JDOM build ).

Thanks in Advance for your help
Vijay Anandh

 

[Vijay Anandh]

JDOM experts........, help me please on my above question.
I am eagerly waiting...,

 

[=?ISO-8859-1?Q?J=FCrgen_Kahrs?=]

I am new to JDOM and I am currently coding a program to build xml
files using JDOM.
The size of the xml file which i am goin to build will be in the order
of a few GB's( 1 to 5 GB).
Will I be able to build such huge documents with JDOM??

No, probably not. In this newsgroup, we have
explained many many times what a DOM does:
Read the complete XML file into memory.
This procedure needs at least so much memory
(RAM) as the XML file takes on disk.

So, unless you have many GB of RAM, the answer is no.
Use Google to find all the other details that
have been posted in this newsgroup earlier.

 

[Joe Kesselman]

So, unless you have many GB of RAM, the answer is no.

Slight quibble: Most systems these days support virtual memory, so it is
theoretically possible to build models that exceed the size of physical
memory if you're willing to spill to disk. But that costs performance
and Java itself may impose some memory-size limits.

There are two possible approaches. One is to do active memory
management. Another is to replace the JDOM (which is a rather naive
design, I believe, consuming an object per node) or DOM with a more
compact data model; this is part of why we invented the DTM model for Xalan.

If you don't want to do a lot of coding to reinvent those solutions, you
may want to investigate XML databases. Managing huge amounts of data is
*supposed* to be what they're tuned for, and they should have applied
these sorts of tricks. (Note that I say "should"; I can't advise you on
which ones are good or not other than to say that I'm favorably
impressed with IBM's new XML support in the latest version of DB2. But
I'm an IBMer, so I may be biased.)

 

[Vijay Anandh]

Thanks for your replies..
I have tried benchmarking my JDOM application
on our solaris 8 server( more RAM now ).

sizes of files to create 10M 20M 30M 40M
increase heap size to none none 128M 128M
~ time taken in seconds 2 4 7 11


And fortunately the maximum file size I will be creating is ~70M.
So I will right now persist with JDOM ( while increasing my JVM heap
size )
and test its limits and then plan for XML databases

Vijay Anandh