XSLT Large xml files

[Rajesh Patel]

I am currently using JDOM to access a rather large xml document(2mb).
JDOM is taking up too much memory, I am looking for somethng that
is more light weight.

I know that SAX will work, but I would rather uses something like
xslt since I'm tranforming the xml document into HTML. Don't
ask me why they want to transform such a large document into HTML,
especially considering the resultant HTML is going to be over 20meg.
That is what the customer wants...

So, is XSLT suitable for this type of thing, or is it going to
end up loading up the entire document into memory?

Raj

 

[Martin Honnen]

I am currently using JDOM to access a rather large xml document(2mb).
JDOM is taking up too much memory, I am looking for somethng that
is more light weight.

I know that SAX will work, but I would rather uses something like
xslt since I'm tranforming the xml document into HTML. Don't
ask me why they want to transform such a large document into HTML,
especially considering the resultant HTML is going to be over 20meg.
That is what the customer wants...

So, is XSLT suitable for this type of thing, or is it going to
end up loading up the entire document into memory?

XSLT builds a tree of the document so yes, it loads the entire document
into memory however depending on the implementation it might not use a
usual DOM tree but something lighter.

 

[Chris Huebsch]

Rajesh Patel (Mon, 28 Jun 2004 18:00:19 GMT):

I am currently using JDOM to access a rather large xml document(2mb).
JDOM is taking up too much memory, I am looking for somethng that
is more light weight.

I use libxml2 (wrapped in pyhton) with a slightly larger document
(120 MB ;-)

It takes 400 MB memory.

So, is XSLT suitable for this type of thing, or is it going to
end up loading up the entire document into memory?

XLSTing a 3 MB sized document into an other 3 MB sized document can be
done with libxsl (of the libxml2-project) within practically no time (2
or 3 seconds) and 30 MB of RAM.

A good approximation is: 4x(size inpzt + size output) bytes memory.


Chris

 

[GIMME]

Off hand the problem probably isn't JDOM ...

The problem is that the JDOM solution that you envision reads a
20 MB file and then attempts to transform the thing in one shot.

You need to read the file element by element and then transform
element by element.