process large file

[Jimmy Zhang]

I am having some trouble processing some large file (40mb) in Java. For the problem I have, I have tried to use SAX, but doesn't find it suitable (well, coding just becomes a little complicated). So DOM is better but a little overhead on memory. Can someone share with me their experiences in dealing with their situations?

Cheers,
JZ

 

[Toivo Lainevool]

[QUOTE="Jimmy Zhang" wrote:
I am having some trouble processing some large file (40mb) in Java. For the
problem I have, I have tried to use SAX, but doesn't
find it suitable (well, coding just becomes a little complicated). So DOM is
better but a little overhead on memory. Can someone share with me their
experiences in dealing with their situations?[/QUOTE]One good option is to use a pull parser. It has a simpler interface than SAX, but doesn't have the memory overhead of DOM.

See http://www.extreme.indiana.edu/xgws/xsoap/xpp/ or http://www.xmlpull.org/

Toivo Lainevool
http://www.XMLPatterns.com - Develop effective DTDs and XML Schema documents for your XML using structural design patterns.

 

[Alexey Shirshov]

Hello, Toivo!
You wrote on 7 Apr 2004 12:09:05 -0700:

[Sorry, skipped]

TL> One good option is to use a pull parser. It has a simpler interface TL> than SAX, but doesn't have the memory overhead of DOM.

TL> See http://www.extreme.indiana.edu/xgws/xsoap/xpp/ or TL> http://www.xmlpull.org/

Intresting, that MS calls it cursor model processing (XPathNavigator), and based on SAX calls push/pull model (XmlWriter/XmlReader).

With best regards, Alexey Shirshov.

 

[Jimmy Zhang]

How much memory does XPathNavigator consume? I assume it loads everything in memory like DOM.

 

[Alexey Shirshov]

Hello, Jimmy!
You wrote on Thu, 08 Apr 2004 23:40:00 GMT:

JZ> How much memory does XPathNavigator consume? I assume it loads JZ> everything in memory like DOM.

Well, XPathNavigator is just an interface (actually abstract class) and we cann't talk about it performance. The important thing is an implementation of this class. XmlDocument, which represents DOM implements it in DocumentXPathNavigator class. You can create the instance of this class via CreateNavigator method. Another implementation you can get via CreateNavigator of the XPathDocument class. First implementation uses DOM as underlying data model, while the second - XPath data model. I think, for very large documents the XPathDocument will be much faster.

[Sorry, skipped]

With best regards, Alexey Shirshov.