XML Processing

[Jay]

It seems to me that when I retrieve large amounts of XML data using Visual
Basic 6 (lets say 900 records 20 fields), processing seems to slow way down.

Has anyone else experienced this?

Thank You,

 

[Martin Boehm]

It seems to me that when I retrieve large amounts of XML data using
Visual Basic 6 (lets say 900 records 20 fields), processing seems to
slow way down.

That depends. But - yes. If you work with a lot of data, processing
slows down. I can see no reation to XML, this is the normal behaviour of
a computer.

You have given no information of what you do and of what "slowing way
down" means to you, so do not expect something more helpful than this.

Martin

 

[Andy Dingley]

It seems to me that when I retrieve large amounts of XML data using Visual
Basic 6 (lets say 900 records 20 fields), processing seems to slow way down.

A DOM holds the entire XML document in memory, a relational database
will hold it on disk and just load pages as required. You can imagine
the effects for yourself, when single documents become large.

XML DOM's are a good way to manipulate data, and not a bad way to
extract pieces from a larger working set. But when your potential
source set is really big, then it's not the best technology to choose.
Try building a smaller document (or even several of them) as the
result of a database query, and only put into it what you need.

 

[arachno]

There are many ways how to "optimize" your XML for increasing speed of
processign and parsing. So perfomance is not always down to how good your
PC, but to your skills also.

Ev.

 

[Martin Boehm]

So perfomance is not always down to how good your PC,
but to your skills also.

I did not deny that. ;-)

Martin

 

[Patrick TJ McPhee]

% On Wed, 22 Oct 2003 11:59:15 -0500, "Jay" <[email protected]>
% wrote:
%
% >It seems to me that when I retrieve large amounts of XML data using Visual
% >Basic 6 (lets say 900 records 20 fields), processing seems to slow way down.
%
% A DOM holds the entire XML document in memory

It doesn't have to. One question for the world at large: is there a DOM
implementation that either parses lazily or creates the tree on disk?

 

[Andy Dingley]

% A DOM holds the entire XML document in memory

It doesn't have to. One question for the world at large: is there a DOM
implementation that either parses lazily or creates the tree on disk?

I've often thought of that, but as yet I haven't seen one. With VB, I
assume the original poster was using MSXML anyway.

To be honest, I gave up on DOMs for handling data when I abandoned XML
in favour of RDF. Now I use Jena and a database-backed triple store,
so that's effectively what I'm getting.

It's actually quite an awkward task to "parse lazily" in a useful
manner, which is one of those issues that RDF had to address a long
time ago. You can lazy-parse XML quite easily, but this is only
useful if your demands on the data model are similar to the linear
serialisation. When they're not, things get hairy.

 

[Julian F. Reschke]

I've often thought of that, but as yet I haven't seen one. With VB, I
assume the original poster was using MSXML anyway.

...

http://www.infonyte.com/en/prod_pdom.html

 

[Rob Tweed]

% A DOM holds the entire XML document in memory

It doesn't have to. One question for the world at large: is there a DOM
implementation that either parses lazily or creates the tree on disk?

Have a look at eXtc - a W3C DOM based Native XML Database
implementation that creates persistent DOMs directly to disc

Find out more at www.mgateway.com



---
Rob Tweed
M/Gateway Developments Ltd

Global DOMination with eXtc : http://www.mgateway.tzo.com
---

 

[Jon Best]

Hi,

I have a project to create a tool to analyse performances of an athlte being
tested on a jump mat. (what it is doesn't really matter... it gives me a CSV
with data in of an athletes scores).

I need to be able to attch athletes to groups of athletes, and access
different parts of the data for different tests. I would like to save
athletes so they can be loaded at a later data and have more data added to
them. Does it sound like I want to be using XML? And if so what tools do I
need to consider to do it?

With the small amount of reading I have done it sounds like DOM would be
more suited to the job them SAX but would a be better suited using JDOM or
DOM4J. (It is my intention to implement in java...)

Any advise or reading material would be very much appreciated. If I am
really barking up the wrong tree would someone please let me know!


Thanks

Jon