Large Size XML files for data transfer

R

RAJ

Hi,
Need some advice with regards to XML files. Here is what I am trying to
achieve:

I have an SQL Server 7.0 database and a Web service running on a Windows
2003 enterprise machine (for development). The web service (developed using
VB.NET) creates an XML file which contains most of the database table. The
creation of this file takes less then couple of minutes. This XML is about
57 MB in size and is used to build a local database. The file is then zipped
to about 2.5 MB and sent to the calling application.

On the client the file is unzipped to 57 MB file, before the application
starts reading it. I am using the load method of the DOM object model to
check whether it is a well formed XML. The problem starts as soon as I try
to load this XML file. The client machine becomes unresponsive for a long
time and the whole process from there on is very slow as I assume that the
XML is loaded into memory while it is parsed. I have to read the whole XML
file and do not need to search for any nodes/tags inside the XML.

I would like to know whether it is right to use XML in this scenario? Am I
better off by generating a simple delimited file instead of an XML file? The
size of the XML file is at least 3 folds as compared to any other delimited
file.

Any advice in this regard will be helpful.

Thanks and regards,

RAJ
 
B

bruce barker

you should use an xmlreader to parse the file. this will only require a
small amout of the xml (read buufer) to be in memory at a time.

-- bruce (sqlwork.com)
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

Forum statistics

Threads
473,744
Messages
2,569,483
Members
44,902
Latest member
Elena68X5

Latest Threads

Top