caching large XML files in an ASP.net application

[Guest]

hi all,

im currently writing an ASP.net front end to display data found in an XML
file. the XML file is quite large (700k or so), and so takes a while to
load. it will be changing around every second, so needs to be loaded at
least at that rate. at the moment, the ASP application loads the xml every
refresh - im thinking that this may cause problems when there are 20 clients
each refreshing once per second.

the solution im thinking of is to have a thread accessible to all the ASP
instances that deals with loading and refreshing the XML into a dataset. it
would do this every second and make the dataset available to all ASP
instances to read from.

how would this work? assuming you can create threads as normal in an ASP
application, how do i make it accessible to all of them? by making it
static? which applicaiton thread instance would be in charge of starting and
stopping the XML reading thread? i mean the first should start it and the
last to exit should quit it, but can this be detected? or is a timeout a
best way to quit it (not accessed in a minute -> quit). is this thread a
particularly efficient way of doing this, or would it put another
unnecessary load on to the webserver?

if i dont use a thread, whats the best way to implement the XML reading? it
would have to be a shared dataset, but only updated once every second - how
would this timekeeping be implemented by the various application threads?

thanks in advance, and sorry for the many questions!

spammy

 

[Janaka]

Spammy

sorry i haven't tried using any of the Threading classes with ASP.NET,
however you might be able to change some other parts of the application to
solve your problem?

first of all is all the data changing in the xml file? If possible i've
found it easier to split out a large xml file into several smaller files and
use xslt on them to make a kind of view of the combined data.

try seeing if its possible to use the Cache object to store the data
(however if the cache has to be cleared every second i don't think this
would give you a large benefit).

J

 

[Guest]

janaka,

thanks for the response. yes, the whole xml file is changing. further all of
it needs to be made available to each client, as they would be viewing
arbitrary parts of it.

the cache object sounds ideal if the same cache is available to all loading
pages, but like you said it would need to be refreshed each second. if this
refreshing can be done in the backgroud by another thread, then it may work
out well. the question is however - can a thread be set up to do this? ive
been searching groups further and it sounds like asp threading isnt worth
the bother.

has anyone managed to create a working threading ASP application?

spammy

 

[Jason DeFontes]

You can put an object in the cache and tie it's expiration to a file on
the file system, so when the XML file changes it will trigger the
cache's expiration event, and you can reload the data and recreate the
dataset. If you store the dataset in the Application then all your
requests can access the same dataset object.

-Jason