webservice based on large XML file as datasource

Discussion in 'ASP .Net' started by Tarun Jain, Dec 1, 2003.

  1. Tarun Jain

    Tarun Jain Guest

    Hi,
    I am trying to create a WebService which uses data stored in a large
    xml file (200-400 MB). Obviously parsing the xml for every webservice
    request is not feasible since that would make the response very slow.

    What should I do to make sure that I can load the xml file once and
    keep it in memory. Then I should be able to use the same in-memory
    parsed xml document.

    Thanks

    Tarun
    -=-
    Tarun Jain, Dec 1, 2003
    #1
    1. Advertising

  2. Tarun Jain

    SF Guest

    Tarun,

    have you considered reading it in a DataSet and then use DataViews
    to do the querying? Or you could read it in a XMLDocument and then
    use XPath to access the specific elements you are looking for.

    In any case I'd put the data structure you use for in-memory storage
    in the application cache and use a CacheDependency to keep
    the original XML file and the in-memory data in sync.

    Hope that helps
    Stefan

    "Tarun Jain" <> wrote in message
    news:...
    > Hi,
    > I am trying to create a WebService which uses data stored in a large
    > xml file (200-400 MB). Obviously parsing the xml for every webservice
    > request is not feasible since that would make the response very slow.
    >
    > What should I do to make sure that I can load the xml file once and
    > keep it in memory. Then I should be able to use the same in-memory
    > parsed xml document.
    >
    > Thanks
    >
    > Tarun
    > -=-
    SF, Dec 2, 2003
    #2
    1. Advertising

  3. Tarun Jain

    bruce barker Guest

    you just store in a static (shared in vb). while accessing a dom is
    threadsafe, updating is not. you will need to supply locking for your
    threads. if you need to a lot of updates. then you will not want to lock the
    whole doms, but rather lock at the lowest node you need.

    -- bruce (sqlwork.com)



    "Tarun Jain" <> wrote in message
    news:...
    > Hi,
    > I am trying to create a WebService which uses data stored in a large
    > xml file (200-400 MB). Obviously parsing the xml for every webservice
    > request is not feasible since that would make the response very slow.
    >
    > What should I do to make sure that I can load the xml file once and
    > keep it in memory. Then I should be able to use the same in-memory
    > parsed xml document.
    >
    > Thanks
    >
    > Tarun
    > -=-
    bruce barker, Dec 2, 2003
    #3
  4. Tarun Jain

    Tarun Jain Guest

    Thanks for your input so far.

    A more complete description of the problem is that

    --> I have to only read the data. There are no updates to
    the xml document that are required.

    --> I am planning to use XPath to retrieve the data.

    --> I am planning to use the cache. But the problem is
    that this web service will not see a lot of traffic,
    something like 10 queries a day. I know that ASP.NET shuts
    down the application when there are no more sessions.
    So what will happen is that if I cache the XmlDocument
    object, once the application shuts down the cache will
    disappear along with it.

    What can I do to get around these problems ??

    Tarun
    -=-


    >-----Original Message-----
    >Tarun,
    >
    >have you considered reading it in a DataSet and then use

    DataViews
    >to do the querying? Or you could read it in a XMLDocument

    and then
    >use XPath to access the specific elements you are looking

    for.
    >
    >In any case I'd put the data structure you use for in-

    memory storage
    >in the application cache and use a CacheDependency to keep
    >the original XML file and the in-memory data in sync.
    >
    >Hope that helps
    >Stefan
    >
    >"Tarun Jain" <> wrote in message
    >news:...
    >> Hi,
    >> I am trying to create a WebService which uses data

    stored in a large
    >> xml file (200-400 MB). Obviously parsing the xml for

    every webservice
    >> request is not feasible since that would make the

    response very slow.
    >>
    >> What should I do to make sure that I can load the xml

    file once and
    >> keep it in memory. Then I should be able to use the

    same in-memory
    >> parsed xml document.
    >>
    >> Thanks
    >>
    >> Tarun
    >> -=-

    >
    >
    >.
    >
    Tarun Jain, Dec 2, 2003
    #4
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. BLF
    Replies:
    4
    Views:
    1,268
    Mike Treseler
    Aug 7, 2004
  2. Ron
    Replies:
    5
    Views:
    9,216
    Steven Cheng[MSFT]
    Oct 13, 2004
  3. Replies:
    0
    Views:
    2,888
  4. 8
    Replies:
    1
    Views:
    1,540
  5. Replies:
    5
    Views:
    852
    Xho Jingleheimerschmidt
    Apr 2, 2009
Loading...

Share This Page