XQuery on big files

Discussion in 'XML' started by Pascal Sartoretti, Apr 10, 2007.

  1. Hello,

    I want to process big files (e.g. 100 MB) with XQuery. Are there XQuery
    processors that work in a "SAX-like" way (and not DOM-like), i.e. which
    don't assume that they have everything in memory ?

    Thanks for any help

    Pascal
    Pascal Sartoretti, Apr 10, 2007
    #1
    1. Advertising

  2. Pascal Sartoretti wrote:
    > Hello,
    >
    > I want to process big files (e.g. 100 MB) with XQuery. Are there XQuery
    > processors that work in a "SAX-like" way (and not DOM-like), i.e. which
    > don't assume that they have everything in memory ?
    >
    > Thanks for any help
    >
    > Pascal


    hi,

    with RefleX, you can process big files with XPath ; unlike many
    streaming-XPath processors, RefleX doesn't restrict the XPath syntax,
    but you must be aware that using XPath queries such as //* on the root
    would cause caching the complete document causing an OutOfMemoryError ;
    however, if you intend to use backwards axis, you'd rather "cast" small
    chunks of your input in DOM subtrees, and use the full capabilities of XPath

    an example here :
    http://reflex.gforge.inria.fr/tutorial.html#N801C30
    (the input file is not as big as yours, but big enough to make a simple
    DOM parsing fail ; it was the case at the time I wrote this example)

    in this example, you'll see how an inline-filter can split a big file in
    about 13000 small files with 2 flavours : SAX to SAX and SAX to DOM

    enjoy !

    --
    Cordialement,

    ///
    (. .)
    --------ooO--(_)--Ooo--------
    | Philippe Poulard |
    -----------------------------
    http://reflex.gforge.inria.fr/
    Have the RefleX !
    Philippe Poulard, Apr 10, 2007
    #2
    1. Advertising

  3. Pascal Sartoretti wrote:

    > I want to process big files (e.g. 100 MB) with XQuery. Are there XQuery
    > processors that work in a "SAX-like" way (and not DOM-like), i.e. which
    > don't assume that they have everything in memory ?


    SAXON-SA for XSLT (not for XQuery it seems) has a streaming processing
    mode, see
    <http://www.saxonica.com/documentation/sourcedocs/serial.html>


    --

    Martin Honnen
    http://JavaScript.FAQTs.com/
    Martin Honnen, Apr 10, 2007
    #3
  4. Pascal Sartoretti

    Tony Lavinio Guest

    DataDirect XQuery is optimized for very large files. It supports both
    SAX and StAX modes. We've tested it with multi-gigabyte files.

    See http://www.xquery.com

    There is also an embedded version inside of Stylus Studio that you
    can try out. See http://www.stylusstudio.com



    Pascal Sartoretti wrote:
    > Hello,
    >
    > I want to process big files (e.g. 100 MB) with XQuery. Are there XQuery
    > processors that work in a "SAX-like" way (and not DOM-like), i.e. which
    > don't assume that they have everything in memory ?
    >
    > Thanks for any help
    >
    > Pascal


    --
    Tony Lavinio <> DataDirect <> Stylus Studio XML <>
    XQuery, XSLT, XML Schema and EDI Toolset <> http://www.stylusstudio.com/
    <> There is no problem that brute force and ignorance cannot overcome <>
    Tony Lavinio, Apr 10, 2007
    #4
  5. ....and it's free of charge :)

    --
    Cordialement,

    ///
    (. .)
    --------ooO--(_)--Ooo--------
    | Philippe Poulard |
    -----------------------------
    http://reflex.gforge.inria.fr/
    Have the RefleX !
    Philippe Poulard, Apr 10, 2007
    #5
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Replies:
    11
    Views:
    696
    Roedy Green
    Sep 18, 2005
  2. Shaguf
    Replies:
    0
    Views:
    329
    Shaguf
    Dec 24, 2008
  3. Shaguf
    Replies:
    0
    Views:
    432
    Shaguf
    Dec 26, 2008
  4. Shaguf
    Replies:
    0
    Views:
    219
    Shaguf
    Dec 26, 2008
  5. Shaguf
    Replies:
    0
    Views:
    202
    Shaguf
    Dec 24, 2008
Loading...

Share This Page