Any parser can handle 2.1GB+ file?

Discussion in 'XML' started by Craig Petty, Sep 13, 2003.

  1. Craig Petty

    Craig Petty Guest

    Well, I tried to use SAX to process a large document. However, we are
    hitting an unfortunate limit in the xerces sax parser. I wish they
    had used longs or unsigned ints (see below, which is throwing a
    runtime exception) to keep track of the position in the document.
    Atleast I'm guessing thats whats happening here.

    Any ideas?


    (from a utils class is xerces)
    221 public int addString(int offset, int length) {
    222 int chunk = offset >> CHUNK_SHIFT;
    223 if (chunk != fChunk) {
    224 if (fPreviousChunk == null)
    225 throw new RuntimeException(new
    ImplementationMessages().createMessage(null,
    ImplementationMessages.INT_PCN, 0, null));
    226 return fPreviousChunk.addString(offset, length);
    227 }
    228 int lastChunk = (offset + length - 1) >> CHUNK_SHIFT;
    229 if (chunk == lastChunk) {
    230 addRef();
    231 return fStringPool.addString(this, offset &
    CHUNK_MASK, length);
    232 }
    233 String str = toString(offset & CHUNK_MASK, length);
    234 return fStringPool.addString(str);
    235 }

    here's the java stack trace...

    java.lang.RuntimeException: Internal Error: fPreviousChunk == NULL
    at org.apache.xerces.framework.XMLParser.parse(Unknown Source)
    at org.apache.xerces.framework.XMLParser.parse(Unknown Source)
    at Test.main(Test.java:177)
     
    Craig Petty, Sep 13, 2003
    #1
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Bryan Parkoff

    Use Real 1GB and Fake 3GB Memory?

    Bryan Parkoff, Mar 9, 2005, in forum: C++
    Replies:
    1
    Views:
    417
    Victor Bazarov
    Mar 9, 2005
  2. Bennie
    Replies:
    6
    Views:
    459
    =?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=
    Mar 27, 2005
  3. Replies:
    0
    Views:
    332
  4. spaceman-spiff

    Trying to parse a HUGE(1gb) xml file

    spaceman-spiff, Dec 20, 2010, in forum: Python
    Replies:
    41
    Views:
    2,319
  5. spaceman-spiff

    Re: Trying to parse a HUGE(1gb) xml file

    spaceman-spiff, Dec 20, 2010, in forum: Python
    Replies:
    3
    Views:
    733
    John Nagle
    Dec 22, 2010
Loading...

Share This Page