Any parser can handle 2.1GB+ file?

[Craig Petty]

Well, I tried to use SAX to process a large document. However, we are
hitting an unfortunate limit in the xerces sax parser. I wish they
had used longs or unsigned ints (see below, which is throwing a
runtime exception) to keep track of the position in the document.
Atleast I'm guessing thats whats happening here.

Any ideas?


(from a utils class is xerces)
221 public int addString(int offset, int length) {
222 int chunk = offset >> CHUNK_SHIFT;
223 if (chunk != fChunk) {
224 if (fPreviousChunk == null)
225 throw new RuntimeException(new
ImplementationMessages().createMessage(null,
ImplementationMessages.INT_PCN, 0, null));
226 return fPreviousChunk.addString(offset, length);
227 }
228 int lastChunk = (offset + length - 1) >> CHUNK_SHIFT;
229 if (chunk == lastChunk) {
230 addRef();
231 return fStringPool.addString(this, offset &
CHUNK_MASK, length);
232 }
233 String str = toString(offset & CHUNK_MASK, length);
234 return fStringPool.addString(str);
235 }

here's the java stack trace...

java.lang.RuntimeException: Internal Error: fPreviousChunk == NULL
at org.apache.xerces.framework.XMLParser.parse(Unknown Source)
at org.apache.xerces.framework.XMLParser.parse(Unknown Source)
at Test.main(Test.java:177)