Perl + SAX2 = slow?

[Jesse Thompson]

Greetings fell XML folk.

I've just gotten started making SAX filters in Perl. I was hoping to
build an XML templating engine this way, but the performance of
XML::SAX::Expat and XML::SAX::Writer *appear* to be unthinkably bad.

This code:

XML::SAX::Expat->new(Handler => XML::SAX::Writer->new( Output => '>-'
))->parse_uri("test.xml");

takes 1 second to parse a 5 kilobyte peice of XML on my machine. Being
a 500Mhz, that's 10kilobytes per gigahertz second.

Is this in any way normal? I was hoping to be able to process XML
about a hundred times this fast, maybe 1mb per gigahertz second, or
about a thousand clock cycles per byte of consumed XML. I think that
sounds reasonable for the bare parsing/writing of XML in a zippy
language like Perl.. so I have to assume I am doing something very
very wrong in my setup

So am I somehow getting the PurePerl parser instead of Expat? I'm
asking for Expat by name and die($parser) gives me
"XML::SAX::Expat=HASH(0x83b0090)".

Further, I simply cannot believe these results are typical. SAX was
invented to handle multi-megabyte documents that DOM can't fit
in-memory, but at these rates that would mean it would take a dual
4Ghz Xeon server twenty minutes just to parse a 100 megabyte XML
document and write it back out to disk unaltered. What happens when
you want to plug in a pipeline of any merit? I'm not really sure how
fast DOM is, but big servers can have 3 gigabytes of ram (100mb * 30x
for dom memory bloat), and I know my web browser reads XHTML and
builds DOM trees out of it at better than 10 kb per gigahertz second..
So my results must, must be flawed somehow.

Does anyone know what could be going wrong, or how fast that code
snippet should be parsing XML? If I'm waiting around and then only
transforming nested XML nodes that match certain criterion (custom tag
names, attribute names, or maybe even just a custom namespace), sort
of like a templating engine (replace fake template data with data
pulled from a DB, or todays date for instance) would that mean there
is an XML solution more efficient for my goals than SAX? Twig maybe,
or Essex? (I can't find much to read about Essex)

I was hoping to become an XML evangelist because I love everything
about it (and even understand namespaces and encodings but these
results kind of made it feel like my bubble had burst. Everyone I know
keeps molesting their XML projects with Regex, which just seems to me
so much like towing a inoperative car around with a team of horses.

Any insight will be appreciated.

- - Jesse Thompson
Lightsecond Technologies
http://www.lightsecond.com/

 

[=?ISO-8859-1?Q?J=FCrgen_Kahrs?=]

Further, I simply cannot believe these results are typical. SAX was
invented to handle multi-megabyte documents that DOM can't fit

Reading XML files of several GigaBytes length
with a SAX-implementation (Expat) took me some
minutes. I have tested this while integrating
Expat into GNU Awk.

I was hoping to become an XML evangelist because I love everything
about it (and even understand namespaces and encodings but these
results kind of made it feel like my bubble had burst. Everyone I know
keeps molesting their XML projects with Regex, which just seems to me
so much like towing a inoperative car around with a team of horses.

I cannot help you with Perl, but maybe xmlgawk
can help you. This is GNU Awk extended (experimentally)
with Expat.

 

[Jesse Thompson]

Reading XML files of several GigaBytes length
with a SAX-implementation (Expat) took me some
minutes. I have tested this while integrating
Expat into GNU Awk.

Yeah, well several gigabytes in several minutes (lets say 1 gigabyte
per minute) on a 1 Gigahertz machine would be 16Mb/Ghz*s. That is so
fast I wouldn't know what to do with myself: one thousand six hundred
times faster than my results are showing (10kb/Ghz*s). Break me off a
peice, would you?

I cannot help you with Perl, but maybe xmlgawk
can help you. This is GNU Awk extended (experimentally)
with Expat.

As interesting as that sounds I don't know anything about Awk.. But if
it's fast in awk it must also be fast in Perl. I simply have to
believe my results are atypical for Perl::SAX.

- - Jesse

 

[=?ISO-8859-1?Q?J=FCrgen_Kahrs?=]

As interesting as that sounds I don't know anything about Awk.. But if
it's fast in awk it must also be fast in Perl. I simply have to
believe my results are atypical for Perl::SAX.

I am a bit surprised to hear about something built upon
SAX that is slow. Maybe this comparison helps you:

http://xmlbench.sourceforge.net/results/benchmark/index.html

 

[Jesse Thompson]

I am a bit surprised to hear about something built upon

SAX that is slow. Maybe this comparison helps you:

http://xmlbench.sourceforge.net/results/benchmark/index.html

Well, since you just quoted half the XML parsing benchmarks I've seen
on the internet, I'll go ahead and post the other half
http://www.xml.com/pub/a/Benchmark/article.html

On that page you can notice something like a 30x speed difference
between C expat and Perl Expat. You're quoting a C/Java benchmark.
It's still not enough to explain my results though, Perl Expat (in
what looks like SAX1) is still shown to process 32 times faster than
my demonstration in SAX2. Perl::Expat vs. Perl::SAX::Expat *could*
explain that difference, but I don't buy it.

Just to recap, all I have is this:
XML::SAX::Expat->new(Handler => XML::SAX::Writer->new( Output => '>-'
))->parse_uri("test.xml");

That is the simplest possible command: "eat this file with expat and
write it back out again". There is no application built on top of it
yet, that's just the raw drivers turning my CPU into a radiator.

- - Jesse

 

[Malcolm Dew-Jones]

Jesse Thompson ([email protected]) wrote:
: Greetings fell XML folk.

: I've just gotten started making SAX filters in Perl. I was hoping to
: build an XML templating engine this way, but the performance of
: XML::SAX::Expat and XML::SAX::Writer *appear* to be unthinkably bad.

: This code:

: XML::SAX::Expat->new(Handler => XML::SAX::Writer->new( Output => '>-'
: ))->parse_uri("test.xml");

: takes 1 second to parse a 5 kilobyte peice of XML on my machine. Being
: a 500Mhz, that's 10kilobytes per gigahertz second.

: Is this in any way normal?

Might I suggest you ask this on

comp.lang.perl.modules

Various people hang out there who might have some definitive
answers, or useful suggestions.

$0.02

 

[=?ISO-8859-1?Q?J=FCrgen_Kahrs?=]

Well, since you just quoted half the XML parsing benchmarks I've seen
on the internet, I'll go ahead and post the other half
http://www.xml.com/pub/a/Benchmark/article.html

Thanks for the link. Interesting.

 

[Bjoern Hoehrmann]

* Jesse Thompson wrote in comp.text.xml:

This code:

XML::SAX::Expat->new(Handler => XML::SAX::Writer->new( Output => '>-'
))->parse_uri("test.xml");

takes 1 second to parse a 5 kilobyte peice of XML on my machine. Being
a 500Mhz, that's 10kilobytes per gigahertz second.

That does not say much. Did you eliminate all overhead from this test?
Like loading Perl, all the modules, initializing the parser, etc? If
not, that would explain a lot. Also note that a 10kb document is a bad
test case if you want to know how it performs with several MBs of data.
Further note that XML::SAX::Expat is a bad choice if you desire good
performance, what you are doing here is

Expat -> XML:arser::Expat -> XML:arser -> XML::SAX::Expat

along with other modules like XML::SAX::Base in the chain which is quite
some overhead. A better choice would be to use XML::SAX::ExpatXS which
omits XML:arser::Expat and XML:arser and other parts from the chain,
and is thus much faster. Other modules like XML::LibXML::SAX might give
even better performance. Another problem with your test is that you
generate XML in the chain through XML::SAX::Writer and then through
STDOUT, both of which might significantly slow down processing speed.
In other words, there might be many reasons why this might show poor
performance.

Using http://lists.w3.org/Archives/Public/www-archive/2004Mar/0169.html
as input your script takes 96 seconds on a 1066 MHz Mobile Celeron FWIW.
That's about 77KB per second. With no handler 40 seconds, 184KB/s. With
XML::LibXML::SAX and no handler 18 seconds, 409KB/s. And with the direct
Expat wrapper XML::SAX::ExpatXS and no handler 9 seconds, 819KB/s. Each
for a single run though, so the results are not all that meaningful. For
better results see `perldoc Benchmark`.

 

[=?ISO-8859-1?Q?J=FCrgen_Kahrs?=]

Using http://lists.w3.org/Archives/Public/www-archive/2004Mar/0169.html
as input your script takes 96 seconds on a 1066 MHz Mobile Celeron FWIW.
That's about 77KB per second. With no handler 40 seconds, 184KB/s. With
XML::LibXML::SAX and no handler 18 seconds, 409KB/s. And with the direct
Expat wrapper XML::SAX::ExpatXS and no handler 9 seconds, 819KB/s. Each
for a single run though, so the results are not all that meaningful. For
better results see `perldoc Benchmark`.

This is interesting. For comparison: On a 1200 MHz AMD Duron,
xmlgawk parses between 4000 and 5000 KB/s.

 

[Bjoern Hoehrmann]

* Jürgen Kahrs wrote in comp.text.xml:

This is interesting. For comparison: On a 1200 MHz AMD Duron,
xmlgawk parses between 4000 and 5000 KB/s.

As I wrote, these results don't tell you much. If you have no handler
the processor might be optimized to ignore all data and just evaluate
the document for well-formedness; others might not as it is uncommon to
have no start_element handler, for example. SGML:arser::OpenSP, a soon
to be released SGML/XML processor based on OpenSP is optimized like that
and for 100 iterations for the documented generated via

`get http://www.w3.org/TR/REC-xml | tidy -utf8 -n --doctype omit`

with no handler versus with handlers for all events that don't do
anything,

Rate OpenSP1 OpenSP2
OpenSP1 1.34/s -- -87%
OpenSP2 10.4/s 677% --

which just shows that creating Perl data structures is quite expensive.
XML:arser has similar optimizations,

Rate XML:arser1 XML:arser2
XML:arser1 7.25/s -- -81%
XML:arser2 39.0/s 438% --

and both compared

Rate OpenSP2 XML:arser2 OpenSP1 XML:arser1
OpenSP2 1.35/s -- -82% -87% -97%
XML:arser2 7.34/s 445% -- -31% -81%
OpenSP1 10.6/s 685% 44% -- -73%
XML:arser1 39.0/s 2795% 431% 269% --

XML::SAX::Expat is not optimized like that in any way and does many more
things than what XML:arser1 would do. And as I wrote, the input is
highly relevant, too, using the 7,2MB example above (which is quite
different in terms of markup/pcdata, etc.) I get

s/iter OpenSP1 XML:arser1
OpenSP1 3.89 -- -82%
XML:arser1 0.681 470% --

which is quite different from the 269% before. So here you'd get a rate
of about 10MB/s using XML:arser with throw-away processing versus the
about 2MB/s of SGML:arser::OpenSP.

 

[=?ISO-8859-1?Q?J=FCrgen_Kahrs?=]

As I wrote, these results don't tell you much. If you have no handler
the processor might be optimized to ignore all data and just evaluate
the document for well-formedness; others might not as it is uncommon to
have no start_element handler, for example. SGML:arser::OpenSP, a soon

Good argument, of course. But I really made sure that
all handlers were active. xmlgawk is stupid enough to
have all handlers active all the time. I also made sure
that the XML file was really parsed.

things than what XML:arser1 would do. And as I wrote, the input is
highly relevant, too, using the 7,2MB example above (which is quite
different in terms of markup/pcdata, etc.) I get

s/iter OpenSP1 XML:arser1
OpenSP1 3.89 -- -82%
XML:arser1 0.681 470% --

which is quite different from the 269% before. So here you'd get a rate
of about 10MB/s using XML:arser with throw-away processing versus the
about 2MB/s of SGML:arser::OpenSP.

Interesting, I just downloaded the file and xmlgawk (based
on Expat) parses around 2MB/s on a Pentium with 550 MHz;
which is not much different from 4MB/s with a Duron 1200 MHz.

 

[William Park]

Good argument, of course. But I really made sure that
all handlers were active. xmlgawk is stupid enough to
have all handlers active all the time. I also made sure
that the XML file was really parsed.


Interesting, I just downloaded the file and xmlgawk (based
on Expat) parses around 2MB/s on a Pentium with 550 MHz;
which is not much different from 4MB/s with a Duron 1200 MHz.

On my P3/800, the 7.5MB file (W3C-Member-Validity.xml) takes
- 6sec for Bash + Expat --> 1.2MB/s
- 2.3sec for Gawk + Expat --> 3.2MB/s
which is in agreement with you data.

 

[Jesse Thompson]

Thank you Jürgen and Bjoern, that has all been very enlightning

So the way Bjoern puts it, Expat is quite fast and due to some
streamlining ExpatXS is faster (it bumped me up to 33kb/Ghz*s) (libXML
appears incompatable with my Glib2.2 system at the moment) but that
XML::SAX::Writer is very very slow.

The reason I'm keeping that in is because it /is/ a constant in my
operations.. in that I'm trying to set up a mechanism where I read an
XML file, I have a flexible chain of filters that transform the file
in the SAX stream, and then it gets written back out again. I don't
know if there's an obvious way to factor out need for Writer in a
scenario like this. Passing an event to a C-module has to be faster
than writing my own writing routines in Perl to avoid the SAX faucet.

But if Writer is the slowbe I guess I'll want to research a faster
Writer? Also I have to wonder about SAX in general for my goals. I
don't think SAX supports any prefilters. My filters will be looking
for XML tags with certain names or attributes before they begin doing
their jobs, so if there was a prefilter, like my filter said "skip me
unless tagname =~ /^abc/ (or) tagname eq 'abc' (or)
defined(attribute->{'{mynamespace}process'})" that might make things
much quicker.

Otherwise, since nearly all of my files will be less than 200k, maybe
I should start looking at DOM.

Jürgen: your xmlgawk project sounds very very interesting. I took a
look at some Gawk tutorials and it looks like a capable tool for very
many applications, more so with XML support. Is there anywhere I could
snag it from? Google seems only to know of some of your discussions
with William over the project

Thank you again!

- - Jesse

 

[=?ISO-8859-1?Q?J=FCrgen_Kahrs?=]

Jürgen: your xmlgawk project sounds very very interesting. I took a
look at some Gawk tutorials and it looks like a capable tool for very
many applications, more so with XML support. Is there anywhere I could
snag it from? Google seems only to know of some of your discussions
with William over the project

There is very few doc about xmlgawk currently.
A collection of pointers can be found in my
posting to comp.lang.awk on 2004-08-13:

http://groups.google.de/groups?hl=d...groups?hl=de&lr=&ie=UTF-8&group=comp.lang.awk