Parsing HTML/XML documents

[pabloski]

I need to parse real world HTML/XML documents and I found two nice python
solution: BeautifulSoup and Tidy.

However I found pyXPCOM that is a wrapper for Gecko. So I was thinking
Gecko surely handles bad html in a more consistent and error-proof way
than BS and Tidy.

I'm interested in using Mozilla DOM from inside a Python script, however
I'm a bit confused about how can I use pyXPCOM to accomplish this job.

Any suggestions?

 

[Stefan Behnel]

I need to parse real world HTML/XML documents and I found two nice python
solution: BeautifulSoup and Tidy.

There's also lxml, in case you want a real XML tool.
http://codespeak.net/lxml/
http://codespeak.net/lxml/dev/parsing.html#parsers

However I found pyXPCOM that is a wrapper for Gecko. So I was thinking
Gecko surely handles bad html in a more consistent and error-proof way
than BS and Tidy.

I'm interested in using Mozilla DOM from inside a Python script, however
I'm a bit confused about how can I use pyXPCOM to accomplish this job.

I've never used it, but I doubt Gecko would yield substantially better results
than any of the three above. You're dealing with broken data here, so it just
depends on your input which one of them wins.

Stefan

 

[Max M]

Stefan Behnel skrev:

There's also lxml, in case you want a real XML tool.
http://codespeak.net/lxml/
http://codespeak.net/lxml/dev/parsing.html#parsers

I have used both BeautiullSoup and lxml. They are both good tools.

lxml is blindingly fast compared to BeautifulSoup though.

A simple tool for importing contact information from 6000 xml files of
23 MBytes into Zope runs in about 30 seconds. No optimisations at all.
Just inefficient xpath expressions.

That is pretty good in my book.

--

hilsen/regards Max M, Denmark

http://www.mxm.dk/
IT's Mad Science