Performance comparison between screen scrapers

[Conrad Chu]

Does anyone know how the following screen scrapers perform against one
another?

* ScrAPI
* RubyfulSoup
* HTree
* Hpricot

I'm trying to write up a tool where a person enters in a URL, and I use
an AJAX call to scrape the contents of that URL for title, description,
etc. So speed is really important (I suppose, regular expressions would
be the fastest, but I need something that is tree-based and supports
HTML tidying)

Thanks
Conrad

 

[Jan Svitok]

Does anyone know how the following screen scrapers perform against one
another?

* ScrAPI
* RubyfulSoup
* HTree
* Hpricot

I'm trying to write up a tool where a person enters in a URL, and I use
an AJAX call to scrape the contents of that URL for title, description,
etc. So speed is really important (I suppose, regular expressions would
be the fastest, but I need something that is tree-based and supports
HTML tidying)

Thanks
Conrad

There was a comparision done on this list some time ago. Search for lib names.

 

[Timothy Goddard]

Does anyone know how the following screen scrapers perform against one
another?

* ScrAPI
* RubyfulSoup
* HTree
* Hpricot

I'm trying to write up a tool where a person enters in a URL, and I use
an AJAX call to scrape the contents of that URL for title, description,
etc. So speed is really important (I suppose, regular expressions would
be the fastest, but I need something that is tree-based and supports
HTML tidying)

Thanks
Conrad

I haven't used them all but Hpricot is fast (the parser is written in C
with Ragel), error tolerant and perfect for this task. Take a look at
its website for a guide on how to use it.