Q re nokogiri - searching Google and counting web pages withspecific text

[Philip Rhoades]

People,

I think using nokogiri with Ruby should be able to do what I want but
I'm not sure how to do it - I want to search Google and list or count
the number of web pages from a particular domain:

http://www.unitednations.org

that contain specific text eg "george harrison" or "oxfam" etc.

Could someone get me started?

Thanks,

Phil.
--
Philip Rhoades

GPO Box 3411
Sydney NSW 2001
Australia
E-mail: (e-mail address removed)

 

[Eric Hodel]

I think using nokogiri with Ruby should be able to do what I want
but I'm not sure how to do it - I want to search Google and list or
count the number of web pages from a particular domain:

http://www.unitednations.org

that contain specific text eg "george harrison" or "oxfam" etc.

Could someone get me started?

Why don't you use the google search API?

 

[Philip Rhoades]

Eric,

Why don't you use the google search API?

Seems like a good idea but I have hassles installing the dependency
ruby-SOAP4R rpm . .

Phil.
--
Philip Rhoades

GPO Box 3411
Sydney NSW 2001
Australia
E-mail: (e-mail address removed)

 

[Jason Stewart]

Eric,





Seems like a good idea but I have hassles installing the dependency
ruby-SOAP4R rpm . .

The google search API does not use SOAP anymore, that's been
deprecated for a while. You can use this:

http://github.com/visionmedia/google-search/tree/master

Or just use httparty or open-uri and parse the json that's returned
from the new search API.

Cheers,
Jason

 

[Eric Hodel]

Seems like a good idea but I have hassles installing the dependency
ruby-SOAP4R rpm . .

Looks like the SOAP API isn't being maintained anymore, use the modern
one:

http://code.google.com/apis/ajaxsearch/