Spidering Hacks

[alexxx.magni]

Working on a project for a spider able to gather info for my books
list,
I read a bit of "Spidering Hacks" (OReilly) - which fortunately is all
in Perl.

I was deciding to use the Google API, so I looked also to "Google
Hacks".

Up to now I never registered for a Google API, so when going to the
place suggested by the book (http://www.google.com/apis/download.html)
I was redirected to http://code.google.com, where a huge number of
possible APIs are present.

Do you know where I have to look, to simply obtain the (authorization?
key?) to use the Google search api?

thanks!

Alessandro

 

[Spiros Denaxas]

Working on a project for a spider able to gather info for my books
list,
I read a bit of "Spidering Hacks" (OReilly) - which fortunately is all
in Perl.

I was deciding to use the Google API, so I looked also to "Google
Hacks".

Up to now I never registered for a Google API, so when going to the
place suggested by the book (http://www.google.com/apis/download.html)
I was redirected tohttp://code.google.com, where a huge number of
possible APIs are present.

Do you know where I have to look, to simply obtain the (authorization?
key?) to use the Google search api?

thanks!

Alessandro

Hi,

which API does the book prompt you to use? AFAIK, Google has both a
SOAP and an AJAX version of their search API.
Their SOAP API is no longer open to new registrations and thus cannot
be used (unless of course you already have an SOAP API code).

The AJAX API page can be found at : http://code.google.com/apis/ajaxsearch/
You can register for a key at : http://code.google.com/apis/ajaxsearch/signup.html

Hope this helps,

Spiros

ps. You might also want to check out the GoogleHack proect,
http://google-hack.sourceforge.net/

 

[alexxx.magni]

From what I just read, SOAP was just what I needed - if only I had a
key, which I havent.
And AYAX if Javascript only, right? So I'm out of luck, with Perl -
damn!

Thank you for your help...

Alessandro




Spiros Denaxas ha scritto:

 

[alexxx.magni]

Yes, I had it, and other people kindly answered it.
Seriously, however, I don't see the reason for such narrowness of
mind...
I documented myself on Spidering Hacks, a Perl-only book on such kind
of programs, and intended to find a way to develop my own using a
Google <Perl> API.
If you believe this group forbids this kind of questions, it's your
opinion, and yours only

Best regards

Alessandro

Sherman Pendley ha scritto:

 

[Randal L. Schwartz]

alexxx> Do you know where I have to look, to simply obtain the (authorization?
alexxx> key?) to use the Google search api?

If I recall, Google withdrew that API about six months ago, replacing with a
page-embedded search result API that ensures that their ads will also show up
in the right place. Yech.

If you want programmatic access to a search result, Yahoo is now the best
player in the space, offering a variety of searches, including JSON results
for direct embedding with no server-side code required. See
http://developer.yahoo.com/search/ and Yahoo::Search in the CPAN, and my usage
of same at <http://www.stonehenge.com/merlyn/LinuxMag/col85.html>.

So much for Google "doing no evil". Yahoo for the win.

print "Just another Perl hacker,"; # the original!

 

[Ben Morrow]

[top-posting fixed. don't do that]

Sherman Pendley ha scritto:

If you believe this group forbids this kind of questions, it's your
opinion, and yours only

It's not Sherm's opinion only; see the section in the Posting Guidelines
called 'Question should be about Perl, not about the application area'.
Those guidelines were drawn up by general consensus among the regulars
here.

Ben

 

[Ted Zlatanov]

On Mon, 26 Nov 2007 06:53:34 -0800 (e-mail address removed) (Randal L. Schwartz) wrote:

RLS> If I recall, Google withdrew that API about six months ago, replacing with a
RLS> page-embedded search result API that ensures that their ads will also show up
RLS> in the right place. Yech.
....
RLS> So much for Google "doing no evil". Yahoo for the win.

Withdrawing a public API is an inconvenience to its users but hardly
qualifies as "evil" in my opinion. Let's be fair.

Ted

 

[Randal L. Schwartz]

Ted> Withdrawing a public API is an inconvenience to its users but hardly
Ted> qualifies as "evil" in my opinion. Let's be fair.

I disagree. The closest equivalent API (the client-side embedded javascript)
burdens all results with google ads. So, this seems to be a change only to
line the pocketbooks of the stockholders. Still convinced it's not evil?

print "Just another Perl hacker,"; # the original

 

[alexxx.magni]

Upside-down, and quoting signatures. Please don't do that. Have you read
the posting guidelines that appear here frequently?



Yes, but that was another thread - and I was one of those people.

My point was that the question in *this* thread has nothing to do with Perl.
Ruby, Python, C, Java, or any other programmers all obtain their Google keys
the same way - the procedure for doing so has nothing whatsoever to do with
the language with which you intend to use it.


It's simple. There are many Perl programmers here, and only a small subset
of us may have used Google's API. But, there's nothing Perl-specific about
obtaining a key; a programmer who's done so to use the key in his Ruby app
could help you just as well as anyone here.

By asking your Google questions in a language-neutral forum that's dedicated
to such questions, and your Perl questions in a forum that's dedicated to
those, you increase your chances of getting an answer.

Probably then it's my mistake.
I often asked for help/advice in this group, since I use Perl heavily,
and it seldom let me down - I used to feel rather at home here.
Granted, I agree that my question was not related to <perl> per se,
but to a <perl API>, but I failed to see it as a big problem - I wasnt
spamming (I posted on comp.lang.perl.misc only) and I believed the
*misc leaf allowed this.


Alessandro

 

[xhoster]

Ted> Withdrawing a public API is an inconvenience to its users but hardly
Ted> qualifies as "evil" in my opinion. Let's be fair.

I disagree. The closest equivalent API (the client-side embedded
javascript) burdens all results with google ads. So, this seems to be a
change only to line the pocketbooks of the stockholders. Still convinced
it's not evil?

Yes. Not being a charity is not the same thing as being evil.

Xho

--
-------------------- http://NewsReader.Com/ --------------------
The costs of publication of this article were defrayed in part by the
payment of page charges. This article must therefore be hereby marked
advertisement in accordance with 18 U.S.C. Section 1734 solely to indicate
this fact.

 

[Ted Zlatanov]

Ted> Withdrawing a public API is an inconvenience to its users but hardly
Ted> qualifies as "evil" in my opinion. Let's be fair.

RLS> I disagree. The closest equivalent API (the client-side embedded
RLS> javascript) burdens all results with google ads. So, this seems to
RLS> be a change only to line the pocketbooks of the stockholders.

I've worked for a content delivery company and I can assure you it's not
free to provide bandwidth and server load. Furthermore, every
for-profit company's express mission is to line the pocketbooks of the
shareholders.

RLS> Still convinced it's not evil?

I'd say Yahoo's actions have been a lot more morally and legally
suspect, actually, bringing them before the US Congress a few weeks ago:

http://www.theregister.co.uk/2005/09/07/yahoo_china_dissident_case/
http://www.internetnews.com/breakingnews/article.php/3710916

In the context of 10 years in jail for speaking your mind, withdrawing a
public API doesn't seem all that evil.

Ted

 

[alexxx.magni]

Don't lose sleep over it - it wasn't a very big one.


Please feel free to continue feeling that way! Granted, my first "did you
have a Perl question" was kind of snarky, and I apologize for that. But I
honestly intended to be helpful - it's my opinion that choosing the right
group for a question is as critical a research skill as choosing the right
keywords for a Google search.

Well, thank you for the kind words, I feel better!

Alessandro

P.S. still unsure, however, which would have been the right
group??? ;-)