Question about Mechanize and Google API

  • Thread starter Jesús Gabriel y Galán
  • Start date
J

Jesús Gabriel y Galán

Hi,

I'm trying to use Mechanize to retrieve spreadsheets from Google Docs
using the Google API. So far I got the ClientLogin to work, but I
can't seem to be able to use the Auth token they give me back. This is
what I have:

require 'logger'
require 'mechanize'

agent = WWW::Mechanize.new {|a| a.log = Logger.new("test.log")}
response = agent.post("https://www.google.com/accounts/ClientLogin",
"Email" => "(e-mail address removed)", "Passwd" => "xxxxx", "service" => "wise",
"source" => "personal-test-1.0", "accountType" => "GOOGLE")
h = {}
response.body.each {|line| a = line.split("="); h[a[0]] = a[1]}
auth = h["Auth"]
agent.basic_auth("GoogleLogin", auth)
# this didn't work either
#agent.auth("GoogleLogin", auth)
page = agent.get("http://spreadsheets.google.com/feeds/spreadsheets/private/full")

The problem is, according to google I have to:
"After a successful authentication request, use the Auth value to
create an Authorization header for each request:

Authorization: GoogleLogin auth=yourAuthValue"

I've checked and the ClientLogin is working and returning me an Auth
token, but I can't get Mechanize to use it. In the Mechanize logs I
see this:
I, [2007-10-08T17:01:52.211778 #24929] INFO -- : Net::HTTP::post:
/accounts/ClientLogin
D, [2007-10-08T17:01:52.394047 #24929] DEBUG -- : request-header:
accept-language => en-us,en;q0.5
D, [2007-10-08T17:01:52.394214 #24929] DEBUG -- : request-header:
connection => keep-alive
D, [2007-10-08T17:01:52.394305 #24929] DEBUG -- : request-header: accept => */*
D, [2007-10-08T17:01:52.394394 #24929] DEBUG -- : request-header:
accept-encoding => gzip,identity
D, [2007-10-08T17:01:52.394482 #24929] DEBUG -- : request-header:
user-agent => WWW-Mechanize/0.6.10
(http://rubyforge.org/projects/mechanize/)
D, [2007-10-08T17:01:52.394570 #24929] DEBUG -- : request-header:
content-type => application/x-www-form-urlencoded
D, [2007-10-08T17:01:52.394656 #24929] DEBUG -- : request-header:
accept-charset => ISO-8859-1,utf-8;q=0.7,*;q=0.7
D, [2007-10-08T17:01:52.394742 #24929] DEBUG -- : request-header:
content-length => 104
D, [2007-10-08T17:01:52.394828 #24929] DEBUG -- : request-header:
keep-alive => 300
D, [2007-10-08T17:01:52.613138 #24929] DEBUG -- : Read 563 bytes
D, [2007-10-08T17:01:52.613358 #24929] DEBUG -- : response-header:
cache-control => no-cache
D, [2007-10-08T17:01:52.613483 #24929] DEBUG -- : response-header:
date => Mon, 08 Oct 2007 15:00:17 GMT
D, [2007-10-08T17:01:52.613603 #24929] DEBUG -- : response-header:
content-type => text/plain
D, [2007-10-08T17:01:52.613721 #24929] DEBUG -- : response-header:
server => GFE/1.3
D, [2007-10-08T17:01:52.613837 #24929] DEBUG -- : response-header:
content-length => 563
D, [2007-10-08T17:01:52.613969 #24929] DEBUG -- : response-header:
pragma => no-cache
I, [2007-10-08T17:01:52.614363 #24929] INFO -- : status: 200
I, [2007-10-08T17:13:15.370472 #24929] INFO -- : Net::HTTP::Get:
/feeds/spreadsheets/private/full
D, [2007-10-08T17:13:15.482357 #24929] DEBUG -- : request-header:
accept-language => en-us,en;q0.5
D, [2007-10-08T17:13:15.482481 #24929] DEBUG -- : request-header:
connection => keep-alive
D, [2007-10-08T17:13:15.482534 #24929] DEBUG -- : request-header: accept => */*
D, [2007-10-08T17:13:15.482587 #24929] DEBUG -- : request-header:
accept-encoding => gzip,identity
D, [2007-10-08T17:13:15.482639 #24929] DEBUG -- : request-header:
user-agent => WWW-Mechanize/0.6.10
(http://rubyforge.org/projects/mechanize/)
D, [2007-10-08T17:13:15.482692 #24929] DEBUG -- : request-header:
referer => https://www.google.com/accounts/ClientLogin
D, [2007-10-08T17:13:15.482745 #24929] DEBUG -- : request-header:
accept-charset => ISO-8859-1,utf-8;q=0.7,*;q=0.7
D, [2007-10-08T17:13:15.482796 #24929] DEBUG -- : request-header:
keep-alive => 300
D, [2007-10-08T17:13:15.652098 #24929] DEBUG -- : Read 141 bytes
D, [2007-10-08T17:13:15.652313 #24929] DEBUG -- : response-header:
cache-control => no-cache, no-store, max-age=0, must-revalidate
D, [2007-10-08T17:13:15.652438 #24929] DEBUG -- : response-header:
expires => Fri, 01 Jan 1990 00:00:00 GMT
D, [2007-10-08T17:13:15.652604 #24929] DEBUG -- : response-header:
date => Mon, 08 Oct 2007 15:11:40 GMT
D, [2007-10-08T17:13:15.652725 #24929] DEBUG -- : response-header:
content-type => text/html; charset=UTF-8
D, [2007-10-08T17:13:15.652859 #24929] DEBUG -- : response-header:
www-authenticate => No credentials were included in your request.
D, [2007-10-08T17:13:15.652977 #24929] DEBUG -- : response-header:
server => GFE/1.3
D, [2007-10-08T17:13:15.653094 #24929] DEBUG -- : response-header:
set-cookie => S=trix=Xc7oom0LVJ8; Path=/
D, [2007-10-08T17:13:15.653211 #24929] DEBUG -- : response-header:
content-length => 141
D, [2007-10-08T17:13:15.653326 #24929] DEBUG -- : response-header:
pragma => no-cache
D, [2007-10-08T17:13:15.659559 #24929] DEBUG -- : saved cookie:
S=trix=Xc7oom0LVJ8
I, [2007-10-08T17:13:15.659765 #24929] INFO -- : status: 404

So, it seems that the credentials are not correctly set. Anybody knows
how to do this?

Thanks,

Jesus.
 
R

Richard Conroy

Hi,

I'm trying to use Mechanize to retrieve spreadsheets from Google Docs
using the Google API. So far I got the ClientLogin to work, but I
can't seem to be able to use the Auth token they give me back. This is
what I have:

From my experience Mechanize is a bit low level, especially in this area.
You are expected to write the code yourself to handle, redirects,
authentication,
SSL etc.

You might want to try open-uri or Hpricot as well. But I think there is no
right-sized solution - you will still need to do a bit of code writing your=
self.

BTW - you should check out the new RESTful Web Services book (O'Reilly).
In addition to just being an excellent book, it has a lot of practical advi=
ce
on HTTP clients and directly manipulating HTTP headers, which is I
suspect what you will ultimately need to do.
 
J

Jesús Gabriel y Galán

the Gdata gem works with the google Gdata API , I'm currently implementing
the gcalendar API for that gem , but luckely for you, I think the
0.4version has some spreadsheet implementation.

Hi, yes, I had checked the Gdata gem, but saw it was pre alpha. In any
case I installed it and checked the spreadsheet class but I just saw a
couple of methods to demo how to evaluate a cell, and not the complete
functionality of the Google API. Am I right?

I'm thinking of stop using Mechanize for this and use just net/http
for the simple post and get I need to do, so I can customize the
headers myself, as I've seen you do in the gdata classes.

Thanks,

Jesus.
 
M

mortee

Richard said:
From my experience Mechanize is a bit low level, especially in this area.
You are expected to write the code yourself to handle, redirects,
authentication,
SSL etc.

You might want to try open-uri or Hpricot as well. But I think there is no
right-sized solution - you will still need to do a bit of code writing yourself.

Just for the record, last time I checked, Mechanize *did* use Hpricot to
parse pages. AFAIK, you can even provide Mechainze with your own page
parser class before fetching the stuff.

mortee
 
J

Jesús Gabriel y Galán

Just for the record, last time I checked, Mechanize *did* use Hpricot to
parse pages. AFAIK, you can even provide Mechainze with your own page
parser class before fetching the stuff.

Yes, Mechanize parses pages with Hpricot, and also allows you to use
an Hpricot-like API on them for search:

agent.get('http://rubyforge.org').search("//strong")

The problem I was having with it is the fact that I didn't know how to
specify arbitrary headers for my request, or make the Google
authentication stuff to work.

Working directly with net/https worked for me (code "inspired" by the
Gdata gem :):

require 'net/http'
require 'net/https'
require 'uri'
require 'hpricot'

module Net
class HTTPS < HTTP
def initialize(address, port = nil)
super(address, port)
self.use_ssl = true
end
end
end


h = {"Email" => "(e-mail address removed)",
"Passwd" => "xxxxx",
"service" => "wise",
"source" => "personal-test-1.0",
"accountType" => "GOOGLE"
}

url = URI.parse('https://www.google.com/accounts/ClientLogin')
response = Net::HTTPS.post_form(url, h)
auth = ""
response.body.each do |line|
if line =~ /Auth=/
auth = line.split("=")[1].chomp
end
end

headers = {"Authorization" => "GoogleLogin auth=#{auth}"}
http = Net::HTTP.new("spreadsheets.google.com", 80)
response, data = http.get("/feeds/spreadsheets/private/full", headers)
document = Hpricot(data)
document.search("//entry").each do |entry|
p entry
end

This gives you a document with the list of private spreadsheets of the
user. Then you can use Hpricot or whatever you want to parse the atom
feed.

Jesus.
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

Forum statistics

Threads
473,744
Messages
2,569,479
Members
44,899
Latest member
RodneyMcAu

Latest Threads

Top