Question about Mechanize and Google API

Discussion in 'Ruby' started by Jesús Gabriel y Galán, Oct 8, 2007.

  1. Hi,

    I'm trying to use Mechanize to retrieve spreadsheets from Google Docs
    using the Google API. So far I got the ClientLogin to work, but I
    can't seem to be able to use the Auth token they give me back. This is
    what I have:

    require 'logger'
    require 'mechanize'

    agent = WWW::Mechanize.new {|a| a.log = Logger.new("test.log")}
    response = agent.post("https://www.google.com/accounts/ClientLogin",
    "Email" => "", "Passwd" => "xxxxx", "service" => "wise",
    "source" => "personal-test-1.0", "accountType" => "GOOGLE")
    h = {}
    response.body.each {|line| a = line.split("="); h[a[0]] = a[1]}
    auth = h["Auth"]
    agent.basic_auth("GoogleLogin", auth)
    # this didn't work either
    #agent.auth("GoogleLogin", auth)
    page = agent.get("http://spreadsheets.google.com/feeds/spreadsheets/private/full")

    The problem is, according to google I have to:
    "After a successful authentication request, use the Auth value to
    create an Authorization header for each request:

    Authorization: GoogleLogin auth=yourAuthValue"

    I've checked and the ClientLogin is working and returning me an Auth
    token, but I can't get Mechanize to use it. In the Mechanize logs I
    see this:
    I, [2007-10-08T17:01:52.211778 #24929] INFO -- : Net::HTTP::post:
    /accounts/ClientLogin
    D, [2007-10-08T17:01:52.394047 #24929] DEBUG -- : request-header:
    accept-language => en-us,en;q0.5
    D, [2007-10-08T17:01:52.394214 #24929] DEBUG -- : request-header:
    connection => keep-alive
    D, [2007-10-08T17:01:52.394305 #24929] DEBUG -- : request-header: accept => */*
    D, [2007-10-08T17:01:52.394394 #24929] DEBUG -- : request-header:
    accept-encoding => gzip,identity
    D, [2007-10-08T17:01:52.394482 #24929] DEBUG -- : request-header:
    user-agent => WWW-Mechanize/0.6.10
    (http://rubyforge.org/projects/mechanize/)
    D, [2007-10-08T17:01:52.394570 #24929] DEBUG -- : request-header:
    content-type => application/x-www-form-urlencoded
    D, [2007-10-08T17:01:52.394656 #24929] DEBUG -- : request-header:
    accept-charset => ISO-8859-1,utf-8;q=0.7,*;q=0.7
    D, [2007-10-08T17:01:52.394742 #24929] DEBUG -- : request-header:
    content-length => 104
    D, [2007-10-08T17:01:52.394828 #24929] DEBUG -- : request-header:
    keep-alive => 300
    D, [2007-10-08T17:01:52.613138 #24929] DEBUG -- : Read 563 bytes
    D, [2007-10-08T17:01:52.613358 #24929] DEBUG -- : response-header:
    cache-control => no-cache
    D, [2007-10-08T17:01:52.613483 #24929] DEBUG -- : response-header:
    date => Mon, 08 Oct 2007 15:00:17 GMT
    D, [2007-10-08T17:01:52.613603 #24929] DEBUG -- : response-header:
    content-type => text/plain
    D, [2007-10-08T17:01:52.613721 #24929] DEBUG -- : response-header:
    server => GFE/1.3
    D, [2007-10-08T17:01:52.613837 #24929] DEBUG -- : response-header:
    content-length => 563
    D, [2007-10-08T17:01:52.613969 #24929] DEBUG -- : response-header:
    pragma => no-cache
    I, [2007-10-08T17:01:52.614363 #24929] INFO -- : status: 200
    I, [2007-10-08T17:13:15.370472 #24929] INFO -- : Net::HTTP::Get:
    /feeds/spreadsheets/private/full
    D, [2007-10-08T17:13:15.482357 #24929] DEBUG -- : request-header:
    accept-language => en-us,en;q0.5
    D, [2007-10-08T17:13:15.482481 #24929] DEBUG -- : request-header:
    connection => keep-alive
    D, [2007-10-08T17:13:15.482534 #24929] DEBUG -- : request-header: accept => */*
    D, [2007-10-08T17:13:15.482587 #24929] DEBUG -- : request-header:
    accept-encoding => gzip,identity
    D, [2007-10-08T17:13:15.482639 #24929] DEBUG -- : request-header:
    user-agent => WWW-Mechanize/0.6.10
    (http://rubyforge.org/projects/mechanize/)
    D, [2007-10-08T17:13:15.482692 #24929] DEBUG -- : request-header:
    referer => https://www.google.com/accounts/ClientLogin
    D, [2007-10-08T17:13:15.482745 #24929] DEBUG -- : request-header:
    accept-charset => ISO-8859-1,utf-8;q=0.7,*;q=0.7
    D, [2007-10-08T17:13:15.482796 #24929] DEBUG -- : request-header:
    keep-alive => 300
    D, [2007-10-08T17:13:15.652098 #24929] DEBUG -- : Read 141 bytes
    D, [2007-10-08T17:13:15.652313 #24929] DEBUG -- : response-header:
    cache-control => no-cache, no-store, max-age=0, must-revalidate
    D, [2007-10-08T17:13:15.652438 #24929] DEBUG -- : response-header:
    expires => Fri, 01 Jan 1990 00:00:00 GMT
    D, [2007-10-08T17:13:15.652604 #24929] DEBUG -- : response-header:
    date => Mon, 08 Oct 2007 15:11:40 GMT
    D, [2007-10-08T17:13:15.652725 #24929] DEBUG -- : response-header:
    content-type => text/html; charset=UTF-8
    D, [2007-10-08T17:13:15.652859 #24929] DEBUG -- : response-header:
    www-authenticate => No credentials were included in your request.
    D, [2007-10-08T17:13:15.652977 #24929] DEBUG -- : response-header:
    server => GFE/1.3
    D, [2007-10-08T17:13:15.653094 #24929] DEBUG -- : response-header:
    set-cookie => S=trix=Xc7oom0LVJ8; Path=/
    D, [2007-10-08T17:13:15.653211 #24929] DEBUG -- : response-header:
    content-length => 141
    D, [2007-10-08T17:13:15.653326 #24929] DEBUG -- : response-header:
    pragma => no-cache
    D, [2007-10-08T17:13:15.659559 #24929] DEBUG -- : saved cookie:
    S=trix=Xc7oom0LVJ8
    I, [2007-10-08T17:13:15.659765 #24929] INFO -- : status: 404

    So, it seems that the credentials are not correctly set. Anybody knows
    how to do this?

    Thanks,

    Jesus.
     
    Jesús Gabriel y Galán, Oct 8, 2007
    #1
    1. Advertising

  2. On 10/8/07, Jes=FAs Gabriel y Gal=E1n <> wrote:
    > Hi,
    >
    > I'm trying to use Mechanize to retrieve spreadsheets from Google Docs
    > using the Google API. So far I got the ClientLogin to work, but I
    > can't seem to be able to use the Auth token they give me back. This is
    > what I have:


    From my experience Mechanize is a bit low level, especially in this area.
    You are expected to write the code yourself to handle, redirects,
    authentication,
    SSL etc.

    You might want to try open-uri or Hpricot as well. But I think there is no
    right-sized solution - you will still need to do a bit of code writing your=
    self.

    BTW - you should check out the new RESTful Web Services book (O'Reilly).
    In addition to just being an excellent book, it has a lot of practical advi=
    ce
    on HTTP clients and directly manipulating HTTP headers, which is I
    suspect what you will ultimately need to do.
     
    Richard Conroy, Oct 8, 2007
    #2
    1. Advertising

  3. On 10/8/07, Matias Pablo Brutti <> wrote:

    > the Gdata gem works with the google Gdata API , I'm currently implementing
    > the gcalendar API for that gem , but luckely for you, I think the
    > 0.4version has some spreadsheet implementation.


    Hi, yes, I had checked the Gdata gem, but saw it was pre alpha. In any
    case I installed it and checked the spreadsheet class but I just saw a
    couple of methods to demo how to evaluate a cell, and not the complete
    functionality of the Google API. Am I right?

    I'm thinking of stop using Mechanize for this and use just net/http
    for the simple post and get I need to do, so I can customize the
    headers myself, as I've seen you do in the gdata classes.

    Thanks,

    Jesus.
     
    Jesús Gabriel y Galán, Oct 8, 2007
    #3
  4. Jesús Gabriel y Galán

    mortee Guest

    Richard Conroy wrote:
    > From my experience Mechanize is a bit low level, especially in this area.
    > You are expected to write the code yourself to handle, redirects,
    > authentication,
    > SSL etc.
    >
    > You might want to try open-uri or Hpricot as well. But I think there is no
    > right-sized solution - you will still need to do a bit of code writing yourself.


    Just for the record, last time I checked, Mechanize *did* use Hpricot to
    parse pages. AFAIK, you can even provide Mechainze with your own page
    parser class before fetching the stuff.

    mortee
     
    mortee, Oct 8, 2007
    #4
  5. On 10/9/07, mortee <> wrote:
    > Richard Conroy wrote:
    > > From my experience Mechanize is a bit low level, especially in this area.
    > > You are expected to write the code yourself to handle, redirects,
    > > authentication,
    > > SSL etc.
    > >
    > > You might want to try open-uri or Hpricot as well. But I think there is no
    > > right-sized solution - you will still need to do a bit of code writing yourself.

    >
    > Just for the record, last time I checked, Mechanize *did* use Hpricot to
    > parse pages. AFAIK, you can even provide Mechainze with your own page
    > parser class before fetching the stuff.


    Yes, Mechanize parses pages with Hpricot, and also allows you to use
    an Hpricot-like API on them for search:

    agent.get('http://rubyforge.org').search("//strong")

    The problem I was having with it is the fact that I didn't know how to
    specify arbitrary headers for my request, or make the Google
    authentication stuff to work.

    Working directly with net/https worked for me (code "inspired" by the
    Gdata gem :):

    require 'net/http'
    require 'net/https'
    require 'uri'
    require 'hpricot'

    module Net
    class HTTPS < HTTP
    def initialize(address, port = nil)
    super(address, port)
    self.use_ssl = true
    end
    end
    end


    h = {"Email" => "",
    "Passwd" => "xxxxx",
    "service" => "wise",
    "source" => "personal-test-1.0",
    "accountType" => "GOOGLE"
    }

    url = URI.parse('https://www.google.com/accounts/ClientLogin')
    response = Net::HTTPS.post_form(url, h)
    auth = ""
    response.body.each do |line|
    if line =~ /Auth=/
    auth = line.split("=")[1].chomp
    end
    end

    headers = {"Authorization" => "GoogleLogin auth=#{auth}"}
    http = Net::HTTP.new("spreadsheets.google.com", 80)
    response, data = http.get("/feeds/spreadsheets/private/full", headers)
    document = Hpricot(data)
    document.search("//entry").each do |entry|
    p entry
    end

    This gives you a document with the list of private spreadsheets of the
    user. Then you can use Hpricot or whatever you want to parse the atom
    feed.

    Jesus.
     
    Jesús Gabriel y Galán, Oct 9, 2007
    #5
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Replies:
    2
    Views:
    2,066
    Jeffrey Schwab
    Nov 28, 2005
  2. JSH
    Replies:
    0
    Views:
    414
  3. Ethan Herdrick
    Replies:
    1
    Views:
    123
    Ethan Herdrick
    May 11, 2006
  4. Kevin Lee
    Replies:
    1
    Views:
    234
    Peter Szinek
    Jun 11, 2007
  5. Replies:
    0
    Views:
    511
Loading...

Share This Page