Problem with open-uri

[Jonathan Nichols]

Hello,

I'm trying to use open-uri on yahoo mail's login page. If I do the
following:

#!/usr/bin/env ruby

require 'open-uri'
require 'uri'

f = open("http://mail.yahoo.com")

--------

I get the following error:

/usr/lib/ruby/1.8/open-uri.rb:174:in `open_loop': redirection forbidden:
http://mail.yahoo.com/ ->
https://login.yahoo.com/config/login_verify2?&.src=ym (RuntimeError)
from /usr/lib/ruby/1.8/open-uri.rb:132:in `open_uri'
from /usr/lib/ruby/1.8/open-uri.rb:528:in `open'
from /usr/lib/ruby/1.8/open-uri.rb:30:in `open'
from ./readForms.rb:11

-------

I've tried to use mechanize instead, but I can't seem to figure out
how to emulate the base_uri functionality in open-uri. Any ideas?

Regards,

jon

 

[Todd Benson]

Hello,

I'm trying to use open-uri on yahoo mail's login page. If I do the
following:

#!/usr/bin/env ruby

require 'open-uri'
require 'uri'

f = open("http://mail.yahoo.com")

--------

I get the following error:

/usr/lib/ruby/1.8/open-uri.rb:174:in `open_loop': redirection forbidden:
http://mail.yahoo.com/ ->
https://login.yahoo.com/config/login_verify2?&.src=ym (RuntimeError)
from /usr/lib/ruby/1.8/open-uri.rb:132:in `open_uri'
from /usr/lib/ruby/1.8/open-uri.rb:528:in `open'
from /usr/lib/ruby/1.8/open-uri.rb:30:in `open'
from ./readForms.rb:11

-------

I've tried to use mechanize instead, but I can't seem to figure out
how to emulate the base_uri functionality in open-uri. Any ideas?

Regards,

jon

You're being redirected. Try the redirect url (and you can leave the
?&.src=ym off depending on what you want to do)

require 'open-uri'
document = open( "https://login.yahoo.com/config/login_verify2" ).read

 

[Jonathan Nichols]

Hi Todd,

Thanks for the quick response. Unfortunately, I need to have the
ability to take a simple uri and follow the redirections. open-uri
works for some uris, but not for yahoo for some reason. If you do
open("https://mail.google.com"), it will follow the redirects and
successfully download the html at the redirected uri, which is
https://www.google.com/accounts/ServiceLogin. this looks like a bug in
open-uri to me, as mechanize (and firefox) is able to follow the
redirects for yahoo.

I can't find a way to report bugs for open-uri. Anybody know how to
do this?

Regards,

jon

 

[Jonathan Nichols]

I think I figured this out. The following is the code that generates
the exception:

----
def OpenURI.redirectable?(uri1, uri2) # :nodoc:
# This test is intended to forbid a redirection from http://... to
# file:///etc/passwd.
# However this is ad hoc. It should be extensible/configurable.
uri1.scheme.downcase == uri2.scheme.downcase ||
(/\A(?:http|ftp)\z/i =~ uri1.scheme && /\A(?:http|ftp)\z/i =~
uri2.scheme)
end
----

This code is telling us that if there's a redirection, it will only work
if the protocols are the same (http -> http, or https -> https), or if
both the first and the second uris are either http or ftp. This logic
prohibits a redirect from http to https, which is what the yahoo mail
uri does. Changing the last line to

(/\A(?:http|ftp|https)\z/i =~ uri1.scheme &&
/\A(?:http|ftp|https)\z/i =~ uri2.scheme)

will do the trick. Seems like this fix should get incorporated into
open-uri. Anyone out there on the open-uri team listening?

-jon