Problem: open-uri blocking redirection from http to https

Discussion in 'Ruby' started by Xavier Del Castillo, Feb 18, 2011.

  1. Hello,

    I was working on a small script to verify the presence of an element
    through a list of URL, some of these URLs have a redirections from http to
    https, when the script crawled into them I got the following error:

    /usr/lib/ruby/1.9.1/open-uri.rb:216:in `open_loop': redirection forbidden:
    http://beta.carsdirect.com/auto-loans/finance-app ->
    https://beta.carsdirect.com/auto-loans/finance-app (RuntimeError)

    I understand that this is intentional, as per the comments on open-uri.rb

    # This test is intended to forbid a redirection from http://... to
    # file:///etc/passwd.
    # https to http redirect is also forbidden intentionally.
    # It avoids sending secure cookie or referer by non-secure HTTP
    protocol.
    # (RFC 2109 4.3.1, RFC 2965 3.3, RFC 2616 15.1.3)
    # However this is ad hoc. It should be extensible/configurable.

    This mentions that "https to http" redirects are forbidden intentionally,
    but redirections from "http to https" are also blocked.

    Is there a way to override this security check? currently I had to change the
    following line in the library to allow "http to https" re-directions:

    (/\A(?:http|ftp)\z/i =~ uri1.scheme && /\A(?:http|ftp)\z/i =~ uri2.scheme)

    to

    (/\A(?:http|ftp|https)\z/i =~ uri1.scheme && /\A(?:http|ftp|https)\z/i =~
    uri2.scheme)

    Thanks,
    Xavi
     
    Xavier Del Castillo, Feb 18, 2011
    #1
    1. Advertising

  2. Xavier Del Castillo

    Tanaka Akira Guest

    2011/2/18 Xavier Del Castillo <>:

    > # However this is ad hoc. It should be extensible/configurable.
    >
    > This mentions that "https to http" redirects are forbidden intentionally,
    > but redirections from "http to https" are also blocked.
    >
    > Is there a way to override this security check? currently I had to change
    > the
    > following line in the library to allow "http to https" re-directions:
    >
    > (/\A(?:http|ftp)\z/i =~ uri1.scheme && /\A(?:http|ftp)\z/i =~ uri2.scheme)


    Currently it is not configurable (as the comment says) except monkey patching.

    Maybe open-uri should have some hooks.
    --
    Tanaka Akira
     
    Tanaka Akira, Feb 28, 2011
    #2
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Simon Harris
    Replies:
    0
    Views:
    6,391
    Simon Harris
    May 10, 2005
  2. Randy Lawrence

    open-uri: problem handling https?

    Randy Lawrence, Jul 8, 2004, in forum: Ruby
    Replies:
    4
    Views:
    90
    Carl Youngblood
    Jul 10, 2004
  3. jotto
    Replies:
    4
    Views:
    402
    jotto
    Oct 2, 2006
  4. Brad Tilley

    open-uri and redirection

    Brad Tilley, Oct 29, 2006, in forum: Ruby
    Replies:
    1
    Views:
    274
    Aaron Patterson
    Oct 30, 2006
  5. Jay 99
    Replies:
    2
    Views:
    195
    Jay 99
    Apr 4, 2009
Loading...

Share This Page