Making urllib2's POST 302 handle same as Perl LWP's behaviour

Discussion in 'Python' started by Karra, Dec 29, 2010.

  1. Karra

    Karra Guest

    I am doing a POST to a webserver and get a 302 Found response
    (redirect). urllib2's default behaviour is to do a GET on the new url
    from the Location: URI in the 302 response.

    This is different from what I have found with LWP::UserAgent-
    >request() in perl. After much searching I understand there is a view

    that automatic redirection for a 302 in response to a POST is not in
    conformance to the relevant RFCs. Therefore, I believe urllib2's
    behaviour appears to be non-conformant (as, I believe are many
    browsers).

    Now, regardless of what is the "correct" approach to handling the 302,
    there is some information in the returned html of the 302 which I am
    losing because of the subsequent GET. I tried to raise a HTTPError
    from redirect_request() but that just kills the connection with the
    server. I tried returning None, same result.

    Can someone point me to how I can get the default LWP:UserAgent
    behaviour of handling this scenario using urllib2?
     
    Karra, Dec 29, 2010
    #1
    1. Advertising

  2. Karra

    Karra Guest

    On Dec 29, 3:57 pm, Karra <> wrote:

    > Can someone point me to how I can get the default LWP:UserAgent
    > behaviour of handling this scenario using urllib2?


    Out of frustration, I decided to give 'mechanize' a try. It came as an
    awesome surprise that mechanize implements the exact api of urllib2 -
    meaning all I had to do was a query-replace of urllib2 to mechanize,
    and lo-and-behold I got what I wanted!

    Which is to say, the behaviour of mechanize matches Perl's
    LWP::UserAgent as afar as handling of POST redirects go.
     
    Karra, Dec 29, 2010
    #2
    1. Advertising

  3. I have tried:

    In httpd.conf:

    Redirect /one/ http://localhost/two/
    Redirect /two/ http://localhost/three/
    redirect /three/ http://www.google.com/

    Then I made a POST request with LWP::UserAgent to /one:

    use LWP::UserAgent;
    print LWP::UserAgent->new->post('http://localhost/one/')->as_string;

    And the result was:
    HTTP/1.1 302 Found
    Connection: close
    Date: Wed, 29 Dec 2010 13:58:37 GMT
    Location: http://localhost/two/
    Server: Apache/2.2.15 (Win32) mod_perl/2.0.4-dev Perl/v5.10.1
    ....

    Exactly as you described.

    Then I made a POST request with WWW::Mechanize:

    use WWW::Mechanize;
    print WWW::Mechanize->new->post('http://localhost/one/')->as_string;

    And the result was:
    HTTP/1.1 200 OK
    Cache-Control: private, max-age=0
    Connection: close
    Date: Wed, 29 Dec 2010 14:00:44 GMT
    Server: gws

    So it seems that LWP::UserAgent works similarly with mechanize and WWW::Mechanize similarly to urllib. Nice. :)

    Octavian

    ----- Original Message -----
    From: "Karra" <>
    Newsgroups: comp.lang.python
    To: <>
    Sent: Wednesday, December 29, 2010 2:35 PM
    Subject: Re: Making urllib2's POST 302 handle same as Perl LWP's behaviour


    On Dec 29, 3:57 pm, Karra <> wrote:

    > Can someone point me to how I can get the default LWP:UserAgent
    > behaviour of handling this scenario using urllib2?


    Out of frustration, I decided to give 'mechanize' a try. It came as an
    awesome surprise that mechanize implements the exact api of urllib2 -
    meaning all I had to do was a query-replace of urllib2 to mechanize,
    and lo-and-behold I got what I wanted!

    Which is to say, the behaviour of mechanize matches Perl's
    LWP::UserAgent as afar as handling of POST redirects go.
    --
    http://mail.python.org/mailman/listinfo/python-list
     
    Octavian Rasnita, Dec 29, 2010
    #3
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. =?ISO-8859-1?Q?Eino_M=E4kitalo?=

    urllib2 and Set-Cookie with "302 Moved temporarily"

    =?ISO-8859-1?Q?Eino_M=E4kitalo?=, Dec 13, 2004, in forum: Python
    Replies:
    2
    Views:
    828
    =?ISO-8859-1?Q?Eino_M=E4kitalo?=
    Dec 13, 2004
  2. Laszlo Nagy

    urllib2 and HTTP 302

    Laszlo Nagy, May 26, 2006, in forum: Python
    Replies:
    0
    Views:
    360
    Laszlo Nagy
    May 26, 2006
  3. Antoni Villalonga

    urllib2: handle an error (302)

    Antoni Villalonga, Sep 11, 2007, in forum: Python
    Replies:
    2
    Views:
    986
    Antoni Villalonga
    Sep 11, 2007
  4. John Bokma

    LWP gives 302 Found after update?

    John Bokma, Aug 16, 2005, in forum: Perl Misc
    Replies:
    5
    Views:
    364
    J. Gleixner
    Aug 16, 2005
  5. IanW

    LWP and 302 redirects

    IanW, Oct 4, 2006, in forum: Perl Misc
    Replies:
    2
    Views:
    368
    J. Gleixner
    Oct 5, 2006
Loading...

Share This Page