Inexplicable Urllib2 problem between virtualenv's when POSTing to aTomcat server

Discussion in 'Python' started by Timmy O'Mahony, Sep 25, 2011.

  1. Hey, I have a question on Stackoverflow at the moment that I thought I would put up here as it might get some more eyes (It has a bounty so feel free to answer there if you have a SO account!)

    ----

    I have some test code (as a part of a webapp) that uses urllib2 to perform an operation I would usually perform via a browser:

    - Log in to a remote website (this works)
    - Move to another page (this works)
    - Perform a POST by filling in a form (read on ...!)

    I've created 4 separate, clean virtualenvs (with --no-site-packages) on 3 different machines, all with different versions of python but the exact samepackages (via pip requirements file), and the code only works on the two virtualenvs on my local development machine(2.6.1 and 2.7.2) - it won't workon either of my production VPSs :(

    In the failing cases, I can log in successfully, move to the correct page but when I submit the form, the remote server replies telling me that there has been an error - it's an application server error page ('we couldn't complete your request') and not a webserver error.

    - because I can successfully log in and maneuver to a second page, this doesn't seem to be a session or a cookie problem - it's particular to the final POST

    - because I can perform the operation on a particular machine with the EXACT same headers and data, this doesn't seem to be a problem with what I am requesting/posting

    - because I am trying the code on two separate VPS rented from different companies, this doesn't seem to be a problem with the VPS physical environment

    - because the code works on 2 different python versions, I can't imagine itbeing an incompabilty problem
    I'm completely lost at this stage as to why this wouldn't work. I've even 'turned-it-off-and-turn-it-on-again' because I just can't see what the problem could be.

    I've tried everything I can think of to get this working. I've been throughall of the headers of the requests & responses and they are all the same between machines.

    The server I am trying to contact is a Tomcat server. It gives me a cookie called JSESSIONID and it also requires an apache.struct.token in the POST data which I am succesfully extracting with BeautifulSoup/lxml. Again, this works on 2 machines, so I don't think it's an ommision on my behalf, I think it's a compatibility or an encoding error.

    Any ideas welcome!
     
    Timmy O'Mahony, Sep 25, 2011
    #1
    1. Advertising

  2. Re: Inexplicable Urllib2 problem between virtualenv's when POSTing toa Tomcat server

    It might help to give more information about the machines. In
    particular, what versions of Python are on the production machines?

    Devin

    On Sun, Sep 25, 2011 at 8:31 AM, Timmy O'Mahony
    <> wrote:
    > Hey, I have a question on Stackoverflow at the moment that I thought I would put up here as it might get some more eyes (It has a bounty so feel free to answer there if you have a SO account!)
    >
    > ----
    >
    > I have some test code (as a part of a webapp) that uses urllib2 to perform an operation I would usually perform via a browser:
    >
    > - Log in to a remote website (this works)
    > - Move to another page (this works)
    > - Perform a POST by filling in a form (read on ...!)
    >
    > I've created 4 separate, clean virtualenvs (with --no-site-packages) on 3different machines, all with different versions of python but the exact same packages (via pip requirements file), and the code only works on the twovirtualenvs on my local development machine(2.6.1 and 2.7.2) - it won't work on either of my production VPSs :(
    >
    > In the failing cases, I can log in successfully, move to the correct pagebut when I submit the form, the remote server replies telling me that there has been an error - it's an application server error page ('we couldn't complete your request') and not a webserver error.
    >
    > - because I can successfully log in and maneuver to a second page, this doesn't seem to be a session or a cookie problem - it's particular to the final POST
    >
    > - because I can perform the operation on a particular machine with the EXACT same headers and data, this doesn't seem to be a problem with what I amrequesting/posting
    >
    > - because I am trying the code on two separate VPS rented from different companies, this doesn't seem to be a problem with the VPS physical environment
    >
    > - because the code works on 2 different python versions, I can't imagine it being an incompabilty problem
    > I'm completely lost at this stage as to why this wouldn't work. I've even'turned-it-off-and-turn-it-on-again' because I just can't see what the problem could be.
    >
    > I've tried everything I can think of to get this working. I've been through all of the headers of the requests & responses and they are all the samebetween machines.
    >
    > The server I am trying to contact is a Tomcat server. It gives me a cookie called JSESSIONID and it also requires an apache.struct.token in the POSTdata which I am succesfully extracting with BeautifulSoup/lxml. Again, this works on 2 machines, so I don't think it's an ommision on my behalf, I think it's a compatibility or an encoding error.
    >
    > Any ideas welcome!
    >
    >
    >
    > --
    > http://mail.python.org/mailman/listinfo/python-list
    >
     
    Devin Jeanpierre, Sep 25, 2011
    #2
    1. Advertising

  3. Re: Inexplicable Urllib2 problem between virtualenv's when POSTing toa Tomcat server

    Good point.

    The two machines that the code works with are running python 2.6.1 and 2.7.2 and are running on my Mac (Snow Leopard)

    The two non-working machines are running python 2.6.6 and 2.7.1 and are on Debian 6 and Debian 5 respectively. They are VPSs managed by different providers.

    All of the installs are on separate virtuaenvs.
     
    Timmy O'Mahony, Sep 25, 2011
    #3
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Wolfgang
    Replies:
    9
    Views:
    430
    S Manohar
    Feb 2, 2004
  2. Dave
    Replies:
    13
    Views:
    749
    Cy Edmunds
    Jan 1, 2004
  3. Josef Cihal
    Replies:
    0
    Views:
    765
    Josef Cihal
    Sep 5, 2005
  4. Fabiano Sidler

    Inexplicable behaviour of <type 'function'>

    Fabiano Sidler, Apr 23, 2006, in forum: Python
    Replies:
    1
    Views:
    286
    Leif K-Brooks
    Apr 23, 2006
  5. Gelonida N
    Replies:
    4
    Views:
    982
    Gelonida N
    Aug 11, 2011
Loading...

Share This Page