Inexplicable Urllib2 problem between virtualenv's when POSTing to aTomcat server


Timmy O'Mahony

Hey, I have a question on Stackoverflow at the moment that I thought I would put up here as it might get some more eyes (It has a bounty so feel free to answer there if you have a SO account!)


I have some test code (as a part of a webapp) that uses urllib2 to perform an operation I would usually perform via a browser:

- Log in to a remote website (this works)
- Move to another page (this works)
- Perform a POST by filling in a form (read on ...!)

I've created 4 separate, clean virtualenvs (with --no-site-packages) on 3 different machines, all with different versions of python but the exact samepackages (via pip requirements file), and the code only works on the two virtualenvs on my local development machine(2.6.1 and 2.7.2) - it won't workon either of my production VPSs :(

In the failing cases, I can log in successfully, move to the correct page but when I submit the form, the remote server replies telling me that there has been an error - it's an application server error page ('we couldn't complete your request') and not a webserver error.

- because I can successfully log in and maneuver to a second page, this doesn't seem to be a session or a cookie problem - it's particular to the final POST

- because I can perform the operation on a particular machine with the EXACT same headers and data, this doesn't seem to be a problem with what I am requesting/posting

- because I am trying the code on two separate VPS rented from different companies, this doesn't seem to be a problem with the VPS physical environment

- because the code works on 2 different python versions, I can't imagine itbeing an incompabilty problem
I'm completely lost at this stage as to why this wouldn't work. I've even 'turned-it-off-and-turn-it-on-again' because I just can't see what the problem could be.

I've tried everything I can think of to get this working. I've been throughall of the headers of the requests & responses and they are all the same between machines.

The server I am trying to contact is a Tomcat server. It gives me a cookie called JSESSIONID and it also requires an apache.struct.token in the POST data which I am succesfully extracting with BeautifulSoup/lxml. Again, this works on 2 machines, so I don't think it's an ommision on my behalf, I think it's a compatibility or an encoding error.

Any ideas welcome!



Devin Jeanpierre

It might help to give more information about the machines. In
particular, what versions of Python are on the production machines?




Timmy O'Mahony

Good point.

The two machines that the code works with are running python 2.6.1 and 2.7.2 and are running on my Mac (Snow Leopard)

The two non-working machines are running python 2.6.6 and 2.7.1 and are on Debian 6 and Debian 5 respectively. They are VPSs managed by different providers.

All of the installs are on separate virtuaenvs.

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question