[2.5] Script to POST to web page with cookies?

[Gilles Ganault]

Hello

I need to write a script to automate fetching data from a web site:
1. using the POST method, log on, with login/password saved as cookies
2. download page and extract relevent information using regexes
3. log off
4. wait for a random number of minutes, and GOTO 1

I'm a bit confused with how to get POST and cookies in the same
script:
- urllib vs urllib2 vs httplib?
- "ClientCookie.urlopen("www") behaves identically to urllib2.urlopen,
except that it deals with cookies automatically"

Has anyone some working code that I could borrow (er... steal) to do
this?

Thank you.

 

[Miki]

Hello Gille,

I need to write a script to automate fetching data from a web site:
1. using the POST method, log on, with login/password saved as cookies
2. download page and extract relevent information using regexes
3. log off
4. wait for a random number of minutes, and GOTO 1
...

http://wwwsearch.sourceforge.net/mechanize/

HTH,

 

[7stud]

Hello

I need to write a script to automate fetching data from a web site:
1. using the POST method, log on, with login/password saved as cookies
2. download page and extract relevent information using regexes
3. log off
4. wait for a random number of minutes, and GOTO 1

I'm a bit confused with how to get POST and cookies in the same
script:
- urllib vs urllib2 vs httplib?
- "ClientCookie.urlopen("www") behaves identically to urllib2.urlopen,
except that it deals with cookies automatically"

Has anyone some working code that I could borrow (er... steal) to do
this?

Thank you.

Also, see here:

http://www.voidspace.org.uk/python/articles/cookielib.shtml