Accessing Web site contents from a Python script (with use of Cookie)

[Aki Niimura]

Hello everyone,

I have been trying to write a Python script to access a Web site.

I'm currently having a problem completing my script because the contents
I would like to access in the Web site is a dynamic content and it is
generated based on the Cookie contents which are set in another page.

url-a (setting Cookies)
url-b (dynamic content based on the Cookie value)

I'm using httplib.HTTPConnection() to first access the url-a to get other info
(and Cookies in header). Then my script accesses the url-b but it doesn't
work as no Cookie info is returned when the url-b is being accessed.

The Cookie values vary time to time such that the script must obtain them
from the url-a. Also, no expiration is set in the Cookies such that they are
supposed to live only in browser not in Cookie files.

Is there any way to handle the situation mentioned above?

Any comments, any hints are highly appreciated.

Best regards,
Aki Niimura

 

[Ian Bicking]

I'm currently having a problem completing my script because the contents
I would like to access in the Web site is a dynamic content and it is
generated based on the Cookie contents which are set in another page.

You should use ClientCookie:
http://wwwsearch.sourceforge.net/ClientCookie/

Ian

 

[Aki Niimura]

You should use ClientCookie:
http://wwwsearch.sourceforge.net/ClientCookie/

Thank you for getting me know such interesting Python module is available.

I made my script working using HTTPResponse.getheader('Set-Cookie') and
HTTPResponse.putheader('Cookie', cookie). Using ClientCookie would make
such process hidden. I should use it next time.

Thanks!
Aki Niimura