send cookie on request with urllib2

[itay_k]

Hi,

I dont understand why this is so complicated, just to add one line of
cookie header on the GET request.

This is my unworking code:
import time
import Cookie
import cookielib, urllib2

c= cookielib.Cookie(1,"Name","Tom", 80,False, "itay", False, False,
"d:\\asddd",False, False,time.time()+1000,False,None,None,None)
cj = cookielib.CookieJar()
cj.set_cookie(c)
opener = urllib2.build_opener(urllib2.HTTPCookieProcessor(cj))
opener.open(r'http://itay/temp.html")

why this isn't working?
Itay is my computer name. it is an IIS server too.
the code is running, but i dont see any cookie that attached to the GET
request.

thanks alot!

 

[John J. Lee]

Hi,

I dont understand why this is so complicated, just to add one line of
cookie header on the GET request.

You haven't said what you're really trying to do.

http://www.catb.org/~esr/faqs/smart-questions.html#goal

This is my unworking code:
import time
import Cookie
import cookielib, urllib2

c= cookielib.Cookie(1,"Name","Tom", 80,False, "itay", False, False,
"d:\\asddd",False, False,time.time()+1000,False,None,None,None)

Constructing your own Cookie instances is rarely necessary or
sensible. If you do, I recommend getting the constructor arguments by
inspecting the Cookie object the server actually returns, rather than
using the values you *think* you know are correct.

cj = cookielib.CookieJar()
cj.set_cookie(c)
opener = urllib2.build_opener(urllib2.HTTPCookieProcessor(cj))
opener.open(r'http://itay/temp.html")

why this isn't working?

Please define "working".

If you want to handle cookies when opening URLs, just do this:

opener = urllib2.build_opener(urllib2.HTTPCookieProcessor())
opener.open("http://itay/temp.html")
# more .open() calls go here...


Of course, on the first HTTP request, cookie handling cannot have any
effect at all unless you've somehow loaded some old cookies first.
Your .open() may only involve a single HTTP request (though
server-side software may detect the absence of a cookie on the first
request, and do a page refresh or redirect so it can see its cookie
returned to it again; vanilla urllib2 handles redirects but not
refreshes; package ClientCookie handles the latter, amongst other
things).

Regardless, turning on cookielib and httplib debug output will likely
be helpful if you're stuck (if only to post the output to this
newsgroup):

import logging, urllib2, sys

hh = urllib2.HTTPHandler()
hsh = urllib2.HTTPSHandler()
hh.set_http_debuglevel(1)
hsh.set_http_debuglevel(1)
opener = urllib2.build_opener(hh, hsh, urllib2.HTTPCookieProcessor())
logger = logging.getLogger("cookielib")
logger.addHandler(logging.StreamHandler(sys.stdout))
logger.setLevel(logging.DEBUG)

response = opener.open("http://wwwsearch.sf.net/cgi-bin/cookietest.cgi")


John

 

[Kent Johnson]

Hi,

I dont understand why this is so complicated, just to add one line of
cookie header on the GET request.

This is my unworking code:
import time
import Cookie
import cookielib, urllib2

c= cookielib.Cookie(1,"Name","Tom", 80,False, "itay", False, False,
"d:\\asddd",False, False,time.time()+1000,False,None,None,None)

^^^^ path is the server path to which the cookie applies. Try '/'.

Kent

 

[John J. Lee]

^^^^ path is the server path to which the cookie applies. Try '/'.

"""
"No," scolded Yoda. "Do, or do not. There is no try."
"""

(there, a Star Wars quote -- I guess there's truly no hope of ever
erasing my geek status now!-)

Why guess? Why not log in (using Python) and see what the cookie
actually is? Once you've actually done that, there's nothing to stop
you storing it as a Cookie constructor call.

I know I certainly don't remember all the *truly horrendous* detail of
what *exactly* all those parameters mean The Cookie object, as is
documented, is merely a struct object and it is easy to construct
incorrect and even self-inconsistent Cookie objects; CookieJar has all
the knowledge about constructing cookies -- and allows use of that
knowledge through .make_cookies() and .load()/.revert().


John

 

[itay_k]

ok.
i will explain what exactly i wanna do.

i want to simulate the following client-side script with python:
<body>
<img name="Pic">

<script>
document.cookie="name=tom";
document.images["Pic"].src="temp2.html"
</script>

what that happen here, is when this page loading, the browser loads
"temp2.html" with HTTP header:
Cookie: name=tom;

this cookie does not come for the server, this is temporary cookie that
sending from the client to the server.
this cookie doesnt save on disk at all.

is it possible to implements this at python??

 

[John J. Lee]

ok.
i will explain what exactly i wanna do.

i want to simulate the following client-side script with python:
<body>
<img name="Pic">

<script>
document.cookie="name=tom";
document.images["Pic"].src="temp2.html"
</script>

Ah! In which case what you're trying to do is a reasonable hack, but
better (UNTESTED):

import urllib2, cookielib
cj = cookielib.CookieJar
opener = urllib2.build_opener(urllib2.HTTPCookieProcessor(cj))
request = urllib2.Request(url)
response = opener.open(request)
response["Set-Cookie"] = "name=tom"
cj.extract_cookies(response, request)


If you have HTML-parsing code to extract these JS cookies that you
want to run on every request (e.g. so even cookies set by JS during
redirections get handled), you can make all this transparent very
easily by using a handler similar to HTTPCookieProcessor itself (using
a recent version of package ClientCookie here for (a recent version
of) the response_seek_wrapper class) UNTESTED:


import urllib2, cookielib

class JSHTTPCookieProcessor(urllib2.BaseHandler):
handler_order = 400 # before HTTPCookieProcessor
def process_response(self, request, response):
from ClientCookie import response_seek_wrapper
if not hasattr(response, "seek"):
response = response_seek_wrapper(response)
try:
name, value = get_js_cookie(response) # your ugly HTML parsing code here ;-)
finally:
response.seek(0)
response["Set-Cookie"] = "%s=%s" % (name, value)
return response

opener = urllib2.build_opener(urllib2.HTTPCookieProcessor(),
JSHTTPCookieProcessor())
response = opener.open(url) # now we're handling JS cookies transparently!



John

 

[Lawrence D'Oliveiro]

"No," scolded Yoda. "Do, or do not. There is no try."

Convincing argument against exceptions, don't you think.

 

[itay_k]

Thanks!