Looking for a urllib(2) cookie handler

Discussion in 'Python' started by Mark Carter, Apr 19, 2011.

  1. Mark Carter

    Mark Carter Guest

    I'm in python 2.6.5, and have Firefox 3.6.13. I would like to download
    some html from a site and scrape it programatically. The site requires
    a cookie, which I have in Firefox.

    Is there a simple python recipe I can use to read the contents of a
    url and say "just use the cookie that I have in Firefox"?
    Mark Carter, Apr 19, 2011
    #1
    1. Advertising

  2. Mark Carter

    Chris Rebert Guest

    On Tue, Apr 19, 2011 at 4:44 AM, Mark Carter <> wrote:
    > I'm in python 2.6.5, and have Firefox 3.6.13. I would like to download
    > some html from a site and scrape it programatically. The site requires
    > a cookie, which I have in Firefox.
    >
    > Is there a simple python recipe I can use to read the contents of a
    > url and say "just use the cookie that I have in Firefox"?


    Untested (3rd Google hit for "import firefox cookies python"):
    http://blog.mithis.net/archives/python/90-firefox3-cookies-in-python

    At a minimum, I think you'll need to replace:
    from pysqlite2 import dbapi2 as sqlite
    With:
    import sqlite3 as sqlite

    You'll also have to figure out where Firefox's `cookies.sqlite` file
    is located on your system.

    Cheers,
    Chris
    --
    My compiler is compiling, I swear!
    http://blog.rebertia.com
    Chris Rebert, Apr 19, 2011
    #2
    1. Advertising

  3. Mark Carter

    Mark Carter Guest

    On Apr 19, 12:44 pm, Mark Carter <> wrote:

    > url and say "just use the cookie that I have in Firefox"?


    "mechanize" looks kinda like what I want, but i still can't get it to
    work properly. So far I have:

    import cookielib
    import mechanize

    cookiefile = "C:\\Users\\$ME\\AppData\\Roaming\\Mozilla\\Firefox\
    \Profiles\\zl648qvt.default\\cookies.sqlite"
    cookies = mechanize.MozillaCookieJar(filename = cookiefile,
    delayload=True)
    #cookies = cookielib.MozillaCookieJar()
    #cookies = cookielib.MSIECookieJar()
    #cookies.load_from_registry() # finds cookie index file from registry
    br = mechanize.Browser()
    br.set_cookiejar(cookies)
    br.addheaders = [('User-agent', 'Mozilla/5.0 (X11; U; Linux i686; en-
    US; rv:1.9.0.1) Gecko/2008071615 Fedora/3.0.1-1.fc9 Firefox/3.0.1')]
    url = "$URL"
    r = br.open(url)

    #print cj
    #opener =
    mechanize.build_opener(mechanize.HTTPCookieProcessor(cookies))


    html = r.read()
    print html

    where $ME and $URL are replaced with suitable values. It doesn't
    appear to acutally be "using" the cookies.
    Mark Carter, Apr 19, 2011
    #3
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. shruds
    Replies:
    1
    Views:
    757
    John C. Bollinger
    Jan 27, 2006
  2. Replies:
    1
    Views:
    686
    Damien
    Feb 22, 2007
  3. Replies:
    0
    Views:
    343
  4. Jonathan Gardner

    Asynchronous urllib (urllib+asyncore)?

    Jonathan Gardner, Feb 26, 2008, in forum: Python
    Replies:
    1
    Views:
    462
    Terry Jones
    Feb 27, 2008
  5. Chris McDonald
    Replies:
    0
    Views:
    276
    Chris McDonald
    Nov 1, 2010
Loading...

Share This Page