More on Urllib, and Urllib2

Discussion in 'Python' started by Alex Bryan, Jun 28, 2008.

  1. Alex Bryan

    Alex Bryan Guest

    Okay, so I am having issues figuring anything out about this and have
    read the "missing manual" about it so please don't send me that link
    again. To put it simply I want to be able to input a word and get the
    definition from dictionary.com. Now I found a work-around for
    searching for the word, I just make it in the actual address. For
    example I want to search for cheese, I can just do a:

    urllib2.urlopen("http://dictionary.reference.com/browse/cheese")

    However, the actual definition is in javascript on the page. I used
    firebug to see it, and the first def, looks like this:

    <table class="luna-Ent">
    <tbody>
    <tr>
    <td class="dn" valign="top">1.</td>
    <td valign="top">the curd of milk separated from the whey and prepared
    in many ways as a food. </td>

    the problem being that if I use code like this to get the html of that
    page in python:

    response = urllib2.urlopen("the webiste....")
    html = response.read()
    print html

    I get the html source of the page, but no table with my definitions.
    So what can I do? Also, is there a book or a better tutorial or
    explanation of this urllib2, and urllib? If so, PLEASE let me know
    about it; I will be eternally grateful.
    Alex Bryan, Jun 28, 2008
    #1
    1. Advertising

  2. Alex Bryan

    CracKPod Guest

    On 28 Jun., 01:42, Alex Bryan <> wrote:
    > Okay, so I am having issues figuring anything out about this and have  
    > read the "missing manual" about it so please don't send me that link  
    > again. To put it simply I want to be able to input a word and get the  
    > definition from dictionary.com. Now I found a work-around for  
    > searching for the word, I just make it in the actual address. For  
    > example I want to search for cheese, I can just do a:
    >
    > urllib2.urlopen("http://dictionary.reference.com/browse/cheese")
    >
    > However, the actual definition is in javascript on the page. I used  
    > firebug to see it, and the first def, looks like this:
    >
    > <table class="luna-Ent">
    > <tbody>
    > <tr>
    > <td class="dn" valign="top">1.</td>
    > <td valign="top">the curd of milk separated from the whey and prepared  
    > in many ways as a food. </td>
    >
    > the problem being that if I use code like this to get the html of that  
    > page in python:
    >
    > response = urllib2.urlopen("the webiste....")
    > html = response.read()
    > print html
    >
    > I get the html source of the page, but no table with my definitions.  
    > So what can I do? Also, is there a book or a better tutorial or  
    > explanation of this urllib2, and urllib? If so, PLEASE let me know  
    > about it; I will be eternally grateful.


    It would probably be a good idea to take a look at mechanize:
    http://wwwsearch.sourceforge.net/mechanize/
    and at BeautifulSoup: http://www.crummy.com/software/BeautifulSoup/

    Greetz,
    CracKPod
    CracKPod, Jun 28, 2008
    #2
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Alex Stapleton
    Replies:
    2
    Views:
    328
    Fuzzyman
    Mar 14, 2005
  2. Alex Stapleton
    Replies:
    5
    Views:
    720
    Mike Meyer
    Mar 15, 2005
  3. Replies:
    0
    Views:
    769
  4. Jonathan Gardner

    Asynchronous urllib (urllib+asyncore)?

    Jonathan Gardner, Feb 26, 2008, in forum: Python
    Replies:
    1
    Views:
    458
    Terry Jones
    Feb 27, 2008
  5. Juanlu_001
    Replies:
    0
    Views:
    491
    Juanlu_001
    Dec 26, 2010
Loading...

Share This Page