Re: How to read webpage

Discussion in 'Python' started by MRAB, Aug 1, 2009.

  1. MRAB

    MRAB Guest

    tarun wrote:
    > Dear All,
    > I want to read a webpage and copy the contents of it in word file. I
    > tried to write following code:
    >
    > import urllib2
    > urllib2.urlopen("http://www.rediff.com/")
    >
    > *Error:-*
    >
    > urllib2.urlopen("http://www.icicibank.com/")
    > File "C:\Python25\lib\urllib2.py", line 121, in urlopen
    > return _opener.open(url, data)
    > File "C:\Python25\lib\urllib2.py", line 374, in open
    > response = self._open(req, data)
    > File "C:\Python25\lib\urllib2.py", line 392, in _open
    > '_open', req)
    > File "C:\Python25\lib\urllib2.py", line 353, in _call_chain
    > result = func(*args)
    > File "C:\Python25\lib\urllib2.py", line 1100, in http_open
    > return self.do_open(httplib.HTTPConnection, req)
    > File "C:\Python25\lib\urllib2.py", line 1075, in do_open
    > raise URLError(err)
    > urllib2.URLError: <urlopen error (11001, 'getaddrinfo failed')>
    >

    I've just tried it. I didn't get an exception, so your problem must be
    elsewhere.
     
    MRAB, Aug 1, 2009
    #1
    1. Advertising

  2. MRAB

    koranthala Guest

    On Aug 1, 6:52 pm, MRAB <> wrote:
    > tarun wrote:
    > > Dear All,
    > > I want to read a webpage and copy the contents of it in word file. I
    > > tried to write following code:

    >
    > > import urllib2
    > > urllib2.urlopen("http://www.rediff.com/")

    >
    > > *Error:-*

    >
    > >     urllib2.urlopen("http://www.icicibank.com/")
    > >   File "C:\Python25\lib\urllib2.py", line 121, in urlopen
    > >     return _opener.open(url, data)
    > >   File "C:\Python25\lib\urllib2.py", line 374, in open
    > >     response = self._open(req, data)
    > >   File "C:\Python25\lib\urllib2.py", line 392, in _open
    > >     '_open', req)
    > >   File "C:\Python25\lib\urllib2.py", line 353, in _call_chain
    > >     result = func(*args)
    > >   File "C:\Python25\lib\urllib2.py", line 1100, in http_open
    > >     return self.do_open(httplib.HTTPConnection, req)
    > >   File "C:\Python25\lib\urllib2.py", line 1075, in do_open
    > >     raise URLError(err)
    > > urllib2.URLError: <urlopen error (11001, 'getaddrinfo failed')>

    >
    > I've just tried it. I didn't get an exception, so your problem must be
    > elsewhere.


    Is it that the website expects a valid browser?
    In that case, spoof a browser and try to get the site.
     
    koranthala, Aug 1, 2009
    #2
    1. Advertising

  3. MRAB

    Jon Clements Guest

    On 1 Aug, 14:52, MRAB <> wrote:
    > tarun wrote:
    > > Dear All,
    > > I want to read a webpage and copy the contents of it in word file. I
    > > tried to write following code:

    >
    > > import urllib2
    > > urllib2.urlopen("http://www.rediff.com/")

    >
    > > *Error:-*

    >
    > >     urllib2.urlopen("http://www.icicibank.com/")
    > >   File "C:\Python25\lib\urllib2.py", line 121, in urlopen
    > >     return _opener.open(url, data)
    > >   File "C:\Python25\lib\urllib2.py", line 374, in open
    > >     response = self._open(req, data)
    > >   File "C:\Python25\lib\urllib2.py", line 392, in _open
    > >     '_open', req)
    > >   File "C:\Python25\lib\urllib2.py", line 353, in _call_chain
    > >     result = func(*args)
    > >   File "C:\Python25\lib\urllib2.py", line 1100, in http_open
    > >     return self.do_open(httplib.HTTPConnection, req)
    > >   File "C:\Python25\lib\urllib2.py", line 1075, in do_open
    > >     raise URLError(err)
    > > urllib2.URLError: <urlopen error (11001, 'getaddrinfo failed')>

    >
    > I've just tried it. I didn't get an exception, so your problem must be
    > elsewhere.


    I'm hoping this adds to MRAB's reply; it is intended however for the
    OP.

    Jeeze -- been a while since I've had to deal with Sockets (directly
    anyway).
    If memory serves correctly, it's where the system can't name resolve
    the required address.
    So best guess is it's either a temporary glitch, or an issue with your
    routing.

    Jon.
    Jon.
     
    Jon Clements, Aug 1, 2009
    #3
  4. MRAB

    catafest Guest

    Maybe your python2.5 not working good!?
    But, I use python 2.6 , and i use this for your problem:
    import urllib
    html = urllib.urlopen("http://www.rediff.com/").read()
    print html

    If you want use authenticate then...
    You make working urllib2 and use this
    >>>auth = urllib2.Request(auth_uri, authreq_data)


    On Aug 1, 4:52 pm, MRAB <> wrote:
    > tarun wrote:
    > > Dear All,
    > > I want to read a webpage and copy the contents of it in word file. I
    > > tried to write following code:

    >
    > > import urllib2
    > > urllib2.urlopen("http://www.rediff.com/")

    >
    > > *Error:-*

    >
    > >     urllib2.urlopen("http://www.icicibank.com/")
    > >   File "C:\Python25\lib\urllib2.py", line 121, in urlopen
    > >     return _opener.open(url, data)
    > >   File "C:\Python25\lib\urllib2.py", line 374, in open
    > >     response = self._open(req, data)
    > >   File "C:\Python25\lib\urllib2.py", line 392, in _open
    > >     '_open', req)
    > >   File "C:\Python25\lib\urllib2.py", line 353, in _call_chain
    > >     result = func(*args)
    > >   File "C:\Python25\lib\urllib2.py", line 1100, in http_open
    > >     return self.do_open(httplib.HTTPConnection, req)
    > >   File "C:\Python25\lib\urllib2.py", line 1075, in do_open
    > >     raise URLError(err)
    > > urllib2.URLError: <urlopen error (11001, 'getaddrinfo failed')>

    >
    > I've just tried it. I didn't get an exception, so your problem must be
    > elsewhere.
     
    catafest, Aug 2, 2009
    #4
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Replies:
    1
    Views:
    428
  2. Juergen Wohnich

    How can i read a webpage in a buffer

    Juergen Wohnich, Dec 3, 2005, in forum: C++
    Replies:
    1
    Views:
    344
    Jonathan Mcdougall
    Dec 3, 2005
  3. Paul
    Replies:
    14
    Views:
    852
    Alexey Smirnov
    Jun 19, 2008
  4. sifar
    Replies:
    5
    Views:
    426
  5. soren625
    Replies:
    2
    Views:
    378
    soren625
    Dec 12, 2006
Loading...

Share This Page