Re: [python] How to detect a remote webpage is accessible? (in HTTP)

Discussion in 'Python' started by Astan Chee, Jan 18, 2008.

  1. Astan Chee

    Astan Chee Guest

    How about:

    import socket, urllib2

    timeout = 10
    socket.setdefaulttimeout(timeout)
    try:
    auth_handler = urllib2.HTTPBasicAuthHandler()
    opener = urllib2.build_opener(auth_handler) #this used if we need
    authentication
    urllib2.install_opener(opener)
    req = urllib2.Request('http://website.com')
    f = urllib2.urlopen(req)
    notes= f.readlines()
    f.close()
    print "Everything is ok"
    except IOError, r:
    p = str(r)
    if re.search(r'urlopen error timed out',p):
    print "Web page timed out"

    You'll need to set up the timeout to whatever duration your website
    takes to load.
    Cheers
    Astan

    ?? wrote:
    > Howdy, all,
    > I want to use python to detect the accessibility of website.
    > Currently, I use urllib
    > to obtain the remote webpage, and see whether it fails. But the problem is that
    > the webpage may be very large; it takes too long time. Certainly, it
    > is no need to download
    > the entire page. Could you give me a good and fast solution?
    > Thank you.
    > --
    > ShenLei
    >
    Astan Chee, Jan 18, 2008
    #1
    1. Advertising

  2. Astan Chee

    coldpizza Guest

    Re: How to detect a remote webpage is accessible? (in HTTP)

    I suppose that if the file is really big and you don't need to read
    all of it
    then instead of f.readlines() you could use f.read(256) to read just
    the first 256 bytes.

    On Jan 18, 7:28 am, Astan Chee <> wrote:
    > How about:
    >
    > import socket, urllib2
    >
    > timeout = 10
    > socket.setdefaulttimeout(timeout)
    > try:
    > auth_handler = urllib2.HTTPBasicAuthHandler()
    > opener = urllib2.build_opener(auth_handler) #this used if we need
    > authentication
    > urllib2.install_opener(opener)
    > req = urllib2.Request('http://website.com')
    > f = urllib2.urlopen(req)
    > notes= f.readlines()
    > f.close()
    > print "Everything is ok"
    > except IOError, r:
    > p = str(r)
    > if re.search(r'urlopen error timed out',p):
    > print "Web page timed out"
    >
    > You'll need to set up the timeout to whatever duration your website
    > takes to load.
    > Cheers
    > Astan
    >
    >
    >
    > ?? wrote:
    > > Howdy, all,
    > >      I want to use python to detect the accessibility of website.
    > > Currently, I use urllib
    > > to obtain the remote webpage, and see whether it fails. But the problem is that
    > > the webpage may be very large; it takes too long time. Certainly, it
    > > is no need to download
    > > the entire page. Could you give me a good and fast solution?
    > >     Thank you.
    > > --
    > > ShenLei
    coldpizza, Jan 18, 2008
    #2
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Charlie363
    Replies:
    2
    Views:
    509
    charlie charlie
    May 31, 2005
  2. Ìð¹Ï
    Replies:
    3
    Views:
    328
    John Nagle
    Jan 18, 2008
  3. dmitrey
    Replies:
    6
    Views:
    431
    André
    Jan 16, 2010
  4. Charlie363
    Replies:
    3
    Views:
    216
    Mark Schupp
    May 31, 2005
  5. soren625
    Replies:
    2
    Views:
    362
    soren625
    Dec 12, 2006
Loading...

Share This Page