urllib IOError Exception

Discussion in 'Python' started by Bart Nessux, Jun 11, 2004.

  1. Bart Nessux

    Bart Nessux Guest

    From the urllib documentation: "If the connection cannot be made, or if
    the server returns an error code, the IOError exception is raised. "

    Suppose I have an array of IPs and I want to pass each element of the
    array to urllib. Basically, I'm just trying to see how many hosts are
    serveing-up Web pages in a certain IP range. Is there a way in which I
    can handle the IOError so that the script will continue on to the next
    host in the array if the host before isn't running a Web server? Below
    is my code:

    def gen_ip_range():
    import urllib
    n = 0
    hosts = []
    networks = []
    while n < 254:
    n = n + 1
    networks.append("192.168.%s." %(n))
    for network in networks:
    h = 0
    while h < 254:
    h = h + 1
    hosts.append(network+str(h))
    for host in hosts:
    f = urllib.urlopen("http://%s" %host)
    print f.read()
    f.close()
    gen_ip_range()

    Thanks,
    Bart
     
    Bart Nessux, Jun 11, 2004
    #1
    1. Advertising

  2. Bart Nessux

    Bart Nessux Guest

    Bart Nessux wrote:
    > From the urllib documentation: "If the connection cannot be made, or if
    > the server returns an error code, the IOError exception is raised. "
    >
    > Suppose I have an array of IPs and I want to pass each element of the
    > array to urllib. Basically, I'm just trying to see how many hosts are
    > serveing-up Web pages in a certain IP range. Is there a way in which I
    > can handle the IOError so that the script will continue on to the next
    > host in the array if the host before isn't running a Web server? Below
    > is my code:
    >
    > def gen_ip_range():
    > import urllib
    > n = 0
    > hosts = []
    > networks = []
    > while n < 254:
    > n = n + 1
    > networks.append("192.168.%s." %(n))
    > for network in networks:
    > h = 0
    > while h < 254:
    > h = h + 1
    > hosts.append(network+str(h))
    > for host in hosts:
    > f = urllib.urlopen("http://%s" %host)
    > print f.read()
    > f.close()
    > gen_ip_range()
    >
    > Thanks,
    > Bart


    I fixed it myself:

    try:
    f = urllib2.urlopen("http://%s" %host)
    except urllib2.URLError:
    print host, "has no http server on port 80"

    Anyway to speed this up??? The timeout per host is several minutes.
     
    Bart Nessux, Jun 11, 2004
    #2
    1. Advertising

  3. Bart Nessux

    JanC Guest

    Bart Nessux <> schreef:

    > try:
    > f = urllib2.urlopen("http://%s" %host)
    > except urllib2.URLError:
    > print host, "has no http server on port 80"
    >
    > Anyway to speed this up??? The timeout per host is several minutes.


    socket.setdefaulttimeout(timeout)

    <http://docs.python.org/lib/module-socket.html>

    --
    JanC

    "Be strict when sending and tolerant when receiving."
    RFC 1958 - Architectural Principles of the Internet - section 3.9
     
    JanC, Jun 11, 2004
    #3
  4. Bart Nessux

    Bart Nessux Guest

    JanC wrote:
    > Bart Nessux <> schreef:
    >
    >
    >> try:
    >> f = urllib2.urlopen("http://%s" %host)
    >> except urllib2.URLError:
    >> print host, "has no http server on port 80"
    >>
    >>Anyway to speed this up??? The timeout per host is several minutes.

    >
    >
    > socket.setdefaulttimeout(timeout)
    >
    > <http://docs.python.org/lib/module-socket.html>
    >


    Thanks, that worked... I find it odd that I have to import the socket
    module to set a timeout while using the urllib2 module... why isn't
    there a function in urllib2 that can handle this?
     
    Bart Nessux, Jun 11, 2004
    #4
  5. Bart Nessux

    John J. Lee Guest

    Bart Nessux <> writes:

    > JanC wrote:
    > > Bart Nessux <> schreef:
    > >
    > >> try:
    > >> f = urllib2.urlopen("http://%s" %host)
    > >> except urllib2.URLError:
    > >> print host, "has no http server on port 80"
    > >>
    > >>Anyway to speed this up??? The timeout per host is several minutes.

    > > socket.setdefaulttimeout(timeout)
    > > <http://docs.python.org/lib/module-socket.html>
    > >

    >
    > Thanks, that worked... I find it odd that I have to import the socket
    > module to set a timeout while using the urllib2 module... why isn't
    > there a function in urllib2 that can handle this?


    Because nobody wrote one.

    Go ahead and add upload a patch :)


    John
     
    John J. Lee, Jun 14, 2004
    #5
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Jim Cobban

    Xerces parse aborted on IOError

    Jim Cobban, Dec 5, 2003, in forum: XML
    Replies:
    0
    Views:
    411
    Jim Cobban
    Dec 5, 2003
  2. Jonathan Gardner

    Asynchronous urllib (urllib+asyncore)?

    Jonathan Gardner, Feb 26, 2008, in forum: Python
    Replies:
    1
    Views:
    500
    Terry Jones
    Feb 27, 2008
  3. Chris McDonald
    Replies:
    0
    Views:
    325
    Chris McDonald
    Nov 1, 2010
  4. loial
    Replies:
    1
    Views:
    177
    Peter Otten
    Apr 28, 2011
  5. Replies:
    6
    Views:
    370
    Ian Kelly
    Apr 9, 2013
Loading...

Share This Page