urllib IOError Exception

[Bart Nessux]

From the urllib documentation: "If the connection cannot be made, or if
the server returns an error code, the IOError exception is raised. "

Suppose I have an array of IPs and I want to pass each element of the
array to urllib. Basically, I'm just trying to see how many hosts are
serveing-up Web pages in a certain IP range. Is there a way in which I
can handle the IOError so that the script will continue on to the next
host in the array if the host before isn't running a Web server? Below
is my code:

def gen_ip_range():
import urllib
n = 0
hosts = []
networks = []
while n < 254:
n = n + 1
networks.append("192.168.%s." %(n))
for network in networks:
h = 0
while h < 254:
h = h + 1
hosts.append(network+str(h))
for host in hosts:
f = urllib.urlopen("http://%s" %host)
print f.read()
f.close()
gen_ip_range()

Thanks,
Bart

 

[Bart Nessux]

From the urllib documentation: "If the connection cannot be made, or if
the server returns an error code, the IOError exception is raised. "

Suppose I have an array of IPs and I want to pass each element of the
array to urllib. Basically, I'm just trying to see how many hosts are
serveing-up Web pages in a certain IP range. Is there a way in which I
can handle the IOError so that the script will continue on to the next
host in the array if the host before isn't running a Web server? Below
is my code:

def gen_ip_range():
import urllib
n = 0
hosts = []
networks = []
while n < 254:
n = n + 1
networks.append("192.168.%s." %(n))
for network in networks:
h = 0
while h < 254:
h = h + 1
hosts.append(network+str(h))
for host in hosts:
f = urllib.urlopen("http://%s" %host)
print f.read()
f.close()
gen_ip_range()

Thanks,
Bart

I fixed it myself:

try:
f = urllib2.urlopen("http://%s" %host)
except urllib2.URLError:
print host, "has no http server on port 80"

Anyway to speed this up??? The timeout per host is several minutes.

 

[JanC]

try:
f = urllib2.urlopen("http://%s" %host)
except urllib2.URLError:
print host, "has no http server on port 80"

Anyway to speed this up??? The timeout per host is several minutes.

socket.setdefaulttimeout(timeout)

<http://docs.python.org/lib/module-socket.html>

 

[Bart Nessux]

socket.setdefaulttimeout(timeout)

<http://docs.python.org/lib/module-socket.html>

Thanks, that worked... I find it odd that I have to import the socket
module to set a timeout while using the urllib2 module... why isn't
there a function in urllib2 that can handle this?

 

[John J. Lee]

Thanks, that worked... I find it odd that I have to import the socket
module to set a timeout while using the urllib2 module... why isn't
there a function in urllib2 that can handle this?

Because nobody wrote one.

Go ahead and add upload a patch


John