Waiting for receiving data

[Anjanesh Lekshminarayanan]

fp = urllib.urlopen(url)
data = fp.read()

Retrieving XML data via an XML service API.
Very often network gets stuck in between. No errors / exceptions.

CTRL+C

File "get-xml.py", line 32, in <module>
fp = urllib.urlopen(url)
File "/usr/lib/python2.6/urllib.py", line 87, in urlopen
return opener.open(url)
File "/usr/lib/python2.6/urllib.py", line 206, in open
return getattr(self, name)(url)
File "/usr/lib/python2.6/urllib.py", line 348, in open_http
errcode, errmsg, headers = h.getreply()
File "/usr/lib/python2.6/httplib.py", line 1048, in getreply
response = self._conn.getresponse()
File "/usr/lib/python2.6/httplib.py", line 974, in getresponse
response.begin()
File "/usr/lib/python2.6/httplib.py", line 391, in begin
version, status, reason = self._read_status()
File "/usr/lib/python2.6/httplib.py", line 349, in _read_status
line = self.fp.readline()
File "/usr/lib/python2.6/socket.py", line 397, in readline
data = recv(1)
KeyboardInterrupt

Is there I can do to try something else if its taking too long to
retrieve from the network ? Like kill previous attempt and retry ?

Thanks
Anjanesh Lekshmnarayanan

 

[Gerald Walker]

fp = urllib.urlopen(url)
data = fp.read()

Retrieving XML data via an XML service API.
Very often network gets stuck in between. No errors / exceptions.

CTRL+C

File "get-xml.py", line 32, in <module>
fp = urllib.urlopen(url)
File "/usr/lib/python2.6/urllib.py", line 87, in urlopen
return opener.open(url)
File "/usr/lib/python2.6/urllib.py", line 206, in open
return getattr(self, name)(url)
File "/usr/lib/python2.6/urllib.py", line 348, in open_http
errcode, errmsg, headers = h.getreply()
File "/usr/lib/python2.6/httplib.py", line 1048, in getreply
response = self._conn.getresponse()
File "/usr/lib/python2.6/httplib.py", line 974, in getresponse
response.begin()
File "/usr/lib/python2.6/httplib.py", line 391, in begin
version, status, reason = self._read_status()
File "/usr/lib/python2.6/httplib.py", line 349, in _read_status
line = self.fp.readline()
File "/usr/lib/python2.6/socket.py", line 397, in readline
data = recv(1)
KeyboardInterrupt

Is there I can do to try something else if its taking too long to
retrieve from the network ? Like kill previous attempt and retry ?

Thanks
Anjanesh Lekshmnarayanan

import socket
from urllib2 import urlopen

# A one-hundredths of a second (0.01) timeout before socket throws
# an exception to demonstrate catching the timeout.
# Obviously, this you will set this greater than 0.01 in real life.
socket.setdefaulttimeout(0.01)

# example xml feed
xml_source = "http://mlb.mlb.com/partnerxml/gen/news/rss/mlb.xml"

try:
data = urlopen(xml_source)
except urllib2.URLError, e:
print 'URLError = ' + str(e.reason)

 

[Gerald Walker]

import socket
from urllib2 import urlopen

# A one-hundredths of a second (0.01) timeout before socket throws
# an exception to demonstrate catching the timeout.
# Obviously, this you will set this greater than 0.01 in real life.
socket.setdefaulttimeout(0.01)

# example xml feed
xml_source = "http://mlb.mlb.com/partnerxml/gen/news/rss/mlb.xml"

try:
data = urlopen(xml_source)
except urllib2.URLError, e:
print 'URLError = ' + str(e.reason)

Also, if you are using multiple threads to retrieve the xml source(s)
and any thread blocks due to network problems, the thread can go way by
itself after the default timeout expires.

 

[Gerald Walker]

Also, if you are using multiple threads to retrieve the xml source(s)
and any thread blocks due to network problems, the thread can go way by
itself after the default timeout expires.

Typo, edited for clarity:
That is: "..the thread can go *away* by itself after the default timeout
expires." You don't need to explicitly kill it.

 

[Gerald Walker]

import socket
from urllib2 import urlopen

# A one-hundredths of a second (0.01) timeout before socket throws
# an exception to demonstrate catching the timeout.
# Obviously, this you will set this greater than 0.01 in real life.
socket.setdefaulttimeout(0.01)

# example xml feed
xml_source = "http://mlb.mlb.com/partnerxml/gen/news/rss/mlb.xml"

try:
data = urlopen(xml_source)
except urllib2.URLError, e:
print 'URLError = ' + str(e.reason)

Oops, the above doesn't run. This version works:

import socket
import urllib2

# A one-hundredths of a second (0.01) timeout before socket throws
# an exception to demonstrate catching the timeout.
# Obviously, this you will set this greater than 0.01 in real life.
socket.setdefaulttimeout(0.01)

# example xml feed
xml_source = "http://mlb.mlb.com/partnerxml/gen/news/rss/mlb.xml"

try:
data = urllib2.urlopen(xml_source)
except urllib2.URLError, e:
print 'URLError = ' + str(e.reason)