URLError

[Jim]

Program randomly aborts when looking up url. The program loop thru
4000+ records looking
up ID via internet and returns html code which is used in subsequent
processing. The code
for looking-up record is below followed by abort details. Can anyone
help with catching the
abort before program aborts or provide code to automatically start
program? Currently, if
program is restarted the process starts after last good record but
must be restarted manually.

Thanks
Jim

code start here
================
#start lookups pass
while lines < 5000:
lines += 1
d =
ifile.read(8)

print str(lines) + ' ' + d
z = ifile.read(1)
f2 = b + d
h4 = h1+f2+h3 #file name for
saving HMTL file
html_file = open(h4, 'w')
url = a + d + c + b #build url to
lookup record in database
contents = urllib2.urlopen(url).read() #lookup account in
database
soup = BeautifulSoup(contents)
html_file.write(soup.prettify()) #write HTML file
targetPage = soup.prettify()
targetHTML = targetPage #set-up
record for edit pass
html_file.close() #close HTML file


abort details
==============================
429 90078431(note:this record 429 is where abort happen-when program
restarted this record processed normally)

Traceback (most recent call last):
File "C:\Python25\Lib\site-packages\pythonwin\pywin\framework
\scriptutils.py", line 310, in RunScript
exec codeObject in __main__.__dict__
File "Lookup Records.py", line 77, in <module>
contents = urllib2.urlopen(url).read()
File "C:\Python25\lib\urllib2.py", line 121, in urlopen
return _opener.open(url, data)
File "C:\Python25\lib\urllib2.py", line 374, in open
response = self._open(req, data)
File "C:\Python25\lib\urllib2.py", line 392, in _open
'_open', req)
File "C:\Python25\lib\urllib2.py", line 353, in _call_chain
result = func(*args)
File "C:\Python25\lib\urllib2.py", line 1100, in http_open
return self.do_open(httplib.HTTPConnection, req)
File "C:\Python25\lib\urllib2.py", line 1075, in do_open
raise URLError(err)
URLError: <urlopen error (10053, 'Software caused connection abort')>

 

[Steven D'Aprano]

Program randomly aborts when looking up url. The program loop thru
4000+ records looking
up ID via internet and returns html code which is used in subsequent
processing. The code
for looking-up record is below followed by abort details. Can anyone
help with catching the
abort before program aborts or provide code to automatically start
program?

Yes. Wrap the offending code with a try...except loop, something similar
to this.


try:
contents = urllib2.urlopen(url).read()
except URLError, e:
if e.errno == 10053:
# "Software caused connection abort"
response = raw_input(
"Temporary failure, Skip/Halt/try Again?")
response = response.lower().strip()
if response in ('h', 'halt'):
break
elif response in ('a', 'again', 't', 'try', 'try again'):
continue
elif response in ('', 's', 'skip'):
lines -= 1
continue
else:
print "Unknown response, boo hiss to you!"
raise

 

[Jim]

Yes. Wrap the offending code with a try...except loop, something similar
to this.

try:
contents = urllib2.urlopen(url).read()
except URLError, e:
if e.errno == 10053:
# "Software caused connection abort"
response = raw_input(
"Temporary failure, Skip/Halt/try Again?")
response = response.lower().strip()
if response in ('h', 'halt'):
break
elif response in ('a', 'again', 't', 'try', 'try again'):
continue
elif response in ('', 's', 'skip'):
lines -= 1
continue
else:
print "Unknown response, boo hiss to you!"
raise

Thanks Steven for recommendation.

The program is my first and I'm not a programmer so it will take me
some time to get your recommendation to work. So far the program runs
after I added code based on your example but the program still aborts
and none of the code ("Temporary failure, Skip/Halt/try Again?" or
"Unknown response, boo hiss to you!") in your example is displayed. It
appears to abort in the imported module "urllib2" which is open
source. The code in urllib2 is way over my skill level.

Jim

 

[Steven D'Aprano]

The program is my first and I'm not a programmer so it will take me some
time to get your recommendation to work. So far the program runs after I
added code based on your example but the program still aborts and none
of the code ("Temporary failure, Skip/Halt/try Again?" or "Unknown
response, boo hiss to you!") in your example is displayed.

Try replacing the line:

except URLError, e:


with:

except urllib2.URLError, e:


and see if that helps.

 

[Jim]

Try replacing the line:

exceptURLError, e:

with:

except urllib2.URLError, e:

and see if that helps.

Steven,

Got it to work with your example see code below. The process continued
bypassing the records when exceptions occurred.
My process displayed 'print here 1' then 'print here 4' and continued
process with next record without aborting. What
should be displayed, if anything, with the line "response =
raw_input("Temporary failure,
Skip/Halt/try Again?")" as I didn't see anything?

Thanks for your help
Jim

Code:
except urllib2.URLError, e:
print "here 1"
if e.errno == 10053:
response = raw_input("Temporary failure, Skip/Halt/try
Again?")
response = response.lower().strip()
if response in ('h', 'halt'):
print "here 2"
break
elif response in ('a', 'again', 't', 'try', 'try again'):
print "here 3"
continue
elif response in ('', 's', 'skip'):
print "here 4"
lines -= 1
continue
else:
print "Unknown response, boo hiss to you!"
raise