"Try:" which only encompasses head of compound statement

[Jameson.Quinn]

I have:

try:
for line in open(myFileName):
count += 1
except IOError:
print "Can't open myfile"

(I know, this is bad, I never close the file, but its just for
illustration). But then I change it to:

try:
for line in open(myFileName):
count += openAndProcessSubfile(line)
except IOError:
print "Can't open myfile"

.... now the 'except' incorrectly catches errors from
openAndProcessSubfile.
Of course, I can always do:

try:
myFile = open(myFileName)
except IOError:
print "Can't open myfile"
for line in myFile:
count += openAndProcessSubfile(line)

....But if my compound statement is "with" instead of "for" that seems
to defeat the purpose:

try:
myFile = open(myFileName)
except IOError:
print "Can't open myfile"
with myFile as anotherNameForMyFile:
....

This is not a very readable "with" statement, I end up giving the same
thing two names, and there's a moment when the above code doesn't know
to close myFile.

It's hard to imagine a good idiom to fix this. The best I can do is:

for line in open(myFileName)
exceptHead IOError:
print "Can't open myfile"
count += openAndProcessSubfile(line)

exceptHead would be usable as the first indented line in any compound
statement, and would catch any errors raised by the preceding line
only. The bad thing about this (besides the silly name exceptHead) is
that after the except block, control flow would skip the entire
compound statement including else blocks.

 

[Bjoern Schliessmann]

I have:

try:
for line in open(myFileName):
count += 1
except IOError:
print "Can't open myfile"

(I know, this is bad, I never close the file, but its just for
illustration). But then I change it to:

try:
for line in open(myFileName):
count += openAndProcessSubfile(line)
except IOError:
print "Can't open myfile"

... now the 'except' incorrectly catches errors from
openAndProcessSubfile.

The typical way to counter this would be by catching the IOError
earlier so it won't propagate to the outmost "try":

try:
for line in open(myFileName):
try:
count += openAndProcessSubfile(line)
except IOError:
print "Can't open subfile"
except IOError:
print "Can't open myfile"

Regards,


Björn

 

[Lawrence D'Oliveiro]

In message <[email protected]>,

try:
for line in open(myFileName):
count += 1
except IOError:
print "Can't open myfile"

(I know, this is bad, I never close the file, but its just for
illustration).

No, that's fine. The file object returned by "open" is refcounted, so when
the last reference to it disappears, it is closed.

 

[Carl Banks]

I have:

try:
for line in open(myFileName):
count += 1
except IOError:
print "Can't open myfile"

(I know, this is bad, I never close the file, but its just for
illustration). But then I change it to:

try:
for line in open(myFileName):
count += openAndProcessSubfile(line)
except IOError:
print "Can't open myfile"

... now the 'except' incorrectly catches errors from
openAndProcessSubfile.
Of course, I can always do:

try:
myFile = open(myFileName)
except IOError:
print "Can't open myfile"
for line in myFile:
count += openAndProcessSubfile(line)

...But if my compound statement is "with" instead of "for" that seems
to defeat the purpose:

try:
myFile = open(myFileName)
except IOError:
print "Can't open myfile"
with myFile as anotherNameForMyFile:
....

I suspect this isn't often complained about because this usage of
try...except kind of misses the boat. The above treats the exception
as nothing more than an error return code, but it's often not optimal
to handle error so close to the call.

The best reason to catch and handle an exception very close to the
call is if you intend to recover from it right away. For example:

try:
f = open("fileA")
except IOError:
f = open("fileB")

If you don't intend to recover right away, it's usually better to let
the exception propogate until up the call chain until the point where
you do want to recover.

Consider this: is there any other code in your program that has to do
something different based on whether you successfully opened this file
or not? If so, how will you notify it whether the call has succeeded
or not? Very often, the caller itself needs to know. You could, say,
set a flag to indicate it's failed, but why do that when you could
simply let the caller detect and handle the error itself?

If you don't intend to recover at all, you should let it propogate up
to the point where you will resume normal processing (for example:
before the next input prompt), and handle it there. Especially if
you're going to print an error message: printing an error message at
the point the error occurs can severely limit code reusability.


Enough rambling. Here's what I'd do in your situation: nothing.
Don't handle the exception here at all. Let it raise IOError. My
code would look like this:

with open('myFile') as myFile:
....

However, I would handle the exeption wherever I intended to take
action based on success or failure. It looks like you are treating
"file-not-found" as non-fatal, and other IOErrors as fatal? In that
case, we would have to check the error code to distinguish, and
reraise if not file not found.

# this is the point where success or failure
# determines what I do next, that's why I handle
# the exception here.
try:
load_myFile()
except IOError, e:
if e.errno == 2: # file not found is usually errno 2
print "file not found"
else:
raise
else:
# load succeeded, do whatever.


One other useful technique is recasting: catching an error close to
the call, and raising a different error.


Carl Banks

 

[Ben Finney]

try:
for line in open(myFileName):
count += openAndProcessSubfile(line)
except IOError:
print "Can't open myfile"

... now the 'except' incorrectly catches errors from
openAndProcessSubfile.

So don't include it. You've discovered a good principle of
exception-based programming: try to only catch exceptions from the
minimum amount of code that does one discrete action.

try:
input_file = open(my_filename)
except IOError, exc:
print "Can't open myfile: %(exc)" % locals()

for line in input_file:
count += open_and_process_subfile(line)

This is easier to read, because it's clearer what's going on. Never be
ashamed to break apart a compound statement into simpler statements if
that will improve readability.

(It's also easier to read because it follows naming conventions in
PEP-8, which I strongly suggest you follow.)

...But if my compound statement is "with" instead of "for" that seems
to defeat the purpose:

try:
myFile = open(myFileName)
except IOError:
print "Can't open myfile"
with myFile as anotherNameForMyFile:
....

This is not a very readable "with" statement, I end up giving the
same thing two names,

Why do that then? You don't need to assign a different name for the
target.

with input_file:
# ... do stuff

and there's a moment when the above code doesn't know to close
myFile.

Closing the file will occur when the object is cleaned up by garbage
collection.

 

[Scott David Daniels]

(e-mail address removed) writes:
... try to only catch exceptions from the
minimum amount of code that does one discrete action.

try:
input_file = open(my_filename)
except IOError, exc:
print "Can't open myfile: %(exc)" % locals()

for line in input_file:
count += open_and_process_subfile(line)

Actually, this will be followed by some foolishness because
(1) print "Can't open myfile: %(exc)" % locals()
Should at least be:
print "Can't open myfile: %(exc)s" % locals()
But more reasonably:
print "Can't open myfile: %s" % exc
and
(2) Even if the print succeeds, the code will fall through into the loop
and you'll (if you are lucky) get a complaint about
NameError: name 'input_file' is not defined
You'd need to "raise" after the print, but the normal IOError failure
to open message already includes the name of the file it tried to get
to in an attribute "filename", so just catch it outside this code (as
others have already suggested).

-Scott David Daniels
(e-mail address removed)

 

[Lawrence D'Oliveiro]

Consider this: is there any other code in your program that has to do
something different based on whether you successfully opened this file
or not? If so, how will you notify it whether the call has succeeded
or not? Very often, the caller itself needs to know. You could, say,
set a flag to indicate it's failed, but why do that when you could
simply let the caller detect and handle the error itself?

I am generally wary of exceptions, and like to contain them as much as
possible. So my answer to your point is something like

try :
f = open(TheFile, "r")
except IOError, (ErrNo, Msg) :
if ErrNo != errno.ENOENT :
raise
#end if
f = None
#end try

(Note how I check for the specific error code I want to handle.) Then later
on I can determine if the file was successfully opened by

if f != None :
... further processing on f ...
#end if

 

[Ben Finney]

Actually, this will be followed by some foolishness because
(1) [the "print" format argument was wrong]
and
(2) [the code attempts to loop over the failed input file anyway]

You're right. Shame on me for posting untested code snippets, Thanks
for the catch.

 

[Jameson.Quinn]

Those are some good responses, but I think they focused a little too
much on the specifics of my example - especially the 'print'
statement. In real life, I want to set some return values to sensible
defaults if the file doesn't exist, but I want the errors from inside
the block to trickle up as normal. And yes, I can easily use a
temporary variable for the open file, but it really seems to me that
this defeats the purpose of the 'with' statement - if this were
something more dangerous than a file, I'd feel mighty leery of having
it in a temporary variable outside a 'with' for even a moment.

So, the options are:
-temporary variable - could this EVER be more dangerous than the with
statement (KeyboardInterrupt which is later caught and handled?)
-'tunnel' internal exceptions past the try by wrapping them in a
TunnelException and then unwrapping them with a containing try block.

The second looks like a hack to me, and the first still feels
dangerous. So I still think that some syntax here would be a bit more
than just sugar.

 

[Ben Finney]

In real life, I want to set some return values to sensible defaults
if the file doesn't exist, but I want the errors from inside the
block to trickle up as normal.

Sure. Just be careful only to catch the exceptions that you do want to
handle at that level of the code, and the others will propagate
normally.

And yes, I can easily use a temporary variable for the open file,
but it really seems to me that this defeats the purpose of the
'with' statement - if this were something more dangerous than a
file, I'd feel mighty leery of having it in a temporary variable
outside a 'with' for even a moment.

You seem to be confused about one or both of "variables" (which Python
doesn't really have) and the 'with' statement.

When you write::

input_file = open("foo.txt")
with input_file:
# do stuff

there's no "temporary variable". You've merely bound the name
'input_file' to the file object; it's the same file object as is then
used in the 'with' statement. You could bind a dozen other names to
the object and it would still be the same file object.

So, when the 'with' block exits, the context manager will close the
file object, no matter how many names you bind to it.

So, the options are:
-temporary variable - could this EVER be more dangerous than the with
statement (KeyboardInterrupt which is later caught and handled?)

Since it's not a "temporary variable" but rather just a way to refer
to the file object, I don't see the issue you're describing. However
you refer to it, that one file object will be closed when the 'with'
block exits.

-'tunnel' internal exceptions past the try by wrapping them in a
TunnelException and then unwrapping them with a containing try block.

The second looks like a hack to me

Yes, it is. Catch the exceptions you want to handle, let the rest
propagate normally. The context manager created by the 'with'
statement will handle closing the file *and* propagate the exception
normally.

and the first still feels dangerous.

If it's still that way, perhaps you could be more explicit about what
danger you think exists?