How to except the unexpected?

[Roman Susi]

Perhaps a MIX emulator and running TeXDoctest on his books will convince
him..

Or maybe Python written for MIX...


-Roman

 

[James Stroud]

Steven D'Aprano:



Yes, I think I'll do something like this. Perhaps combined with Peter's
advice to not micromanage, like so:

Reraise = (LookupError, ArithmeticError, AssertionError) # And then some

try:
process_things()
except Reraise:
raise
except:
log_error()

Why catch an error only to re-raise it?

This falls under http://c2.com/cgi/wiki?YouReallyArentGonnaNeedThis

 

[Alex Martelli]

Why catch an error only to re-raise it?

To avoid the following handler -- a pretty common idiom.


Alex

 

[Paul Rubin]

Why catch an error only to re-raise it?

This falls under http://c2.com/cgi/wiki?YouReallyArentGonnaNeedThis

No, without the reraise, the bare "except:" clause would capture the
exception incorrectly and log it as an error.

 

[James Stroud]

James Stroud:



Your conclusion may be (almost) right in this case. I just don't like this
approach. Basically this is reverse engineering the interface from the
source at the time of writing the app.

This is using the source as documentation, there is no law against that.
It's better than waiting for a closed source software mogul to
benevolently give you the API documentation, which may not be 100%
accurate itself.

Even if you get it right, it may
fail next week when someone added an exception to a module.

Given how all of the exceptions raised by both modules are either (1)
builtin or (2) defined in the module itself and given that all
exceptions defined in these modules inheret from a single class, my
money is on the fact that this approach should be forward compatible.

Why? It shouldn't be.

How things are and how things should be have always been 2 entirely
different things. See early books by Robert J. Ringer for a more
complete discussion.

 

[Bruno Desthuilliers]

Rene Pijlman a écrit :

Jorge Godoy:



That's cheating: pydoc is reading the source

Yes, and that's the Right Thing(tm) to do. Source code don't lie. Source
code don't get out of sync. So source code *is* the best documentation
(or at least the most accurate).

What I meant was, I'm calling robotparser, and there's no mention of
exceptions on http://docs.python.org/lib/module-robotparser.html

That's why reading the source is better. CQFD !-)

 

[Paul Rubin]

This is using the source as documentation, there is no law against
that.

That's completely bogus. Undocumented interfaces in the library
source are allowed to change between Python versions and across Python
implementations. If you write your application according to the
documented interfaces in the Python manual, it should not break when
someone upgrades to the next Python release. You should not have to
depend on undocumented aspects of the Python implementation which
change out of sync with the application.

How things are and how things should be have always been 2 entirely
different things. See early books by Robert J. Ringer for a more
complete discussion.

That's like saying programs shouldn't have bugs, but they often do
anyway. It doesn't mean the bugs are a good thing or that they
shouldn't be fixed.

 

[James Stroud]

That's completely bogus. Undocumented interfaces in the library
source are allowed to change between Python versions and across Python
implementations. If you write your application according to the
documented interfaces in the Python manual, it should not break when
someone upgrades to the next Python release. You should not have to
depend on undocumented aspects of the Python implementation which
change out of sync with the application.

My suggestion was to use some common sense about the source code and
apply it. I've wasted many hours programming to faulty documentation
when a quick peek at the source code would have saved some serious time.
How do you think I happened accross the technique?

I would wager that, in 10 years time, the urllib2 module will not raise
any exceptions that don't inheret from urllib2.URLError or a built-in
(i.e. documented) exception; unless, of course, someone made a change in
a deliberate attempt to win said wager ;o)

That's like saying programs shouldn't have bugs, but they often do
anyway. It doesn't mean the bugs are a good thing or that they
shouldn't be fixed.

The original reference was to the internet, which seems vast in its
sources of unforseen peril. If some gracious soul aspires to fix every
bug in the internet, I invite him to be my guest.

James

 

[Paul Rubin]

My suggestion was to use some common sense about the source code and
apply it.

The common sense to apply is that if the source code says one thing
and the documentation says another, then one of them is wrong and
should be updated. Usually it's the source code that's wrong, but in
either case, saying to ignore the documentation and follow the code,
rather than repairing the discrepancy somehow, is poor advice.

 

[Steven D'Aprano]

Why catch an error only to re-raise it?

As a general tactic, you may want to catch an exception, print extra
debugging information, then raise the exception again, like this:

try:
x = somedata()
y = someotherdata()
z = process(x, y)
except Exception:
print "Error!"
print "Value of x: %s" % x
print "Value of y: %s" % y
raise


Although that's not the reason the Original Poster is doing this.

The OP is doing it because catching all exceptions masks bugs. There are
certain exceptions which should be allowed through, as they indicate a bug
in the OP's code. Normally the tactic is to catch only the exceptions you
are interested in, and let everything else through, but the OP needs to
catch all exceptions because there are rare exceptions which he can't
predict in advance.

 

[Rene Pijlman]

Steven D'Aprano:

The OP is doing it because catching all exceptions masks bugs. There are
certain exceptions which should be allowed through, as they indicate a bug
in the OP's code. Normally the tactic is to catch only the exceptions you
are interested in, and let everything else through, but the OP needs to
catch all exceptions because there are rare exceptions which he can't
predict in advance.

To add to that explanation: this is in a multithreaded ZODB-application.
When an unexpected exception occurs and remains uncaught, a thread
terminates, causing the thread pool to wait forever since some thread is
not consuming its termination request from the queue, causing the app to
not terminate, causing the ZODB to remain locked, causing other apps to
fail, causing havoc on my server.

I don't mind this when it's caused by a bug in my code, since that creates
the sense of urgency required to fix the bug. But it's annoying when it
happens because of an I/O exception caused by some other guys bug on the
other side of the planet

 

[Scott David Daniels]

Steven D'Aprano:

... This is in a multithreaded ZODB-application....
When an unexpected exception occurs and remains uncaught, a thread
terminates, ....

At the base of the thread code, you could put

import sys, threading, logging

class MyThread(threading.Thread):
def run(self):
try:
threading.Thread.run(self) # or whatever
except:
etype, error, traceback = sys.exc_info()
logging.warning('Exception %s: %s seen at %s' %
(etype.__name__, error, _someformat_(traceback)))
_try_to_rescue_or remove_this_thread_

If the condition is infrequent enough. If not (if you run a real
risk of multiple threads accessing the log simultaneously), have
a queue of log messages that you feed to a single logging thread.

 

[plahey]

Yes, and that's the Right Thing(tm) to do. Source code don't lie. Source

code don't get out of sync. So source code *is* the best documentation
(or at least the most accurate).

I could not disagree more strongly with this. Not just no, but hell
no!

Yes, and that's the Right Thing(tm) to do.

No, it is a horrible thing to do. But since the documentation of some
modules is just plain horrible we sometimes have no choice.

Source code don't lie. Source code don't get out of sync.

True but implementation details change from release to release.

So source code *is* the best documentation (or at least the most accurate).

No, source code is the *worst possible* documentation because it makes
no distinction between implementation detail and method contract. If
the implementer considers the implementation to be the documentation
then his/her refactoring options are significantly reduced. Typically
implementers are not aware of this and they refactor anyway, breaking
client code left and right.

The C++ FAQ has a nice discussion of this issue. Minimally acceptable
documentation consists of the following (I think this is language
independent):

PURPOSE: What does this method/function do
REQUIRE: pre-conditions - What must have happened before calling this
method (or restrictions on the domain of the inputs)
PROMISE: post-conditions - What can you expect upon return or what
exceptions can be thrown

I consider the above to be the minimal amount of documentation that is
acceptable. If you have less than that, I consider the method to be
undocumented. Needless to say, I consider lots of code that I see to
be undocumented.

If you don't have the above, you get the problems that OP was hitting
(or worse, see the C++ FAQ). I am not a huge fan of Java's ubiquitous
use of checked exceptions or even of static typing but it does help
supply some of the above documentation (although in a suboptimal way)
that must be supplied by hand in Python. This is the dirty little
secret of dynamically typed languages. It makes proper documentation
even more important because method signatures supply less information.