How to pop the interpreter's stack?

[Steven D'Aprano]

While the idea of being able to do this (create a general validation
exception) sounds mildly appealing at first, what happens when the
module implementing this documented API and documented error, has a bug?
It seems that the user, or even module developer in the midst of
writing, would now have no tools to easily tackle the problem, and no
useful information to submit in the required bug report.

That's very true, but the same applies to *any* use of encapsulation. Any
time you hide information, you hide information (duh!). This doesn't stop
us from doing this:

def public(x):
if condition: return _private(x)
elif other_condition: return _more_private(x+1)
else: return _yet_another_private(x-1)

If you call public(42), and get the wrong answer, it's a bug, but the
source of the bug is hidden from the caller. If you have access to the
source code, you can work out where the bug lies (which of the three
private functions is buggy?) given the argument, but the return result
itself does not expose any information about where the bug lies. This is
considered an unavoidable but acceptable side-effect of an otherwise
desirable state of affairs: information hiding and encapsulation. The
caller being unaware of where and how the result is calculated is
considered a good thing, and the fact that it occasionally adds to the
debugging effort is considered such a trivial cost that it normally isn't
remarked upon, except by lunatics and die-hard fans of spaghetti code
using GOTO. But I repeat myself.

Why should exceptions *necessarily* be different? As I've repeatedly
acknowledged, for an unexpected exception (a bug), the developer needs
all the help he can get, and the current behaviour is the right way to do
it. You won't hear me argue differently. But for a documented, explicit,
expected, deliberate exception, Python breaks encapsulation by exposing
the internal details of any internal subroutines used to generate that
exception. This leads to messy tracebacks that obscure the source of bugs
in the caller's code:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib/python2.6/re.py", line 190, in compile
return _compile(pattern, flags)
File "/usr/lib/python2.6/re.py", line 245, in _compile
raise error, v # invalid expression
sre_constants.error: unbalanced parenthesis

I think critics of my position have forgotten what it's like to learning
the language. One of the most valuable skills to learn is to *ignore
parts of the traceback* -- a skill that takes practice and familiarity
with the library or function in use. To those who are less familiar with
the function, it can be very difficult to determine which parts of the
traceback are relevant and which aren't. In this case, the caller has
nothing to do with _compile, and the traceback looks like it's an
internal bug in a subroutine, when in fact it is actually due to bad
input. The experienced developer learns (by trial and error, possibly) to
ignore nearly half of the error message in this case.

In principle, the traceback could be roughly half as big, which means the
caller would find it half as difficult to read and understand:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib/python2.6/re.py", line 190, in compile
raise error, v # invalid expression
sre_constants.error: unbalanced parenthesis

With a one-to-one correspondence between the function called, and the
function reporting an error, it is easier to recognise that the error
lies in the input rather than some internal error in some subroutine you
have nothing to do with. Unfortunately there's no straightforward way to
consistently get this in Python without giving up the advantages of
delegating work to subroutines.

It need not be that way. This could, in principle, be left up to the
developer of the public function to specify (somehow!) that some specific
exceptions are expected, and should be treated as coming from public()
rather than from some internal subroutine. I don't have a concrete
proposal for such, although I did suggest a work-around. I expected
disinterest ("I don't see the point"). I didn't expect the level of
hostility to the idea that exceptions should (if and when possible) point
to the source of the error rather than some accidental implementation-
specific subroutine. Go figure.

 

[Ethan Furman]

...
>
I think critics of my position have forgotten what it's like to learning
the language. One of the most valuable skills to learn is to *ignore
parts of the traceback* -- a skill that takes practice and familiarity
with the library or function in use. To those who are less familiar with
the function, it can be very difficult to determine which parts of the
traceback are relevant and which aren't. In this case, the caller has
nothing to do with _compile, and the traceback looks like it's an
internal bug in a subroutine, when in fact it is actually due to bad
input. The experienced developer learns (by trial and error, possibly) to
ignore nearly half of the error message in this case.

And it can still be some work to figure out which parts of the traceback
are relevant, even after a couple years...

...
It need not be that way. This could, in principle, be left up to the
developer of the public function to specify (somehow!) that some specific
exceptions are expected, and should be treated as coming from public()
rather than from some internal subroutine. I don't have a concrete
proposal for such, although I did suggest a work-around. I expected
disinterest ("I don't see the point"). I didn't expect the level of
hostility to the idea that exceptions should (if and when possible) point
to the source of the error rather than some accidental implementation-
specific subroutine. Go figure.

My objection is not to the idea, but to the ad-hoc methods that would
currently be required. Resorting to passing exceptions in-band is a
step backwards. If python had a way to specify what level an exception
should be reported from, I might be interested. At this point, if
sparing the user one level of traceback was that high a priority to me,
I would make the validation be either a decorator, or have the
validation *be* the main routine, and the *real work* routine be the
private one.

~Ethan~

 

[Ethan Furman]

How is that any worse than making _pre_spam() a validation function that
returns a bool?

def spam(args):
flag = _pre_spam(args)
if flag: raise SomeException()
do_useful_work()

Also -1.

Is that also frowned upon for being too clever?

Frowned upon for being out-of-band, and not as much fun as being clever.
I'm pretty sure you've expressed similar sentiments in the past
(although my memory could be failing me).

More to the point, the OP had code that said:

args, kwargs = __pre_spam(*args, **kwargs)

and __pre_spam was either passing back verified (and possibly modified)
parameters, or raising an exception.

~Ethan~