What c.l.py's opinions about Soft Exception?

[Lie]

    I actually implemented something like "soft exceptions" in a LISP
program long ago. I called them "gripes".  They were a way that a function
could complain about something it wanted the caller to fix. The caller
could either fix it or decline to do so.

    This was for a robotic planning application, where one module had detected
that some precondition that it needed wasn't satisfied.  This was usually
something like some physical object being in the wrong place for a later
operation, and a previously planned move needed to be modified. Passing
the problem back to the caller sometimes allowed an easy fix, rather than
aborting the whole plan and building a new one.

    This isn't a common problem.

    In the rare cases that it is needed, it can be implemented with callbacks.
It doesn't require a language extension.

                                        John Nagle

The problem with callbacks is that it works only for a small amount of
callbacks, it'd be too messy to have twenty different callbacks.
And the ultimate problem with callbacks is that we can't determine
from the outside whether the operation should continue or breaks at
the point of the callback without some messy trick.

We can't always determine whether we want to do this:
def somefunc(argA, argB, callback = DO_NOTHING):
if argA == argB:
callback()

or this:
def somefunc(argA, argB, callback = DO_NOTHING):
if argA == argB:
callback()
return

perhaps we could do this:
if argA == argB:
if callback() == True: return

but that's so much extra codes written and would've been too messy to
write.

And actually this isn't a rare cases, and in fact is very common. It's
just that most cases that can be more cleanly written in
SoftException, can usually be handled in different ways, using tricks
that range from subtle to messy (like callbacks).

 

[metawilm]

Is this soft-exception implemented anywhere, so that one can see what
experiences and best practices have evolved around using it?

Lie's idea is to separate exceptions in two groups, those that must be
handled and those that don't. A better way is to have two different
ways to raise exceptions: one exceptional situation can be "Hard" in
some situations and "Soft" in others, and it is up to the way of
raising to make the choice, while the exception stays the same.

Common Lisp has two ways of raising: functions "error" and "signal".
Python's "raise" is like CL's "error": you end up in the debugger if
the exception is not handled. Exceptions that are raised by CL's
"signal" don't have to be caught: if there is no matching "except"
clause the raise statement becomes a "pass".

Or as Wikipedia states nicely: "Conditions are a generalization of
exceptions. When a condition arises, an appropriate condition handler
is searched for and selected, in stack order, to handle the condition.
Conditions which do not represent errors may safely go unhandled
entirely; their only purpose may be to propagate hints or warnings
toward the user."

http://en.wikipedia.org/wiki/Exception_handling#Condition_systems


- Willem

 

[Diez B. Roggisch]

The problem with callbacks is that it works only for a small amount of

callbacks, it'd be too messy to have twenty different callbacks.
And the ultimate problem with callbacks is that we can't determine
from the outside whether the operation should continue or breaks at
the point of the callback without some messy trick.

We can't always determine whether we want to do this:
def somefunc(argA, argB, callback = DO_NOTHING):
if argA == argB:
callback()

or this:
def somefunc(argA, argB, callback = DO_NOTHING):
if argA == argB:
callback()
return

perhaps we could do this:
if argA == argB:
if callback() == True: return

but that's so much extra codes written and would've been too messy to
write.

And actually this isn't a rare cases, and in fact is very common. It's
just that most cases that can be more cleanly written in
SoftException, can usually be handled in different ways, using tricks
that range from subtle to messy (like callbacks).

I fail to see how your arguments follow.

Regarding the number of callbacks: you can as well pass an object that
has several methods to call.

And the above example can easily be accomplished with "normal"
exceptions, like this:


def toplelevel():
def callback(a, b):
if a == b:
raise InterruptException()
try:
work(callback)
except InterruptException:
pass

def work(callback=callback):
a = 10
b = 20
callback(a, b)

And even more: the callback-approach can do things like this:

a, b = callback(a,b)

to change values, which makes it superior to SoftExceptions - unless I
missed it's ability to capture context.

So far, you haven't shown a very convincing example of what
SoftException are and how they can be useful.

Diez

 

[Diez B. Roggisch]

Lie's idea is to separate exceptions in two groups, those that must be
handled and those that don't. A better way is to have two different
ways to raise exceptions: one exceptional situation can be "Hard" in
some situations and "Soft" in others, and it is up to the way of
raising to make the choice, while the exception stays the same.

Common Lisp has two ways of raising: functions "error" and "signal".
Python's "raise" is like CL's "error": you end up in the debugger if
the exception is not handled. Exceptions that are raised by CL's
"signal" don't have to be caught: if there is no matching "except"
clause the raise statement becomes a "pass".

Or as Wikipedia states nicely: "Conditions are a generalization of
exceptions. When a condition arises, an appropriate condition handler
is searched for and selected, in stack order, to handle the condition.
Conditions which do not represent errors may safely go unhandled
entirely; their only purpose may be to propagate hints or warnings
toward the user."

How would that differ from something like this:


with add_soft_handler(SoftException):
invoke_something()

.... # deep down in code

raise_soft(SoftException())


The implementation of add_soft_handler and raise_soft is trivial - a bit
of thread-local storage and a stack.

I don't say that such mechanism isn't handy occasionally - I just don't
see the need for specialized syntactical and/or interpreter-support.

Diez

 

[Chris]

If all you wanted was some grouping of exceptions why not something
like...

soft_exception_list = [IndexError, TypeError]
hard_exception_list = [ZeroDivision]

try:
do_something()
except Exception, e:
if e.__class__ in soft_exception_list:
handle_soft_exception()
elif e.__class__ in hard_exception_list:
handle_hard_exception()
else:
raise NotImplementedError

Granted you're most likely looking for something that does this
constantly on every line of code though...

 

[Diez B. Roggisch]

If all you wanted was some grouping of exceptions why not something
like...

soft_exception_list = [IndexError, TypeError]
hard_exception_list = [ZeroDivision]

try:
do_something()
except Exception, e:
if e.__class__ in soft_exception_list:
handle_soft_exception()
elif e.__class__ in hard_exception_list:
handle_hard_exception()
else:
raise NotImplementedError

Granted you're most likely looking for something that does this
constantly on every line of code though...

It's not about grouping, which would be done better with inheritance by
the way.

Its about not raising certain exceptions if there is no handler.

Diez

 

[Lie]

I fail to see how your arguments follow.

Regarding the number of callbacks: you can as well pass an object that
has several methods to call.

If you passed an object that has several methods to call (using tuple
or list) and you want to handle several softexceptions and ignore some
others, you must still pass an empty methods to the one you want to
ignore, cluttering the caller's code by significant degree:

def somefunc(a, b, callback = (DO_NOTHING, DO_NOTHING, DO_NOTHING,
DO_NOTHING)):
if a == 0: raise callback(0)
try:
a += b
except ZeroDivisionError:
raise callback(1)
if a <= 0: raise callback(2)
raise callback(3)
return a

somefunc(a, b, (callbackzero, DO_NOTHING, callbacktwo,
DO_NOTHING))

if instead we use dict, well, we know how <inverse>convenient</
inverse> dict's syntax is for a lot of manual data entry.

## imagine if we want to handle five or more callbacks
somefunc(a, b, {callbackzero:handlerzero,
callbacktwo:handlertwo})

And the above example can easily be accomplished with "normal"
exceptions, like this:

def toplelevel():
     def callback(a, b):
         if a == b:
            raise InterruptException()
     try:
          work(callback)
     except InterruptException:
          pass

def work(callback=callback):
     a = 10
     b = 20
     callback(a, b)

That's why I said most things that can be more cleanly handled by
SoftException can usually be handled in other forms, although in a
more cluttered way.

That could be more cleanly written in SoftException as:
def work():
a = 10
b = 20
raise Equal(a, b)

def toplevel():
try:
work()
except Equal, args:
a, b = args
if a == b:
raise InterruptException

OR ALTERNATIVELY (Which one's better depends on the purpose, generally
the one below is better, but the one in the above is more flexible,
yet a bit less convenient to use)

def work():
a = 10
b = 20
if a == b: raise Equal

def toplevel():
try:
work()
except Equal:
raise InterruptException

The reason why SoftException is useful is similar to the reason why
for-loop and while <condition> is useful. AFAIK, all looping cases can
be handled only by a blind loopers (a.k.a. while True and break, but
a blind loopers is very inconvenient to use, and for-loop and while

And even more: the callback-approach can do things like this:

a, b = callback(a,b)

to change values, which makes it superior to SoftExceptions - unless I
missed it's ability to capture context.

If there is a syntax support, you could also make "resume" able to
transfer values:

def somefunc(a, b):
if a == b: a, b = raise Equal(a, b)

def toplevel():
try:
somefunc(10, 20)
except Equal, args:
a, b = args[0], args[1] + 1
resume a, b

How would that differ from something like this:

with add_soft_handler(SoftException):
invoke_something()

... # deep down in code

raise_soft(SoftException())

The implementation of add_soft_handler and raise_soft is trivial - a bit
of thread-local storage and a stack.

Perhaps you meant:
raise_soft(SoftException)

cause SoftException() may have side effects if called greedily

That could be done, but when raise_soft() returns, it returns to the
code that raises it so it must be 'break'en:

def caller(a, b):
if a == b:
raise_soft(SoftException)
break

but this makes SoftException must break everytime, which make it lost
its original purpose, so you have to add return code to raise_soft
whether you want to break it or not:

def caller(a, b):
if a == b:
if raise_soft(SoftException):
break

Compare to:
def caller(a, b):
if a == b:
raise SoftException

And this also makes it impossible to have state-changing behavior
without some other weirder tricks

 

[Diez B. Roggisch]

If you passed an object that has several methods to call (using tuple
or list) and you want to handle several softexceptions and ignore some
others, you must still pass an empty methods to the one you want to
ignore, cluttering the caller's code by significant degree:

def somefunc(a, b, callback = (DO_NOTHING, DO_NOTHING, DO_NOTHING,
DO_NOTHING)):
if a == 0: raise callback(0)
try:
a += b
except ZeroDivisionError:
raise callback(1)
if a <= 0: raise callback(2)
raise callback(3)
return a

somefunc(a, b, (callbackzero, DO_NOTHING, callbacktwo,
DO_NOTHING))

You misunderstood. I'd pass something like a context-object, which wold look
like this:

def somefunc(a, b, context=NullContext()):
if a == b: context.a_equals_b()
....

Not more clutter than with only one callback. And the NullContext-object
would actually serve as documentation on what events the code produces.

That's why I said most things that can be more cleanly handled by
SoftException can usually be handled in other forms, although in a
more cluttered way.

That could be more cleanly written in SoftException as:
def work():
a = 10
b = 20
raise Equal(a, b)

def toplevel():
try:
work()
except Equal, args:
a, b = args
if a == b:
raise InterruptException

I totally fail to see where

raise Equal(a, b)

is less cluttered or not than

callback(a, b)

Actually, the latter is even less cluttered, misses a raise - if pure number
of literals is your metric, that is.

If there is a syntax support, you could also make "resume" able to
transfer values:

def somefunc(a, b):
if a == b: a, b = raise Equal(a, b)

def toplevel():
try:
somefunc(10, 20)
except Equal, args:
a, b = args[0], args[1] + 1
resume a, b

Sure, you can make all kinds of things, but so far you didn't come up with a
comprehensive feature description that just _does_ specify what SEs are and
what not.

Perhaps you meant:
raise_soft(SoftException)

cause SoftException() may have side effects if called greedily

Nope, I didn't, and it's beside the point.

That could be done, but when raise_soft() returns, it returns to the
code that raises it so it must be 'break'en:

def caller(a, b):
if a == b:
raise_soft(SoftException)
break

but this makes SoftException must break everytime, which make it lost
its original purpose, so you have to add return code to raise_soft
whether you want to break it or not:

You didn't understand my example. If there is a handler registered, it will
be invoked. If not, nothing will be raised. The exact same amount of
state-keeping and lookups needs to be done by the SE-implementation.

That could be done, but when raise_soft() returns, it returns to the
code that raises it so it must be 'break'en:
def caller(a, b):
if a == b:
if raise_soft(SoftException):
break

Compare to:
def caller(a, b):
if a == b:
raise SoftException

And this also makes it impossible to have state-changing behavior
without some other weirder tricks

That's not true. The

with add_soft_handler(SoftException, handler):

approach (I missed the handrel the first time, sorry)
can easily throw an exception to interrupt, like this:

def handler(e):
if some_condition_on_e(e):
raise InterruptException()

with add_soft_handler(SoftException, handler):
try:
work(...)
except InterruptException:
pass

You could also introduce a function

def interruptable(fun, *args, **kwargs):
try:
return fun(*args, **kwargs)
except InterruptException:
pass

to make the code look a bit cleaner - if it fits your usecase, that is of
course.

I don't say that SoftExceptions can't have semantics that go beyond this. I
just don't see a oh-so-compelling use-case that makes things so much better
than they are reachable now, without actually much programming.

Diez

 

[Steven D'Aprano]

If you passed an object that has several methods to call (using tuple or
list) and you want to handle several softexceptions and ignore some
others, you must still pass an empty methods to the one you want to
ignore, cluttering the caller's code by significant degree:

def somefunc(a, b, callback = (DO_NOTHING, DO_NOTHING, DO_NOTHING,
DO_NOTHING)):
if a == 0: raise callback(0)
try:
a += b
except ZeroDivisionError:
raise callback(1)
if a <= 0: raise callback(2)
raise callback(3)
return a

somefunc(a, b, (callbackzero, DO_NOTHING, callbacktwo,
DO_NOTHING))

Yes, you are right that this is a bad idea. But it is a bad idea
regardless of whether you use callbacks or SoftExceptions.

In your example above, you seem to have accidentally written "raise
callback" when you (probably) meant to just call the callback. That's
significant because it shows that replacing the callback is still just as
cluttered, and still puts a large burden on somefunc() to perform every
test that callers might want to perform.

This is a bad idea because you are tightly coupling somefunc() to the
specific needs of some arbitrary callers. You should aim to have loose
coupling between functions, not tight. Tight coupling should be avoided,
not encouraged.

In most circumstances, the right place to put the various tests is in the
caller, not in somefunc().

 

[Steven D'Aprano]

Common Lisp has two ways of raising: functions "error" and "signal".
Python's "raise" is like CL's "error": you end up in the debugger if the
exception is not handled. Exceptions that are raised by CL's "signal"
don't have to be caught: if there is no matching "except" clause the
raise statement becomes a "pass".

Or as Wikipedia states nicely: "Conditions are a generalization of
exceptions. When a condition arises, an appropriate condition handler is
searched for and selected, in stack order, to handle the condition.
Conditions which do not represent errors may safely go unhandled
entirely; their only purpose may be to propagate hints or warnings
toward the user."

http://en.wikipedia.org/wiki/Exception_handling#Condition_systems

If I had come across "signals" before now, I would have thought that they
were a good idea.

But after watching Lie repeatedly argue for tightly coupling functions
together using signal-like "SoftExceptions", all I can see are the
disadvantages.

I'm afraid that if Lie's arguments are the best available for such a
signaling mechanism, then it's probably a good thing Python doesn't have
it.

 

[Lie]

(If there is anything weird that I say, please ignore it since I'm
writing this half-sleeping)

I totally fail to see where

raise Equal(a, b)

is less cluttered or not than

callback(a, b)

Actually, the latter is even less cluttered, misses a raise - if pure number
of literals is your metric, that is.

You don't just compare by the calling code, you've got to compare also
by the surrounding codes. The calling codes in SE might be a little
bit messy, but it worths by making the code surrounding it more clean
and structured. And anyway, if you used Context Object callback, they
will be as messy as each other.

If there is a syntax support, you could also make "resume" able to
transfer values:

    def somefunc(a, b):
        if a == b: a, b = raise Equal(a, b)

    def toplevel():
        try:
            somefunc(10, 20)
        except Equal, args:
            a, b = args[0], args[1] + 1
            resume a, b

Sure, you can make all kinds of things, but so far you didn't come up with a
comprehensive feature description that just _does_ specify what SEs are and
what not.

- Exception that aren't handled when no handler exists for it.
- It's not a way for notifying errors
- It's a way to describe status changes to higher codes
- Everything described in the first post

Nope, I didn't, and it's beside the point.

Then what happen when SoftException is called? And a side-effect
occurs?

You didn't understand my example. If there is a handler registered, it will
be invoked. If not, nothing will be raised. The exact same amount of
state-keeping and lookups needs to be done by the SE-implementation.

I do understand your example. And even if I misunderstood you about
passing context object, the same problem still exists in context
object based solution, i.e. functions can't make the called function
break automatically, it must be 'break' manually or the program will
go astray (break ==> return, sorry I confused it with break for
loops). And if you used InterruptException to break, it doesn't play
well with multiple SoftExceptions.

The final, resulting code by function passing below is extremely
messy, see if you can make it cleaner and with the same
functionalities and all to the SE version.

def called(a, b, cont_obj = Null_CO):
if a == b:
a, b = cont_obj.a_equal_b(a, b)
cont_obj.addition(a, b)
return a + b

def caller():
class cont_obj(object):
def a_equal_b(a, b):
if a < 0 and b < 0:
return a + 1, b # resume
raise InterruptException(('a_e_b',)) # break

def addition(a, b):
if a > b:
return
raise InterruptException(('addit', (a, b))) # break

try:
called(10, 10, cont_obj)
except InterruptException, args: # if breaken
ret, arg = args
if ret == 'a_e_b': return -1
a, b = arg
if ret == 'addit': return a ** b


# by adding try clauses, and you've really equalize the biggest
overhead of SE.
# And I don't think you could create a less messy InterruptException
handler,
# the other solution to it would be to create a handler for each
unique returns
# but that would make it exceed the second SE overhead, the Exception
Declaration
# In other words, the tricks that is used to emulate the SoftException
would all
# have higher code overhead compared to using the clean, structured
SEs

# * Overheads means garbage code that's important to make something
work

# The code is separated into three parts, "try except", and cont_obj,
and called. Worse, the cont_obj can't determine what happen if they
got unresumed errors, without using some tricky part.

Compare that code above with:

def called(a, b):
if a == b:
a, b = raise a_equal_b(a, b)
raise addition(a, b)
return a + b

def caller():
class a_equal_b(Exception): pass
class addition(Exception): pass

try:
ret = called(10, 10)
except a_equal_b(a, b):
if a < 0 and b < 0:
resume a + 1, b
return -1
except addition(a, b):
if a > b: resume
return a ** b

# The code is separated into two parts, the "trys and excepts" and the
called code.

That's not true. The

with add_soft_handler(SoftException, handler):

approach (I missed the handrel the first time, sorry)
can easily throw an exception to interrupt, like this:

def handler(e):
    if some_condition_on_e(e):
       raise InterruptException()

with add_soft_handler(SoftException, handler):
     try:
          work(...)
     except InterruptException:
          pass

You could also introduce a function

def interruptable(fun, *args, **kwargs):
    try:
       return fun(*args, **kwargs)
    except InterruptException:
       passthe

to make the code look a bit cleaner - if it fits your usecase, that is of
course.

The code doesn't work well with multiple excepts that have multiple
fallbacks.

 

[castironpi]

You don't just compare by the calling code, you've got to compare also
by the surrounding codes. The calling codes in SE might be a little
bit messy, but it worths by making the code surrounding it more clean
and structured. And anyway, if you used Context Object callback, they
will be as messy as each other.


I do understand your example. And even if I misunderstood you about
passing context object, the same problem still exists in context
object based solution, i.e. functions can't make the called function
break automatically, it must be 'break' manually or the program will
go astray (break ==> return, sorry I confused it with break for
loops). And if you used InterruptException to break, it doesn't play
well with multiple SoftExceptions.

Are you interested in yielding an object? In situ, a routine has
extra information it wants to return to the caller. Can you protocol
a number of yields? Discard x for x in operation()[:3]; x=
operation(). Or, x= operation()[3], and go back and look operation()
[1], which is cached.