Pythonic API design: detailed errors when you usually don't care

Discussion in 'Python' started by Simon Willison, Oct 2, 2006.

  1. Hi all,

    I have an API design question. I'm writing a function that can either
    succeed or fail. Most of the time the code calling the function won't
    care about the reason for the failure, but very occasionally it will.

    I can see a number of ways of doing this, but none of them feel
    aesthetically pleasing:

    1.

    try:
    do_something()
    except HttpError:
    # An HTTP error occurred
    except ApplicationError:
    # An application error occurred
    else:
    # It worked!

    This does the job fine, but has a couple of problems. The first is that
    I anticipate that most people using my function won't care about the
    reason; they'll just want a True or False answer. Their ideal API would
    look like this:

    if do_something():
    # It succeeded
    else:
    # It failed

    The second is that the common path is success, which is hidden away in
    the 'else' clause. This seems unintuitive.

    2.

    Put the method on an object, which stores the reason for a failure:

    if obj.do_something():
    # It succeeded
    else:
    # It failed; obj.get_error_reason() can be called if you want to know
    why

    This has an API that is closer to my ideal True/False, but requires me
    to maintain error state inside an object. I'd rather not keep extra
    state around if I don't absolutely have to.

    3.

    error = do_something()
    if error:
    # It failed
    else:
    # It succeeded

    This is nice and simple but suffers from cognitive dissonance in that
    the function returns True (or an object evaluating to True) for
    failure.

    4.

    The preferred approach works like this:

    if do_something():
    # Succeeded
    else:
    # Failed

    BUT this works too...

    ok = do_something()
    if ok:
    # Succeeded
    else:
    # ok.reason has extra information
    reason = ok.reason

    This can be implemented by returning an object from do_something() that
    has a __nonzero__ method that makes it evaluate to False. This solves
    my problem almost perfectly, but has the disadvantage that it operates
    counter to developer expectations (normally an object that evaluates to
    False is 'empty').

    I know I should probably just pick one of the above and run with it,
    but I thought I'd ask here to see if I've missed a more elegant
    solution.

    Thanks,

    Simon
    Simon Willison, Oct 2, 2006
    #1
    1. Advertising

  2. Simon Willison enlightened us with:
    > try:
    > do_something()
    > except HttpError:
    > # An HTTP error occurred
    > except ApplicationError:
    > # An application error occurred
    > else:
    > # It worked!
    >
    > This does the job fine, but has a couple of problems.


    > I anticipate that most people using my function won't care about the
    > reason; they'll just want a True or False answer.


    Then they can use an except clause that catches the superclass of all
    the possible exceptions your function raises.

    > Their ideal API would look like this:
    >
    > if do_something():
    > # It succeeded
    > else:
    > # It failed


    This is the C way of doing things. The problem with relying on a
    return value, is that failure could go unnoticed if the value isn't
    checked. This might introduce nasty bugs.

    Sybren
    --
    Sybren Stüvel
    Stüvel IT - http://www.stuvel.eu/
    Sybren Stuvel, Oct 2, 2006
    #2
    1. Advertising

  3. Simon  Willison

    Steve Holden Guest

    Simon Willison wrote:
    > Hi all,
    >
    > I have an API design question. I'm writing a function that can either
    > succeed or fail. Most of the time the code calling the function won't
    > care about the reason for the failure, but very occasionally it will.
    >
    > I can see a number of ways of doing this, but none of them feel
    > aesthetically pleasing:
    >
    > 1.
    >
    > try:
    > do_something()
    > except HttpError:
    > # An HTTP error occurred
    > except ApplicationError:
    > # An application error occurred
    > else:
    > # It worked!
    >
    > This does the job fine, but has a couple of problems. The first is that
    > I anticipate that most people using my function won't care about the
    > reason; they'll just want a True or False answer. Their ideal API would
    > look like this:
    >
    > if do_something():
    > # It succeeded
    > else:
    > # It failed
    >
    > The second is that the common path is success, which is hidden away in
    > the 'else' clause. This seems unintuitive.
    >
    > 2.
    >
    > Put the method on an object, which stores the reason for a failure:
    >
    > if obj.do_something():
    > # It succeeded
    > else:
    > # It failed; obj.get_error_reason() can be called if you want to know
    > why
    >
    > This has an API that is closer to my ideal True/False, but requires me
    > to maintain error state inside an object. I'd rather not keep extra
    > state around if I don't absolutely have to.
    >
    > 3.
    >
    > error = do_something()
    > if error:
    > # It failed
    > else:
    > # It succeeded
    >
    > This is nice and simple but suffers from cognitive dissonance in that
    > the function returns True (or an object evaluating to True) for
    > failure.
    >
    > 4.
    >
    > The preferred approach works like this:
    >
    > if do_something():
    > # Succeeded
    > else:
    > # Failed
    >
    > BUT this works too...
    >
    > ok = do_something()
    > if ok:
    > # Succeeded
    > else:
    > # ok.reason has extra information
    > reason = ok.reason
    >
    > This can be implemented by returning an object from do_something() that
    > has a __nonzero__ method that makes it evaluate to False. This solves
    > my problem almost perfectly, but has the disadvantage that it operates
    > counter to developer expectations (normally an object that evaluates to
    > False is 'empty').
    >
    > I know I should probably just pick one of the above and run with it,
    > but I thought I'd ask here to see if I've missed a more elegant
    > solution.
    >
    > Thanks,
    >
    > Simon
    >

    I should have thought the simplest spelling of your requirements would be

    try:
    do_something()
    print "It worked"
    except:
    # it didn't

    However there are good reasons why this pattern isn't often recommended:
    it masks types of exception that you may not have anticipated.

    regards
    Steve
    --
    Steve Holden +44 150 684 7255 +1 800 494 3119
    Holden Web LLC/Ltd http://www.holdenweb.com
    Skype: holdenweb http://holdenweb.blogspot.com
    Recent Ramblings http://del.icio.us/steve.holden
    Steve Holden, Oct 2, 2006
    #3
  4. Simon  Willison

    Carl Banks Guest

    Simon Willison wrote:
    > I have an API design question. I'm writing a function that can either
    > succeed or fail. Most of the time the code calling the function won't
    > care about the reason for the failure, but very occasionally it will.
    >
    > I can see a number of ways of doing this, but none of them feel
    > aesthetically pleasing:

    [snip]

    Whether it's aesthetically pleasing or not, raising an exception is the
    way to do it.

    You seem to care about your users. Keep this in mind: if your function
    returns an error code, then it puts an burden on users to A. always
    check for errors, and B. always check for errors right at the point
    where the function is called.

    Python has exceptions so that programmers can write code without
    checking for an error in every other line like they would have to in C.
    With exceptions, users are free to not bother checking for errors, if
    program termination is acceptable, or to handle the exception somewhere
    else. Please don't circumvent this on the misguided pretense of
    "helping" the user.

    If you really think most users would prefer to check the return value
    for an error, then I'd recommend two versions of the function: the
    regular, Pythonic version that raises exceptions, and the convenience
    version that swallows the exception and returns a code. (Compare a[1]
    vs. a.get(1).)


    > This is nice and simple but suffers from cognitive dissonance in that
    > the function returns True (or an object evaluating to True) for
    > failure.


    BTW, this is true in a lot of places. Back in the days of DOS, zero
    meant success, non-zero was an error code. Zero also signals success
    for process exit codes on Unix. Also on Unix, negative values are
    often considered errors, non-negative success. The idea that error ==
    0 is not self-evident, and often not true.

    If it helps, consider using string error codes, or defining some
    constants to check against ("if error == NO_ERROR").


    Carl Banks
    Carl Banks, Oct 2, 2006
    #4
  5. Simon  Willison

    Steve Holden Guest

    Steve Holden wrote:
    > Simon Willison wrote:
    >
    >>Hi all,
    >>
    >>I have an API design question. I'm writing a function that can either
    >>succeed or fail. Most of the time the code calling the function won't
    >>care about the reason for the failure, but very occasionally it will.
    >>
    >>I can see a number of ways of doing this, but none of them feel
    >>aesthetically pleasing:
    >>
    >>1.
    >>
    >>try:
    >> do_something()
    >>except HttpError:
    >> # An HTTP error occurred
    >>except ApplicationError:
    >> # An application error occurred
    >>else:
    >> # It worked!
    >>
    >>This does the job fine, but has a couple of problems. The first is that
    >>I anticipate that most people using my function won't care about the
    >>reason; they'll just want a True or False answer. Their ideal API would
    >>look like this:
    >>
    >>if do_something():
    >> # It succeeded
    >>else:
    >> # It failed
    >>
    >>The second is that the common path is success, which is hidden away in
    >>the 'else' clause. This seems unintuitive.
    >>
    >>2.
    >>
    >>Put the method on an object, which stores the reason for a failure:
    >>
    >>if obj.do_something():
    >> # It succeeded
    >>else:
    >> # It failed; obj.get_error_reason() can be called if you want to know
    >>why
    >>
    >>This has an API that is closer to my ideal True/False, but requires me
    >>to maintain error state inside an object. I'd rather not keep extra
    >>state around if I don't absolutely have to.
    >>
    >>3.
    >>
    >>error = do_something()
    >>if error:
    >> # It failed
    >>else:
    >> # It succeeded
    >>
    >>This is nice and simple but suffers from cognitive dissonance in that
    >>the function returns True (or an object evaluating to True) for
    >>failure.
    >>
    >>4.
    >>
    >>The preferred approach works like this:
    >>
    >>if do_something():
    >> # Succeeded
    >>else:
    >> # Failed
    >>
    >>BUT this works too...
    >>
    >>ok = do_something()
    >>if ok:
    >> # Succeeded
    >>else:
    >> # ok.reason has extra information
    >> reason = ok.reason
    >>
    >>This can be implemented by returning an object from do_something() that
    >>has a __nonzero__ method that makes it evaluate to False. This solves
    >>my problem almost perfectly, but has the disadvantage that it operates
    >>counter to developer expectations (normally an object that evaluates to
    >>False is 'empty').
    >>
    >>I know I should probably just pick one of the above and run with it,
    >>but I thought I'd ask here to see if I've missed a more elegant
    >>solution.
    >>
    >>Thanks,
    >>
    >>Simon
    >>

    >
    > I should have thought the simplest spelling of your requirements would be
    >
    > try:
    > do_something()
    > print "It worked"
    > except:
    > # it didn't
    >
    > However there are good reasons why this pattern isn't often recommended:
    > it masks types of exception that you may not have anticipated.
    >

    The recipe for only catching known errors but giving them all the same
    response would be

    try:
    do_something()
    print "It worked!"
    except (ThisError, ThatError, TheOtherError):
    print "It failed!"

    regards
    Steve
    --
    Steve Holden +44 150 684 7255 +1 800 494 3119
    Holden Web LLC/Ltd http://www.holdenweb.com
    Skype: holdenweb http://holdenweb.blogspot.com
    Recent Ramblings http://del.icio.us/steve.holden
    Steve Holden, Oct 2, 2006
    #5
  6. Simon  Willison

    Paul Rubin Guest

    Steve Holden <> writes:
    > I should have thought the simplest spelling of your requirements would be
    >
    > try:
    > do_something()
    > print "It worked"
    > except:
    > # it didn't


    This usage is generally disparaged because of the possibility of
    exceptions having nothing to do with do_something, including
    asynchronous exceptions like KeyboardInterrupt, that really should get
    handed up to higher levels of the program.

    An approach not mentioned is for do_something to arrange that its
    "expected" exceptions would all belong to subclasses of some single
    class, so you could catch all those exceptions at once:

    try: do_something()
    except SomethingException:
    # it didn't work

    or alternatively you could break out the individual subclasses:

    try: do_something()
    except SomethingHTTPException:
    # it didn't work for this reason
    except SomethingWhatsitException:
    # or that one

    Or alternatively you could just put everything in one exception
    with the underlying exception in the returned arg:

    try: do_something()
    except SomethingException, e:
    # it didn't work, e gives the reason
    ...

    def do_something():
    try:
    do_something_1() # do the real something
    except (HTTPError, ApplicationError, ...), e:
    # pass detailed exception info up to the caller
    raise SomethingException, (sys.exc_info(), e)
    Paul Rubin, Oct 2, 2006
    #6
  7. Simon  Willison

    Aahz Guest

    In article <>,
    Simon Willison <> wrote:
    >
    >try:
    > do_something()
    >except HttpError:
    > # An HTTP error occurred
    >except ApplicationError:
    > # An application error occurred
    >else:
    > # It worked!


    Use a similar but different idiom:

    try:
    do_something()
    except DoSomethingError:
    # handle error

    IOW, your do_something() function maps all expected errors to
    DoSomethingError (or at least to a series of related exceptions that are
    all subclasses of DoSomethingError).
    --
    Aahz () <*> http://www.pythoncraft.com/

    "LL YR VWL R BLNG T S" -- www.nancybuttons.com
    Aahz, Oct 2, 2006
    #7
  8. Simon  Willison

    Kay Schluehr Guest

    Simon Willison wrote:
    > Hi all,
    >
    > I have an API design question. I'm writing a function that can either
    > succeed or fail. Most of the time the code calling the function won't
    > care about the reason for the failure, but very occasionally it will.
    >
    > I can see a number of ways of doing this, but none of them feel
    > aesthetically pleasing:
    >
    > 1.
    >
    > try:
    > do_something()
    > except HttpError:
    > # An HTTP error occurred
    > except ApplicationError:
    > # An application error occurred
    > else:
    > # It worked!
    >
    > This does the job fine, but has a couple of problems. The first is that
    > I anticipate that most people using my function won't care about the
    > reason; they'll just want a True or False answer. Their ideal API would
    > look like this:
    >
    > if do_something():
    > # It succeeded
    > else:
    > # It failed
    >
    > The second is that the common path is success, which is hidden away in
    > the 'else' clause. This seems unintuitive.
    >
    > 2.
    >
    > Put the method on an object, which stores the reason for a failure:
    >
    > if obj.do_something():
    > # It succeeded
    > else:
    > # It failed; obj.get_error_reason() can be called if you want to know
    > why
    >
    > This has an API that is closer to my ideal True/False, but requires me
    > to maintain error state inside an object. I'd rather not keep extra
    > state around if I don't absolutely have to.
    >
    > 3.
    >
    > error = do_something()
    > if error:
    > # It failed
    > else:
    > # It succeeded
    >
    > This is nice and simple but suffers from cognitive dissonance in that
    > the function returns True (or an object evaluating to True) for
    > failure.
    >
    > 4.
    >
    > The preferred approach works like this:
    >
    > if do_something():
    > # Succeeded
    > else:
    > # Failed
    >
    > BUT this works too...
    >
    > ok = do_something()
    > if ok:
    > # Succeeded
    > else:
    > # ok.reason has extra information
    > reason = ok.reason
    >
    > This can be implemented by returning an object from do_something() that
    > has a __nonzero__ method that makes it evaluate to False. This solves
    > my problem almost perfectly, but has the disadvantage that it operates
    > counter to developer expectations (normally an object that evaluates to
    > False is 'empty').
    >
    > I know I should probably just pick one of the above and run with it,
    > but I thought I'd ask here to see if I've missed a more elegant
    > solution.
    >
    > Thanks,
    >
    > Simon


    As I see it this can become an architectural concern. If callers care
    "occasionally" which precision or care is actually required? Is it just
    that a developer wants to see a stack-trace for debugging purposes?
    Here logging might be adaequate while otherwise if application logics
    is dependent it might be helpfull to define a contract between caller
    and callee. This could be a handler that is passed by the caller to
    do_something() that must be aware of it by defining an optional
    parameter in its interface. This is definitely framework overhead and
    although I wouldn't qualify it as "ugly" it makes your program more
    complex and dependent on more components. Finally you have to balance
    the tradeoff. All other approaches I've seen here tried to tame the
    effect of throwing exceptions that were only needed occasionally. It's
    like dealing with unwanted checked exceptions in Java...
    Kay Schluehr, Oct 2, 2006
    #8
  9. Simon  Willison

    John J. Lee Guest

    "Simon Willison" <> writes:

    > Hi all,
    >
    > I have an API design question. I'm writing a function that can either
    > succeed or fail. Most of the time the code calling the function won't
    > care about the reason for the failure, but very occasionally it will.
    >
    > I can see a number of ways of doing this, but none of them feel
    > aesthetically pleasing:
    >
    > 1.
    >
    > try:
    > do_something()
    > except HttpError:
    > # An HTTP error occurred
    > except ApplicationError:
    > # An application error occurred
    > else:
    > # It worked!
    >
    > This does the job fine, but has a couple of problems. The first is that
    > I anticipate that most people using my function won't care about the
    > reason; they'll just want a True or False answer. Their ideal API would
    > look like this:
    >
    > if do_something():
    > # It succeeded
    > else:
    > # It failed
    >
    > The second is that the common path is success, which is hidden away in
    > the 'else' clause. This seems unintuitive.


    It's hard to answer this without knowing any of the real details
    you've hidden from us: I'm not sure I believe you that the code at the
    call site necessarily needs to know whether the function call
    succeeded at all. Perhaps the preferred usage might sometimes be:

    do_something()


    Exceptions raised may be caught (or not, if the application does not
    find that necessary), elsewhere than at the call site -- they can be
    caught further up the stack. You are of course aware of this fact,
    but your discussion seems to imply that this crucial piece of
    knowledge was not active in your mind when you wrote it.

    I think Python users are right to "default to" raising exceptions.
    There's a nice label for this practice:

    http://c2.com/cgi/wiki?SamuraiPrinciple

    """
    Samurai Principle

    Return victorious, or not at all.
    """


    :)

    (though I imagine that term is used with subtly different meanings by
    different people...)


    As others have suggested, often it is useful if most exceptions that
    can be raised by a callable are instances of a single class. This can
    be achieved through inheritance (make them all derive from one class)
    or composition (stuff the various original exceptions into a wrapper
    exception).


    John
    John J. Lee, Oct 3, 2006
    #9
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Gregory Huffman

    How to include don't care minterms

    Gregory Huffman, Jan 26, 2004, in forum: VHDL
    Replies:
    3
    Views:
    7,407
    Gregory Huffman
    Jan 28, 2004
  2. SpamProof
    Replies:
    3
    Views:
    636
    SpamProof
    Dec 1, 2003
  3. puzzlecracker
    Replies:
    5
    Views:
    296
    Jorgen Grahn
    Aug 22, 2008
  4. Jason
    Replies:
    0
    Views:
    178
    Jason
    Jul 6, 2004
  5. Anonieko
    Replies:
    0
    Views:
    94
    Anonieko
    Jan 17, 2006
Loading...

Share This Page