Python Code Auditing Tool

Discussion in 'Python' started by Robey Holderith, Feb 2, 2005.

  1. Does anybody know of a tool that can tell me all possible exceptions that
    might occur in each line of code? What I'm hoping to find is something
    like the following:

    given all necessary python source and a given line ( my.py:40 ) it would
    generate a list of possible exception classes sorted by function
    (preferably in a tree).

    Example:
    ------------------

    my.py:40 | parsestring(genstring())

    Possible Exceptions:

    -def parsestring():
    InvalidCharacterException
    EmptyStringException
    -class string, def split():
    (All Exceptions that might occur directly in string.split() that are
    not caught by parsestring())
    (All functions called by string.split() and their exceptions and sub-
    functions)
    -def genstring():
    SomeException
    ...

    --------------------

    This would be extremely useful for deciding how to write try: except
    blocks and in figuring out what all possible errors that might occur would
    be.

    -Robey Holderith
     
    Robey Holderith, Feb 2, 2005
    #1
    1. Advertising

  2. Robey Holderith

    Paul Rubin Guest

    Robey Holderith <> writes:
    > Does anybody know of a tool that can tell me all possible exceptions that
    > might occur in each line of code? What I'm hoping to find is something
    > like the following:


    That is impossible. The parameter to the raise statement is a class
    object, which can be anything. I.e. you could say:

    class ex1: pass
    class ex2: pass

    if something(): my_ex = ex1
    else: my_ex = ex2

    raise my_ex # static tool can't know what exception gets raised here.
     
    Paul Rubin, Feb 2, 2005
    #2
    1. Advertising

  3. On Tue, 01 Feb 2005 21:52:28 -0800, Paul Rubin wrote:

    > Robey Holderith <> writes:
    >> Does anybody know of a tool that can tell me all possible exceptions that
    >> might occur in each line of code? What I'm hoping to find is something
    >> like the following:

    >
    > That is impossible. The parameter to the raise statement is a class
    > object, which can be anything. I.e. you could say:
    >
    > class ex1: pass
    > class ex2: pass
    >
    > if something(): my_ex = ex1
    > else: my_ex = ex2
    >
    > raise my_ex # static tool can't know what exception gets raised here.


    I suppose that I am willing to lessen my expectations from _all_ to most.
    ;-) Regarding your example I could also do:

    if something():
    def nothing(): return 0
    else:
    def nothing(): return 1

    But this doesn't stop IDEs from attempting to do auto-completion. I'm not
    trying to find hidden exceptions... just trying to easily get an idea of
    what could go wrong on each line of code.

    -Robey Holderith
     
    Robey Holderith, Feb 2, 2005
    #3
  4. > I suppose that I am willing to lessen my expectations from _all_ to most.
    > ;-) Regarding your example I could also do:
    >
    > if something():
    > def nothing(): return 0
    > else:
    > def nothing(): return 1
    >
    > But this doesn't stop IDEs from attempting to do auto-completion. I'm not
    > trying to find hidden exceptions... just trying to easily get an idea of
    > what could go wrong on each line of code.


    There is AFAIK only one language that this can de accomplished - java, and
    that's because of these checked exceptions of theirs. But checked
    exceptions are considered harmful:

    http://www.gcek.net/ref/books/sw/ooad/tip/#_Toc41169682

    I totally agree with that - in java, I tend to throw SystemExceptions to rid
    myself of endless try/catch clauses that obscure the real problem.

    So - there is no way of knowing this. The only thing I can think of is to
    keep some docs around that specify what exceptions to be expected, and that
    tool of yours could try and see if it can identify a function/method by
    name and notify you of the possible exceptions thrown. Might actually work
    out quite well for the standardlib, if one does the work for annotating all
    functions/methods properly.

    --
    Regards,

    Diez B. Roggisch
     
    Diez B. Roggisch, Feb 2, 2005
    #4
  5. Robey Holderith

    Neil Benn Guest

    Diez B. Roggisch wrote:

    >>I suppose that I am willing to lessen my expectations from _all_ to most.
    >>;-) Regarding your example I could also do:
    >>
    >><
    >>
    >>

    <snip>

    >>There is AFAIK only one language that this can de accomplished - java, and
    >>that's because of these checked exceptions of theirs. But checked
    >>exceptions are considered harmful:
    >>
    >>http://www.gcek.net/ref/books/sw/ooad/tip/#_Toc41169682
    >>
    >>I totally agree with that - in java, I tend to throw SystemExceptions to rid
    >>myself of endless try/catch clauses that obscure the real problem.
    >>
    >>
    >>

    <snip>
    Hello,

    I'm afraid that the only reliable way to gather what exceptions are
    raised is to read docs and/or come up with test cases. This has been a
    bugbear of mine in Python as it's not common to find a nice :Exceptions:
    IOError <desc>, IllegalArgumentError <desc> type of description in the docs.

    However if you want an incomplete test, you could parse the code and
    check for raises and retrieve the class name of the exception - however
    this would be patchy at best. Therefore it would sort of negate the
    point of doing the analysis in the first place.

    Even in Java you cannot find every exception that will be
    thrown, only 'checked' exceptions but this is a percentage of all the
    exceptions (BTW why do you throw SystemException - it's a CORBA
    exception! OK, it's a runtime exception but why not just simply extend
    RuntimeException?). Also, if someone ever puts - catch (Exception e){}
    in their code they deserve to be kneecapped, IMHO the fault is with
    sloppy coding not with the supplied tools.

    Unfortunately its docs and testing again, that's why we get paid (if
    you're doing a job) or not paid (if you're doing it for fun!). Although
    one language which comes closer is Eiffel which has require and ensure
    clauses on every method (following Meyer's Programming by contract
    philosophy).

    Cheers,

    Neil

    --

    Neil Benn
    Senior Automation Engineer
    Cenix BioScience
    BioInnovations Zentrum
    Tatzberg 46
    D-01307
    Dresden
    Germany

    Tel : +49 (0)351 4173 154
    e-mail :
    Cenix Website : http://www.cenix-bioscience.com
     
    Neil Benn, Feb 2, 2005
    #5
  6. Hi,

    > I'm afraid that the only reliable way to gather what exceptions are
    > raised is to read docs and/or come up with test cases. This has been a
    > bugbear of mine in Python as it's not common to find a nice :Exceptions:
    > IOError <desc>, IllegalArgumentError <desc> type of description in the
    > docs.
    >
    > However if you want an incomplete test, you could parse the code and
    > check for raises and retrieve the class name of the exception - however
    > this would be patchy at best. Therefore it would sort of negate the
    > point of doing the analysis in the first place.


    I don't want that - the OP wants. I agree with you.

    > Even in Java you cannot find every exception that will be
    > thrown, only 'checked' exceptions but this is a percentage of all the
    > exceptions (BTW why do you throw SystemException - it's a CORBA
    > exception! OK, it's a runtime exception but why not just simply extend
    > RuntimeException?). Also, if someone ever puts - catch (Exception e){}
    > in their code they deserve to be kneecapped, IMHO the fault is with
    > sloppy coding not with the supplied tools.


    Most probably I throw RuntimeException - that was out of my head, I luckily
    I haven't been coding java too much lately :)

    > Unfortunately its docs and testing again, that's why we get paid (if
    > you're doing a job) or not paid (if you're doing it for fun!). Although
    > one language which comes closer is Eiffel which has require and ensure
    > clauses on every method (following Meyer's Programming by contract
    > philosophy).


    Full ack again.

    --
    Regards,

    Diez B. Roggisch
     
    Diez B. Roggisch, Feb 2, 2005
    #6

  7. >> Does anybody know of a tool that can tell me all possible exceptions
    >> that might occur in each line of code? What I'm hoping to find is
    >> something like the following:


    Paul> That is impossible. The parameter to the raise statement is a
    Paul> class object, which can be anything.

    Sure, but in all but the rarest of cases the first arg to raise is a
    specific exception, probably one of the standard exceptions. In the Python
    code in the distribution (ignoring the test directory where all sorts of
    mischief is done to stress things), here are the most common words following
    "raise" where "raise" is the first word on the line:

    % find . -name '*.py' \
    > | egrep -v '\./test' \
    > | xargs egrep '^ *raise ' \
    > | awk '{print $3}' \
    > | sed -e 's/[(,].*//' \
    > | sort \
    > | uniq -c \
    > | sort -rn \
    > | head -15

    246 ValueError
    227 aetools.Error
    216 Error
    124 TypeError
    101 error
    75 RuntimeError
    53 IOError
    36 NotImplementedError
    36 ImportError
    36 EOFError
    31 SyntaxError
    23 KeyError
    23 AttributeError
    22 DistutilsPlatformError
    21 UnicodeError

    Without checking, my guess is that #5 ("error") is one of a handful of
    exception classes defined at module scope (ftplib, anydbm, sre_constants,
    poplib, among others all define such an exception class), and not a variable
    that accepts multiple values as in your example.

    In short, while not perfect, simply grepping for '^ *(class|def|raise) ' and
    printing the first and second words of each output line would probably give
    you a pretty good idea of what gets raised where.

    Skip
     
    System Administrator, Feb 2, 2005
    #7
  8. Robey Holderith

    Roy Smith Guest

    In article <>,
    System Administrator <> wrote:
    >
    > >> Does anybody know of a tool that can tell me all possible exceptions
    > >> that might occur in each line of code? What I'm hoping to find is
    > >> something like the following:

    >
    > Paul> That is impossible. The parameter to the raise statement is a
    > Paul> class object, which can be anything.
    >
    >Sure, but in all but the rarest of cases the first arg to raise is a
    >specific exception, probably one of the standard exceptions. In the Python
    >code in the distribution (ignoring the test directory where all sorts of
    >mischief is done to stress things), here are the most common words following
    >"raise" where "raise" is the first word on the line:
    >
    > % find . -name '*.py' \
    > > | egrep -v '\./test' \
    > > | xargs egrep '^ *raise ' \
    > > | awk '{print $3}' \
    > > | sed -e 's/[(,].*//' \
    > > | sort \
    > > | uniq -c \
    > > | sort -rn \
    > > | head -15

    > 246 ValueError
    > 227 aetools.Error
    > 216 Error
    > 124 TypeError
    > 101 error
    > 75 RuntimeError
    > 53 IOError
    > 36 NotImplementedError
    > 36 ImportError
    > 36 EOFError
    > 31 SyntaxError
    > 23 KeyError
    > 23 AttributeError
    > 22 DistutilsPlatformError
    > 21 UnicodeError


    It's kind of interesting (scarry?) that in roughly 20% of the cases
    nothing more specific than Error is raised.
     
    Roy Smith, Feb 2, 2005
    #8
  9. Robey Holderith

    Peter Hansen Guest

    Roy Smith wrote:
    > Skip Montanaro wrote:
    >> 246 ValueError
    >> 227 aetools.Error
    >> 216 Error
    >> 124 TypeError
    >> 101 error
    >> 75 RuntimeError
    >> 53 IOError
    >> 36 NotImplementedError
    >> 36 ImportError
    >> 36 EOFError
    >> 31 SyntaxError
    >> 23 KeyError
    >> 23 AttributeError
    >> 22 DistutilsPlatformError
    >> 21 UnicodeError

    >
    > It's kind of interesting (scarry?) that in roughly 20% of the cases
    > nothing more specific than Error is raised.


    (In case someone reading doesn't know) there isn't actually
    an "Error" in the standard set of exceptions. It seems
    likely that pretty much all of those uses are actually
    module-specific Errors, and as such they are probably much
    more specific than the unadorned name might imply.

    Also, when one is trying to pick an appropriate exception
    to raise, it is often the case that none of the standard
    exceptions seems appropriate. In those cases (although I
    personally would prefer a different name than "Error")
    there's often no good alternative to making your own
    unique module-specific Exception subclass.

    -Peter
     
    Peter Hansen, Feb 2, 2005
    #9

  10. >> 246 ValueError
    >> 227 aetools.Error
    >> 216 Error
    >> 124 TypeError
    >> 101 error
    >> 75 RuntimeError
    >> 53 IOError
    >> 36 NotImplementedError
    >> 36 ImportError
    >> 36 EOFError
    >> 31 SyntaxError
    >> 23 KeyError
    >> 23 AttributeError
    >> 22 DistutilsPlatformError
    >> 21 UnicodeError


    Roy> It's kind of interesting (scarry?) that in roughly 20% of the cases
    Roy> nothing more specific than Error is raised.

    Not really. You might have code in a module named mod like this:

    class Error(Exception): pass

    then later:

    def foo(...):
    ... blah blah blah ...
    if condition:
    raise Error, "hey dummy!"

    The caller might look like:

    import mod

    ...

    try:
    mod.foo()
    except mod.Error, msg:
    print msg

    That said, the tendency for many newer modules seems to be to discriminate
    exceptions based on type (look at urllib2 for example) while older modules
    tended to have just a single exception they raised (look at ftplib). I
    suspect that has something to do with whether the module was originally
    written before or after Python introduced class exceptions.

    Skip
     
    Skip Montanaro, Feb 2, 2005
    #10
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. JimLad
    Replies:
    0
    Views:
    340
    JimLad
    Sep 12, 2006
  2. CptDondo

    Auditing C code

    CptDondo, Sep 19, 2006, in forum: C Programming
    Replies:
    9
    Views:
    474
    Chris Torek
    Sep 24, 2006
  3. Elhanan

    auditing with context?

    Elhanan, Mar 12, 2009, in forum: Java
    Replies:
    4
    Views:
    368
    Arved Sandstrom
    Mar 13, 2009
  4. RM

    Auditing .net generated files

    RM, Oct 6, 2009, in forum: ASP .Net
    Replies:
    0
    Views:
    300
  5. Elhanan

    code auditing tools for dot.net?

    Elhanan, Oct 27, 2009, in forum: ASP .Net
    Replies:
    0
    Views:
    336
    Elhanan
    Oct 27, 2009
Loading...

Share This Page