Using other tools with unittest?

Discussion in 'Python' started by Edward K. Ream, Nov 18, 2003.

  1. Has anyone used tools like pychecker (or even Python's own debugger or
    profiler) during unit testing? Here are some things that might be natural
    to do:

    1. pychecker can run all kinds of fancy tests on code. I'm wondering
    whether these tests could be incorporated somehow into individual unit
    tests. For example, one might use pychecker to assert that a subclass does,
    or does not, override a method of a base class. Yes, one could write that
    test "by hand". The question is: how good is pychecker for this sort of
    thing? Anyone have any experience?

    2. Python's debugger is simple because Python so helpfully provides the
    sys.settrace function. Has anyone used sys.settrace for unit testing? For
    example, suppose you wanted to make a list of the methods that are exercised
    by a test suite. The tracing function registered by sys.settrace could
    easily do this. Etc.

    3 In a slightly different direction, has someone used a simulation of
    aspect-oriented programming in Python to make statements about test
    coverage, or for other testing purposes?

    Thanks.

    Edward
    --------------------------------------------------------------------
    Edward K. Ream email:
    Leo: Literate Editor with Outlines
    Leo: http://webpages.charter.net/edreamleo/front.html
    --------------------------------------------------------------------
     
    Edward K. Ream, Nov 18, 2003
    #1
    1. Advertising

  2. Edward K. Ream

    Jp Calderone Guest

    On Tue, Nov 18, 2003 at 04:11:18PM -0600, Edward K. Ream wrote:
    > Has anyone used tools like pychecker (or even Python's own debugger or
    > profiler) during unit testing? Here are some things that might be natural
    > to do:
    >
    > 1. pychecker can run all kinds of fancy tests on code. I'm wondering
    > whether these tests could be incorporated somehow into individual unit
    > tests. For example, one might use pychecker to assert that a subclass does,
    > or does not, override a method of a base class. Yes, one could write that
    > test "by hand". The question is: how good is pychecker for this sort of
    > thing? Anyone have any experience?


    Dunno. The way I typically write tests, I'll just call the method. If
    it's not implemented (or implemented incorrectly), it won't behave how I
    expect it to, and the test will fail.

    It might be interesting to have a "pychecker test" provided by the test
    framework. Similar to the common "at least import" tests, this could run
    pychecker on all modules in a project, and fail if errors (and/or warnings)
    are emitted. I've never looked inside pychecker, I don't know how easy or
    hard this would be to implement.

    >
    > 2. Python's debugger is simple because Python so helpfully provides the
    > sys.settrace function. Has anyone used sys.settrace for unit testing? For
    > example, suppose you wanted to make a list of the methods that are exercised
    > by a test suite. The tracing function registered by sys.settrace could
    > easily do this. Etc.


    A few test frameworks have support for reporting code coverage. No names
    spring to mind at the moment, unfortunately. Also, a project I work on uses
    this module:

    http://www.ravenbrook.com/project/p4dti/master/test/coverage.py

    in its unit tests to generate coverage statistics (the "test" can't ever
    fail; a graph is generated based on its results, though; it is run for every
    checkin).

    >
    > 3 In a slightly different direction, has someone used a simulation of
    > aspect-oriented programming in Python to make statements about test
    > coverage, or for other testing purposes?


    Not exactly sure what you're getting at here.


    Jp
     
    Jp Calderone, Nov 18, 2003
    #2
    1. Advertising

  3. Edward K. Ream

    John Roth Guest

    "Edward K. Ream" <> wrote in message
    news:...
    > Has anyone used tools like pychecker (or even Python's own debugger or
    > profiler) during unit testing? Here are some things that might be natural
    > to do:
    >
    > 1. pychecker can run all kinds of fancy tests on code. I'm wondering
    > whether these tests could be incorporated somehow into individual unit
    > tests. For example, one might use pychecker to assert that a subclass

    does,
    > or does not, override a method of a base class. Yes, one could write that
    > test "by hand". The question is: how good is pychecker for this sort of
    > thing? Anyone have any experience?


    I don't have any experiance, but the statement on the project page
    http://pychecker.sourceforge.net/
    doesn't suggest that it can do what you mention. It's more a static
    analyzer for common problems.

    That said, it might be a good project practice that all code needs to
    pass pychecker before it can be committed.

    Also, I'd like to know how you write that kind of test 'by hand.'
    I can certainly see checking a single class for a manditory override /
    not override, but I can't see exactly how you'd check *all* subclasses
    for that condition without building a lot of infrastructure. You would at
    least have to *find* all the class definitions.

    > 2. Python's debugger is simple because Python so helpfully provides the
    > sys.settrace function. Has anyone used sys.settrace for unit testing?

    For
    > example, suppose you wanted to make a list of the methods that are

    exercised
    > by a test suite. The tracing function registered by sys.settrace could
    > easily do this. Etc.


    There are at least two coverage tools that use the trace hook, and
    I suspect there are others. One is accessable through the Vaults;
    I don't know the rest of them. I've frequently thought that if you're
    using XP or TDD, it might be a real good practice to run a coverage
    checker. TDD is supposed to get you very close to 100% statement
    coverage without extra effort.

    When I want to check, I simply run my test suite under the coverage
    tool. It takes about three times (or more) to run that way.

    > 3 In a slightly different direction, has someone used a simulation of
    > aspect-oriented programming in Python to make statements about test
    > coverage, or for other testing purposes?


    I don't understand this one at all...

    John Roth

    >
    > Thanks.
    >
    > Edward
    > --------------------------------------------------------------------
    > Edward K. Ream email:
    > Leo: Literate Editor with Outlines
    > Leo: http://webpages.charter.net/edreamleo/front.html
    > --------------------------------------------------------------------
    >
    >
     
    John Roth, Nov 18, 2003
    #3
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Namratha Shah \(Nasha\)
    Replies:
    0
    Views:
    624
    Namratha Shah \(Nasha\)
    Nov 23, 2004
  2. Paul  Moore
    Replies:
    1
    Views:
    403
    Paul Moore
    Oct 14, 2008
  3. Namratha Shah \(Nasha\)

    Article : Security Tools Part -- 2 (.Net FrameWork Tools Series)

    Namratha Shah \(Nasha\), Nov 23, 2004, in forum: ASP .Net Datagrid Control
    Replies:
    0
    Views:
    219
    Namratha Shah \(Nasha\)
    Nov 23, 2004
  4. Namratha Shah \(Nasha\)

    Article : Security Tools Part -- 2 (.Net FrameWork Tools Series)

    Namratha Shah \(Nasha\), Nov 23, 2004, in forum: ASP .Net Mobile
    Replies:
    0
    Views:
    209
    Namratha Shah \(Nasha\)
    Nov 23, 2004
  5. Dwight Hutto
    Replies:
    2
    Views:
    206
    Dwight Hutto
    Oct 18, 2012
Loading...

Share This Page