Unit testing of expected failures -- what do you use?

Discussion in 'C++' started by Alf P. Steinbach, Feb 27, 2010.

  1. OK, this displays my ignorance of what's out there (it's been a long time since
    I developed for a living), and also my laziness not googling. :)

    However.

    I want to unit-test some library code I'm sort of extracting from some old code
    I have.

    The things that should work without error are easy to test, and it's currently
    not so much code that I've considered a testing framework, although the code
    size increases. I'm thinking that perhaps the popular frameworks don't support
    my needs: there are cases where the code /should/ assert at run time. And worse,
    there are cases where the could should assert at compile time...

    How do you deal with this kind of testing, testing that things fail as they
    should (at compile time and at run time)?

    I'm particularly interested in anything that works well with Visual Studio 7.x
    or thereabouts, but I'm thinking that I most likely will have to create makefiles?


    Cheers,

    - Alf
     
    Alf P. Steinbach, Feb 27, 2010
    #1
    1. Advertising

  2. Alf P. Steinbach

    Ian Collins Guest

    Alf P. Steinbach wrote:
    > OK, this displays my ignorance of what's out there (it's been a long
    > time since I developed for a living), and also my laziness not googling.
    > :)
    >
    > However.
    >
    > I want to unit-test some library code I'm sort of extracting from some
    > old code I have.
    >
    > The things that should work without error are easy to test, and it's
    > currently not so much code that I've considered a testing framework,
    > although the code size increases. I'm thinking that perhaps the popular
    > frameworks don't support my needs: there are cases where the code
    > /should/ assert at run time. And worse, there are cases where the could
    > should assert at compile time...


    I used to use cppUnit (and still do for older projects) but now I use
    gtest (http://code.google.com/p/googletest/). Compile time asserts are
    beyond the scope of a unit testing framework; if it won't compile, there
    isn't a unit to test!

    > How do you deal with this kind of testing, testing that things fail as
    > they should (at compile time and at run time)?


    For run time asserts, I interpose whatever function the system's assert
    macro calls (__assert on Solaris) and have __assert throw an exception
    with the file, line and expression passed by assert.

    A boiled down (no framework) example:

    #include <iostream>
    #include <exception>
    #include <assert.h>

    struct AssertionException : std::runtime_error
    {
    const char* expression;
    const char* file;
    int line;

    AssertionException(const char* expression, const char* file, int line)
    : std::runtime_error( expression ),
    expression(expression), file(file), line(line) {}
    };

    void __assert(const char* expression, const char* file, int line)
    {
    throw AssertionException( expression, file, line );
    }

    void fut( const char* p )
    {
    assert( NULL != p );
    }

    int main()
    {
    try
    {
    fut( NULL );
    std::cerr << "Oops" << std::endl;
    }
    catch( const AssertionException& e )
    {
    std::cerr << "Caught" << ": " << e.what() << std::endl;
    }
    }

    --
    Ian Collins
     
    Ian Collins, Feb 27, 2010
    #2
    1. Advertising

  3. Alf P. Steinbach

    Jorgen Grahn Guest

    On Sat, 2010-02-27, Ian Collins wrote:
    > Alf P. Steinbach wrote:
    >> OK, this displays my ignorance of what's out there (it's been a long
    >> time since I developed for a living), and also my laziness not googling.
    >> :)
    >>
    >> However.
    >>
    >> I want to unit-test some library code I'm sort of extracting from some
    >> old code I have.
    >>
    >> The things that should work without error are easy to test, and it's
    >> currently not so much code that I've considered a testing framework,
    >> although the code size increases. I'm thinking that perhaps the popular
    >> frameworks don't support my needs: there are cases where the code
    >> /should/ assert at run time. And worse, there are cases where the could
    >> should assert at compile time...

    >
    > I used to use cppUnit (and still do for older projects) but now I use
    > gtest (http://code.google.com/p/googletest/). Compile time asserts are
    > beyond the scope of a unit testing framework; if it won't compile, there
    > isn't a unit to test!


    I don't even know what a "unit" is in this context. But I disagree --
    with C++ it's sometimes as important to assure that some things don't
    compile or fail noisily at runtime, as it is that the "normal tests"
    pass.

    I seem to recall that *some* of them support static checks. But I
    haven't used any, and it wouldn't surprise me if they were clumpsy.
    It *does* mean stepping into the build system's territory after all.

    I've never done it, but compile-time checks I think I would implement
    myself, in my Makefile. The "make check" target would of course drive
    both the normal unit tests and the compile-time things.

    But I'm not on Windows, and I wouldn't hesitate to use Gnu
    Make-specific features.

    >> How do you deal with this kind of testing, testing that things fail as
    >> they should (at compile time and at run time)?

    >
    > For run time asserts, I interpose whatever function the system's assert
    > macro calls (__assert on Solaris) and have __assert throw an exception
    > with the file, line and expression passed by assert.


    /Jorgen

    --
    // Jorgen Grahn <grahn@ Oo o. . .
    \X/ snipabacken.se> O o .
     
    Jorgen Grahn, Feb 28, 2010
    #3
  4. Alf P. Steinbach

    Robert Fendt Guest

    And thus spake Jorgen Grahn <>
    28 Feb 2010 11:49:48 GMT:

    > > I used to use cppUnit (and still do for older projects) but now I use
    > > gtest (http://code.google.com/p/googletest/). Compile time asserts are
    > > beyond the scope of a unit testing framework; if it won't compile, there
    > > isn't a unit to test!

    >
    > I don't even know what a "unit" is in this context. But I disagree --
    > with C++ it's sometimes as important to assure that some things don't
    > compile or fail noisily at runtime, as it is that the "normal tests"
    > pass.
    >
    > I seem to recall that *some* of them support static checks. But I
    > haven't used any, and it wouldn't surprise me if they were clumpsy.
    > It *does* mean stepping into the build system's territory after all.


    There's some stuff on compile-time assertions in the
    Alexandrescu book ('Modern C++ Design'). Have a look at chapter
    2, 'techniques', right at the beginning.

    Regards,
    Robert
     
    Robert Fendt, Feb 28, 2010
    #4
  5. Alf P. Steinbach wrote:
    > OK, this displays my ignorance of what's out there (it's been a long
    > time since I developed for a living), and also my laziness not googling.
    > :)
    >
    > However.
    >
    > I want to unit-test some library code I'm sort of extracting from some
    > old code I have.
    >


    For unit testing, see this:
    http://cxxtest.sourceforge.net/guide.html

    > The things that should work without error are easy to test, and it's
    > currently not so much code that I've considered a testing framework,
    > although the code size increases. I'm thinking that perhaps the popular
    > frameworks don't support my needs: there are cases where the code
    > /should/ assert at run time. And worse, there are cases where the could
    > should assert at compile time...
    >
    > How do you deal with this kind of testing, testing that things fail as
    > they should (at compile time and at run time)?


    For compile time testing, see this:
    http://www.boost.org/doc/libs/1_42_0/doc/html/boost_staticassert.html

    For run time testing, I am using a macro, which throws an exception if
    the condition fails. The exception class prints backtrace and the failed
    condition.
     
    Vladimir Jovic, Mar 1, 2010
    #5
  6. * Vladimir Jovic:
    > Alf P. Steinbach wrote:
    >> OK, this displays my ignorance of what's out there (it's been a long
    >> time since I developed for a living), and also my laziness not
    >> googling. :)
    >>
    >> However.
    >>
    >> I want to unit-test some library code I'm sort of extracting from some
    >> old code I have.
    >>

    >
    > For unit testing, see this:
    > http://cxxtest.sourceforge.net/guide.html
    >
    >> The things that should work without error are easy to test, and it's
    >> currently not so much code that I've considered a testing framework,
    >> although the code size increases. I'm thinking that perhaps the
    >> popular frameworks don't support my needs: there are cases where the
    >> code /should/ assert at run time. And worse, there are cases where the
    >> could should assert at compile time...
    >>
    >> How do you deal with this kind of testing, testing that things fail as
    >> they should (at compile time and at run time)?

    >
    > For compile time testing, see this:
    > http://www.boost.org/doc/libs/1_42_0/doc/html/boost_staticassert.html
    >
    > For run time testing, I am using a macro, which throws an exception if
    > the condition fails. The exception class prints backtrace and the failed
    > condition.


    Thanks, but I think you misunderstood the question.

    E.g., the problem isn't to produce compile time asserts. The problem is testing
    them, systematically. Preferably in an automated way.


    Cheers, & thanks,

    - Alf
     
    Alf P. Steinbach, Mar 1, 2010
    #6
  7. * Alf P. Steinbach:
    > OK, this displays my ignorance of what's out there (it's been a long
    > time since I developed for a living), and also my laziness not googling.
    > :)
    >
    > However.
    >
    > I want to unit-test some library code I'm sort of extracting from some
    > old code I have.
    >
    > The things that should work without error are easy to test, and it's
    > currently not so much code that I've considered a testing framework,
    > although the code size increases. I'm thinking that perhaps the popular
    > frameworks don't support my needs: there are cases where the code
    > /should/ assert at run time. And worse, there are cases where the could
    > should assert at compile time...


    OK, I made a unit test driver GUI in Python 3.x, <url:
    http://pastebin.com/bB1jeP5Z>.

    It reports success or failure depending on whether build is expected to succeed
    or fail, and depending on whether running the executable is expected to succeed
    or fail.

    It's very unfinished but this is a first usable version.

    Tests and subtests are identified by C++ macro symbols defined in an [.ini]
    file. Running a test the GUI (1) places #define's of the selected symbols in
    file [_config.h], (2) invokes a 'build' script which in Windows can simply be a
    batch file (building a /test/ is ordinarily very fast), and (3) if the build
    succeeds and is expected to succeed, it invokes a 'run' script.

    My 'build' script looks like this:

    <code file="build.bat">
    @echo off
    devenv /nologo my_msvc_solution.sln /project my_lib_project.vcproj /build Debug
    </code>

    And the 'run' script is simply:

    <code file="run.bat">
    @Debug\the_name_of_the_executable
    </code>

    In the Python code the script (batch file) directory is hardcoded as
    build_dir = os.path.normpath( "../build/msvc_7_1" )
    but it should be easy to change.

    For a main test the build is always expected to succeed.

    For a subtest the build is expected to fail if the subtest id starts with "CERR_".


    Cheers,

    - Alf
     
    Alf P. Steinbach, Mar 1, 2010
    #7
  8. Alf P. Steinbach

    tonydee Guest

    On Mar 2, 5:37 am, "Alf P. Steinbach" <> wrote:
    > > The things that should work without error are easy to test, and it's
    > > currently not so much code that I've considered a testing framework,
    > > although the code size increases. I'm thinking that perhaps the popular
    > > frameworks don't support my needs: there are cases where the code
    > > /should/ assert at run time. And worse, there are cases where the could
    > > should assert at compile time...

    >
    > OK, I made a unit test driver GUI in Python 3.x, <url:http://pastebin.com/bB1jeP5Z>.


    I've done something similar (conceptually, didn't check out your
    Python code). Just thought I'd mention another alternative (at least
    on UNIX) ala "configure" scripts - they often check for success or
    failure in compiling little test programs.

    Cheers,
    Tony
     
    tonydee, Mar 2, 2010
    #8
  9. Alf P. Steinbach wrote:
    > * Alf P. Steinbach:
    >> OK, this displays my ignorance of what's out there (it's been a long
    >> time since I developed for a living), and also my laziness not
    >> googling. :)
    >>
    >> However.
    >>
    >> I want to unit-test some library code I'm sort of extracting from some
    >> old code I have.
    >>
    >> The things that should work without error are easy to test, and it's
    >> currently not so much code that I've considered a testing framework,
    >> although the code size increases. I'm thinking that perhaps the
    >> popular frameworks don't support my needs: there are cases where the
    >> code /should/ assert at run time. And worse, there are cases where the
    >> could should assert at compile time...

    >
    > OK, I made a unit test driver GUI in Python 3.x, <url:
    > http://pastebin.com/bB1jeP5Z>.
    >
    > It reports success or failure depending on whether build is expected to
    > succeed or fail, and depending on whether running the executable is
    > expected to succeed or fail.
    >
    > It's very unfinished but this is a first usable version.
    >
    > Tests and subtests are identified by C++ macro symbols defined in an
    > [.ini] file. Running a test the GUI (1) places #define's of the selected
    > symbols in file [_config.h], (2) invokes a 'build' script which in
    > Windows can simply be a batch file (building a /test/ is ordinarily very
    > fast), and (3) if the build succeeds and is expected to succeed, it
    > invokes a 'run' script.
    >
    > My 'build' script looks like this:
    >
    > <code file="build.bat">
    > @echo off
    > devenv /nologo my_msvc_solution.sln /project my_lib_project.vcproj
    > /build Debug
    > </code>
    >
    > And the 'run' script is simply:
    >
    > <code file="run.bat">
    > @Debug\the_name_of_the_executable
    > </code>
    >
    > In the Python code the script (batch file) directory is hardcoded as
    > build_dir = os.path.normpath( "../build/msvc_7_1" )
    > but it should be easy to change.
    >
    > For a main test the build is always expected to succeed.
    >
    > For a subtest the build is expected to fail if the subtest id starts
    > with "CERR_".
    >


    Ok, now I get it. For this kind of testing, you can use this:
    http://expect.nist.gov/
     
    Vladimir Jovic, Mar 2, 2010
    #9
  10. Alf P. Steinbach

    DeMarcus Guest

    Ian Collins wrote:
    > Alf P. Steinbach wrote:
    >> OK, this displays my ignorance of what's out there (it's been a long
    >> time since I developed for a living), and also my laziness not
    >> googling. :)
    >>
    >> However.
    >>
    >> I want to unit-test some library code I'm sort of extracting from some
    >> old code I have.
    >>
    >> The things that should work without error are easy to test, and it's
    >> currently not so much code that I've considered a testing framework,
    >> although the code size increases. I'm thinking that perhaps the
    >> popular frameworks don't support my needs: there are cases where the
    >> code /should/ assert at run time. And worse, there are cases where the
    >> could should assert at compile time...

    >
    > I used to use cppUnit (and still do for older projects) but now I use
    > gtest (http://code.google.com/p/googletest/). Compile time asserts are
    > beyond the scope of a unit testing framework; if it won't compile, there
    > isn't a unit to test!
    >
    >> How do you deal with this kind of testing, testing that things fail as
    >> they should (at compile time and at run time)?

    >
    > For run time asserts, I interpose whatever function the system's assert
    > macro calls (__assert on Solaris) and have __assert throw an exception
    > with the file, line and expression passed by assert.
    >
    > A boiled down (no framework) example:
    >
    > #include <iostream>
    > #include <exception>
    > #include <assert.h>
    >
    > struct AssertionException : std::runtime_error
    > {
    > const char* expression;
    > const char* file;
    > int line;
    >
    > AssertionException(const char* expression, const char* file, int line)
    > : std::runtime_error( expression ),
    > expression(expression), file(file), line(line) {}
    > };
    >
    > void __assert(const char* expression, const char* file, int line)
    > {
    > throw AssertionException( expression, file, line );
    > }
    >
    > void fut( const char* p )
    > {
    > assert( NULL != p );
    > }
    >
    > int main()
    > {
    > try
    > {
    > fut( NULL );
    > std::cerr << "Oops" << std::endl;
    > }
    > catch( const AssertionException& e )
    > {
    > std::cerr << "Caught" << ": " << e.what() << std::endl;
    > }
    > }
    >


    I'm not a complete expert, nor am I a sulky person, but before writing
    assert exceptions one might consider following that I found in the book
    C++ Coding Standards by Sutter & Alexandrescu, Item 68 - Assert
    liberally to document internal assumptions and invariants.

    Quote:
    "It is not recommended to throw an exception instead of asserting, even
    though the standard std::logic_error exception class was originally
    designed for this purpose. The primary disadvantage of using an
    exception to report a programming error is that you don't really want
    stack unwinding to occur - you want the debugger to launch on the exact
    line where the violation was detected, with the line's state intact."


    /Daniel
     
    DeMarcus, Mar 3, 2010
    #10
  11. Alf P. Steinbach wrote:
    > E.g., the problem isn't to produce compile time asserts. The problem is
    > testing them, systematically. Preferably in an automated way.


    I published the following article that addresses specifically this
    problem (after a tutorial on testing C++ programs):
    http://www.drdobbs.com/cpp/205801074

    I published also the open source utility described in that article.
    It is in the files "staticpp.txt" and "staticpp.zip" in the following
    volume:
    ftp://66.77.27.238/sourcecode/ddj/2008/0802.zip

    --

    Carlo Milanesi
    http://digilander.libero.it/carlmila
     
    Carlo Milanesi, Mar 3, 2010
    #11
  12. * Carlo Milanesi:
    > Alf P. Steinbach wrote:
    >> E.g., the problem isn't to produce compile time asserts. The problem
    >> is testing them, systematically. Preferably in an automated way.

    >
    > I published the following article that addresses specifically this
    > problem (after a tutorial on testing C++ programs):
    > http://www.drdobbs.com/cpp/205801074
    >
    > I published also the open source utility described in that article.
    > It is in the files "staticpp.txt" and "staticpp.zip" in the following
    > volume:
    > ftp://66.77.27.238/sourcecode/ddj/2008/0802.zip


    Hm, it's very similar to my approach. But instead of your @LEGAL and @ILLEGAL I
    just use ordinary C++ preprocessor directives. This allows testing to be
    performed manually, without any preprocessing or having a script at hand.

    It also allows having just a single Visual C++ project and project settings for
    the test suite for a library, and almost completely decouples the scripting side
    from the C++ source code side.

    The script -> C++ connection is that at the scripting side one defines the
    relevant preprocessor macros for each test.

    Before building a test the script places the macro definitions for the selected
    test in a file '_config.h', which is the only C++ file that it knows about.

    The Python test driver script (not exactly finished, but working!) is available
    at <url: http://pastebin.com/NK8yVcyv>.


    ---


    The main program for my test suite for the lib I'm testing starts like this:


    <code>
    #include "_config.h" // Generated.

    #if \
    defined( USE_STATIC_ASSERT ) || \
    defined( USE_STATIC_ASSERT__OVERRIDE ) || \
    defined( USE_GENERAL_WARNINGS_SUPPRESSION ) || \
    defined( USE_DEBUGGER_API ) || \
    defined( USE_DEBUGGING ) || \
    defined( USE_FIXED_SIZE_TYPES ) || \
    defined( USE_TYPECHECKING ) || \
    defined( USE_TYPECHECKING__DOWNCAST_OF_POINTER ) || \
    defined( USE_TYPECHECKING__DOWNCAST_OF_REFERENCE ) || \
    defined( USE_PRIMITIVE_TYPES__CASTBYTEPTRFROM ) || \
    0
    // OK
    #else
    # error Unknown test symbol or no test symbol defined.
    #endif
    </code>


    I haven't gotten around to replace the "USE_" prefix with "TEST_" yet... :)

    A typical file to be tested (I just chose a short one):


    <code>
    // Copyright (c) Alf P. Steinbach, 2010.
    #include <progrock/cppx/devsupport/static_assert.h>

    #if \
    defined( CPPXTEST_NO_SUBTESTS ) || \
    defined( CERR_NONEXISTING ) || \
    0
    // OK
    #else
    # error Unknown or no subtest symbol defined (define CPPXTEST_NO_SUBTESTS?).
    #endif


    class Base
    {
    protected:
    virtual void foo( int, char const* ) const = 0;
    };

    class Derived:
    public Base
    {
    protected:
    virtual void foo( int a, char const* b ) const
    {
    CPPX_IS_OVERRIDE_OF( Base::foo,(a, b) ); // Should compile fine.
    }

    virtual void bar() const
    {
    #if defined( CERR_NONEXISTING )
    CPPX_IS_OVERRIDE_OF( Base::bar,() ); // Should yield
    compilation error.
    #endif
    }
    };
    </code>


    Cheers, & thanks for that article ref!,

    - Alf
     
    Alf P. Steinbach, Mar 3, 2010
    #12
  13. Alf P. Steinbach

    Ian Collins Guest

    DeMarcus wrote:
    > Ian Collins wrote:
    >>
    >> A boiled down (no framework) example:
    >>
    >> #include <iostream>
    >> #include <exception>
    >> #include <assert.h>
    >>
    >> struct AssertionException : std::runtime_error
    >> {
    >> const char* expression;
    >> const char* file;
    >> int line;
    >>
    >> AssertionException(const char* expression, const char* file, int line)
    >> : std::runtime_error( expression ),
    >> expression(expression), file(file), line(line) {}
    >> };
    >>
    >> void __assert(const char* expression, const char* file, int line)
    >> {
    >> throw AssertionException( expression, file, line );
    >> }
    >>

    >
    > I'm not a complete expert, nor am I a sulky person, but before writing
    > assert exceptions one might consider following that I found in the book
    > C++ Coding Standards by Sutter & Alexandrescu, Item 68 - Assert
    > liberally to document internal assumptions and invariants.
    >
    > Quote:
    > "It is not recommended to throw an exception instead of asserting, even
    > though the standard std::logic_error exception class was originally
    > designed for this purpose. The primary disadvantage of using an
    > exception to report a programming error is that you don't really want
    > stack unwinding to occur - you want the debugger to launch on the exact
    > line where the violation was detected, with the line's state intact."


    My example above was for a unit test harness used to confirm that
    assertions trigger when expected. It was not a suggestion for
    production code!

    In a test harness, you could get the mock __assert function to set a
    flag to indicate that it has been called, but that won't case the called
    function to terminate. It will blunder on with invalid inputs and do
    nasty things. In this instance, mapping assertions to exceptions is the
    only practical solution.

    --
    Ian Collins
     
    Ian Collins, Mar 3, 2010
    #13
  14. Alf P. Steinbach

    DeMarcus Guest

    Ian Collins wrote:
    > DeMarcus wrote:
    >> Ian Collins wrote:
    >>>
    >>> A boiled down (no framework) example:
    >>>
    >>> #include <iostream>
    >>> #include <exception>
    >>> #include <assert.h>
    >>>
    >>> struct AssertionException : std::runtime_error
    >>> {
    >>> const char* expression;
    >>> const char* file;
    >>> int line;
    >>>
    >>> AssertionException(const char* expression, const char* file, int line)
    >>> : std::runtime_error( expression ),
    >>> expression(expression), file(file), line(line) {}
    >>> };
    >>>
    >>> void __assert(const char* expression, const char* file, int line)
    >>> {
    >>> throw AssertionException( expression, file, line );
    >>> }
    >>>

    >>
    >> I'm not a complete expert, nor am I a sulky person, but before writing
    >> assert exceptions one might consider following that I found in the
    >> book C++ Coding Standards by Sutter & Alexandrescu, Item 68 - Assert
    >> liberally to document internal assumptions and invariants.
    >>
    >> Quote:
    >> "It is not recommended to throw an exception instead of asserting,
    >> even though the standard std::logic_error exception class was
    >> originally designed for this purpose. The primary disadvantage of
    >> using an exception to report a programming error is that you don't
    >> really want stack unwinding to occur - you want the debugger to launch
    >> on the exact line where the violation was detected, with the line's
    >> state intact."

    >
    > My example above was for a unit test harness used to confirm that
    > assertions trigger when expected. It was not a suggestion for
    > production code!
    >
    > In a test harness, you could get the mock __assert function to set a
    > flag to indicate that it has been called, but that won't case the called
    > function to terminate. It will blunder on with invalid inputs and do
    > nasty things. In this instance, mapping assertions to exceptions is the
    > only practical solution.
    >


    Oh, I see. That's useful.
     
    DeMarcus, Mar 3, 2010
    #14
  15. * Ian Collins:
    > DeMarcus wrote:
    >> Ian Collins wrote:
    >>>
    >>> A boiled down (no framework) example:
    >>>
    >>> #include <iostream>
    >>> #include <exception>
    >>> #include <assert.h>
    >>>
    >>> struct AssertionException : std::runtime_error
    >>> {
    >>> const char* expression;
    >>> const char* file;
    >>> int line;
    >>>
    >>> AssertionException(const char* expression, const char* file, int line)
    >>> : std::runtime_error( expression ),
    >>> expression(expression), file(file), line(line) {}
    >>> };
    >>>
    >>> void __assert(const char* expression, const char* file, int line)
    >>> {
    >>> throw AssertionException( expression, file, line );
    >>> }
    >>>

    >>
    >> I'm not a complete expert, nor am I a sulky person, but before writing
    >> assert exceptions one might consider following that I found in the
    >> book C++ Coding Standards by Sutter & Alexandrescu, Item 68 - Assert
    >> liberally to document internal assumptions and invariants.
    >>
    >> Quote:
    >> "It is not recommended to throw an exception instead of asserting,
    >> even though the standard std::logic_error exception class was
    >> originally designed for this purpose. The primary disadvantage of
    >> using an exception to report a programming error is that you don't
    >> really want stack unwinding to occur - you want the debugger to launch
    >> on the exact line where the violation was detected, with the line's
    >> state intact."

    >
    > My example above was for a unit test harness used to confirm that
    > assertions trigger when expected. It was not a suggestion for
    > production code!
    >
    > In a test harness, you could get the mock __assert function to set a
    > flag to indicate that it has been called, but that won't case the called
    > function to terminate. It will blunder on with invalid inputs and do
    > nasty things. In this instance, mapping assertions to exceptions is the
    > only practical solution.


    I must disagree. I think it *can* be practical when you have good knowledge of
    the code being tested. But for arbitrary code there might just be a catch(...)
    somewhere.

    So I think that in general it's better to just have the test harness (script,
    likely) detect the failed execution.

    And sometimes that failed execution is what's expected for a successful test.


    Cheers,

    - Alf
     
    Alf P. Steinbach, Mar 3, 2010
    #15
  16. Alf P. Steinbach

    Ian Collins Guest

    Alf P. Steinbach wrote:
    > * Ian Collins:
    >> DeMarcus wrote:
    >>> Ian Collins wrote:
    >>>>
    >>>> A boiled down (no framework) example:
    >>>>
    >>>> #include <iostream>
    >>>> #include <exception>
    >>>> #include <assert.h>
    >>>>
    >>>> struct AssertionException : std::runtime_error
    >>>> {
    >>>> const char* expression;
    >>>> const char* file;
    >>>> int line;
    >>>>
    >>>> AssertionException(const char* expression, const char* file, int
    >>>> line)
    >>>> : std::runtime_error( expression ),
    >>>> expression(expression), file(file), line(line) {}
    >>>> };
    >>>>
    >>>> void __assert(const char* expression, const char* file, int line)
    >>>> {
    >>>> throw AssertionException( expression, file, line );
    >>>> }
    >>>>
    >>>
    >>> I'm not a complete expert, nor am I a sulky person, but before
    >>> writing assert exceptions one might consider following that I found
    >>> in the book C++ Coding Standards by Sutter & Alexandrescu, Item 68 -
    >>> Assert liberally to document internal assumptions and invariants.
    >>>
    >>> Quote:
    >>> "It is not recommended to throw an exception instead of asserting,
    >>> even though the standard std::logic_error exception class was
    >>> originally designed for this purpose. The primary disadvantage of
    >>> using an exception to report a programming error is that you don't
    >>> really want stack unwinding to occur - you want the debugger to
    >>> launch on the exact line where the violation was detected, with the
    >>> line's state intact."

    >>
    >> My example above was for a unit test harness used to confirm that
    >> assertions trigger when expected. It was not a suggestion for
    >> production code!
    >>
    >> In a test harness, you could get the mock __assert function to set a
    >> flag to indicate that it has been called, but that won't case the
    >> called function to terminate. It will blunder on with invalid inputs
    >> and do nasty things. In this instance, mapping assertions to
    >> exceptions is the only practical solution.

    >
    > I must disagree. I think it *can* be practical when you have good
    > knowledge of the code being tested. But for arbitrary code there might
    > just be a catch(...) somewhere.
    >
    > So I think that in general it's better to just have the test harness
    > (script, likely) detect the failed execution.


    No, we were (at least I was) talking about unit tests here. There might
    be hundreds or thousands of tests. You want to see the test assertions
    pass or fail, not the execution of the tests.

    In my original example:

    try
    {
    fut( NULL );
    std::cerr << "Oops" << std::endl;
    }
    catch( const AssertionException& e )
    {
    std::cerr << "Caught" << ": " << e.what() << std::endl;
    }

    The code is asserting fut() asserts when passed NULL. WIthin a test
    framework (cppUnit in this case) the code would look something like:

    try
    {
    fut( NULL );
    CPPUNT_ASSERT( !"fut failed to assert on NULL" );
    }
    catch( const AssertionException& e ) {}

    --
    Ian Collins
     
    Ian Collins, Mar 3, 2010
    #16
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. John Maclean
    Replies:
    1
    Views:
    356
    Martin P. Hellwig
    Apr 13, 2010
  2. Ulrich Eckhardt

    unit-profiling, similar to unit-testing

    Ulrich Eckhardt, Nov 16, 2011, in forum: Python
    Replies:
    6
    Views:
    349
    Roy Smith
    Nov 18, 2011
  3. Replies:
    1
    Views:
    119
    Jan Svitok
    Aug 7, 2006
  4. Bill Mosteller
    Replies:
    0
    Views:
    239
    Bill Mosteller
    Oct 22, 2009
  5. Avi
    Replies:
    0
    Views:
    507
Loading...

Share This Page