Java - Junit distinguish exceptions

Discussion in 'Java' started by Ronny Mandal, Dec 10, 2010.

  1. Ronny Mandal

    Ronny Mandal Guest

    Hello, I am relatively new to JUnit. I am instrumenting already
    existing tests, what I want to do is to record which exceptions that
    (could) occure when running the tests. One sometimes expected
    exception is java.lang.SecurityException, i.e. it might be thrown, but
    not necessarily. The other does not matter, all that I am interested
    in is that an exception has occured.

    So my results would be Pass / Fail / SecurityException (Other
    exception would normally be recorded as an error. I cannot use
    @Test(expected=...), because the test would fail if no exception was
    thrown. It is preferrable not to duplicate the test methods, where one
    has an expected exception.

    Any suggestions is greatly appreciated.


    Regards,

    Ronny Mandal
     
    Ronny Mandal, Dec 10, 2010
    #1
    1. Advertising

  2. Ronny Mandal

    Ronny Mandal Guest

    On 10 Des, 10:14, Ronny Mandal <> wrote:
    > Hello, I am relatively new to JUnit. I am instrumenting already
    > existing tests, what I want to do is to record which exceptions that
    > (could) occure when running the tests. One sometimes expected
    > exception is java.lang.SecurityException, i.e. it might be thrown, but
    > not necessarily. The other does not matter, all that I am interested
    > in is that an exception has occured.
    >
    > So my results would be Pass / Fail / SecurityException (Other
    > exception would normally be recorded as an error. I cannot use
    > @Test(expected=...), because the test would fail if no exception was
    > thrown. It is preferrable not to duplicate the test methods, where one
    > has an expected exception.
    >
    > Any suggestions is greatly appreciated.

    Strange thing, the answer always emerges when the question is asked.
    WHen an exception occurs, a stack trace is yielded. Capture this,
    parse it and suddenly the task is done!

    /Ronny Mandal
    >
    > Regards,
    >
    > Ronny Mandal
     
    Ronny Mandal, Dec 10, 2010
    #2
    1. Advertising

  3. Ronny Mandal

    Lew Guest

    On 12/10/2010 05:03 AM, Ronny Mandal wrote:
    > On 10 Des, 10:14, Ronny Mandal<> wrote:
    >> Hello, I am relatively new to JUnit. I am instrumenting already
    >> existing tests, what I want to do is to record which exceptions that
    >> (could) occure when running the tests. One sometimes expected
    >> exception is java.lang.SecurityException, i.e. it might be thrown, but
    >> not necessarily. The other does not matter, all that I am interested
    >> in is that an exception has occured.
    >>
    >> So my results would be Pass / Fail / SecurityException (Other
    >> exception would normally be recorded as an error. I cannot use
    >> @Test(expected=...), because the test would fail if no exception was
    >> thrown. It is preferrable not to duplicate the test methods, where one
    >> has an expected exception.
    >>
    >> Any suggestions is greatly appreciated.

    > Strange thing, the answer always emerges when the question is asked.
    > WHen an exception occurs, a stack trace is yielded. Capture this,
    > parse it and suddenly the task is done!


    I was planning to suggest:
    http://junit.sourceforge.net/doc/faq/faq.htm#tests_7

    You can manually catch an exception within the test:

    @Test public void testFoo()
    {
    try
    {
    foo();
    }
    catch ( ExpectedException exc )
    {
    logger.log( "ExpectedException occurred", exc );
    // eat exception so test passes
    }
    assertTrue( true );
    }

    --
    Lew
     
    Lew, Dec 10, 2010
    #3
  4. Ronny Mandal

    John Guest

    On Dec 10, 2:14 am, Ronny Mandal <> wrote:
    > Hello, I am relatively new to JUnit. I am instrumenting already
    > existing tests, what I want to do is to record which exceptions that
    > (could) occure when running the tests. One sometimes expected
    > exception is java.lang.SecurityException, i.e. it might be thrown, but
    > not necessarily. The other does not matter, all that I am interested
    > in is that an exception has occured.
    >
    > So my results would be Pass / Fail / SecurityException (Other
    > exception would normally be recorded as an error. I cannot use
    > @Test(expected=...), because the test would fail if no exception was
    > thrown. It is preferrable not to duplicate the test methods, where one
    > has an expected exception.
    >
    > Any suggestions is greatly appreciated.
    >
    > Regards,
    >
    > Ronny Mandal


    JUnit has a built in way to tell whether an exception was thrown when
    it should have been:

    @Test(expected=IllegalArgumentException.class)
    public final void testThatShouldThrowIllegalArgumentException()
    {
    assertEquals(1.0, methodUnderTest(illegalValue), 0);
    }

    If the methodUnderTest(illegalValue) throws an
    IllegalArgumentException,
    then the test passes. If it doesn't throw the exception, then the test
    fails.

    Hope this helps,

    John
     
    John, Dec 10, 2010
    #4
  5. Ronny Mandal

    Lew Guest

    John wrote:
    >> Hello, I am relatively new to JUnit. I am instrumenting already
    >> existing tests, what I want to do is to record which exceptions that
    >> (could) occure when running the tests. One sometimes expected
    >> exception is java.lang.SecurityException, i.e. it might be thrown, but
    >> not necessarily. The other does not matter, all that I am interested
    >> in is that an exception has occured.

    >
    >> So my results would be Pass / Fail / SecurityException (Other
    >> exception would normally be recorded as an error. I cannot use
    >> @Test(expected=...), because the test would fail if no exception was
    >> thrown. It is preferrable not to duplicate the test methods, where one
    >> has an expected exception.

    >


    Ronny Mandal wrote:
    > JUnit has a built in way to tell whether an exception was thrown when
    > it should have been:
    >
    >      @Test(expected=IllegalArgumentException.class)
    >      public final void testThatShouldThrowIllegalArgumentException()
    >      {
    >          assertEquals(1.0, methodUnderTest(illegalValue), 0);
    >      }
    >
    > If the methodUnderTest(illegalValue) throws an
    > IllegalArgumentException,
    > then the test passes. If it doesn't throw the exception, then the test
    > fails.
    >


    He wants it to succeed when it throws no exception at all.

    --
    Lew
     
    Lew, Dec 10, 2010
    #5
  6. "Lew" <> wrote in message
    news:...
    > John wrote:
    >>> Hello, I am relatively new to JUnit. I am instrumenting already
    >>> existing tests, what I want to do is to record which exceptions that
    >>> (could) occure when running the tests. One sometimes expected
    >>> exception is java.lang.SecurityException, i.e. it might be thrown, but
    >>> not necessarily. The other does not matter, all that I am interested
    >>> in is that an exception has occured.

    >>
    >>> So my results would be Pass / Fail / SecurityException (Other
    >>> exception would normally be recorded as an error. I cannot use
    >>> @Test(expected=...), because the test would fail if no exception was
    >>> thrown. It is preferrable not to duplicate the test methods, where one
    >>> has an expected exception.

    >>

    >
    > Ronny Mandal wrote:
    >> JUnit has a built in way to tell whether an exception was thrown when
    >> it should have been:
    >>
    >> @Test(expected=IllegalArgumentException.class)
    >> public final void testThatShouldThrowIllegalArgumentException()
    >> {
    >> assertEquals(1.0, methodUnderTest(illegalValue), 0);
    >> }
    >>
    >> If the methodUnderTest(illegalValue) throws an
    >> IllegalArgumentException,
    >> then the test passes. If it doesn't throw the exception, then the test
    >> fails.
    >>

    >
    > He wants it to succeed when it throws no exception at all.


    @Test(expected=SecurityException.class)
    public final void testThatShouldThrowIllegalArgumentException()
    {
    assertEquals(1, methodThatMayThrowASecurityException());
    throw new SecurityException("Success!");
    }
     
    Mike Schilling, Dec 10, 2010
    #6
  7. Ronny Mandal

    markspace Guest

    On 12/10/2010 6:38 AM, John wrote:

    > JUnit has a built in way to tell whether an exception was thrown when
    > it should have been:
    >
    > @Test(expected=IllegalArgumentException.class)



    That is very cool, thanks for pointing that out. I've always just
    caught the exception like Lew. One advantage of the latter method is I
    think that you can test a large number of vectors in a loop. Whereas
    using the built in feature necessitates that you exit the test immediately.

    Here's an actual live example:

    for( String test: testVectors ) {
    ByteArrayInputStream ins = new ByteArrayInputStream(
    test.getBytes() );
    MultiBufferCharSeq instance = new MultiBufferCharSeq( ins, null );

    Class<IllegalArgumentException> expResult =
    IllegalArgumentException.class;
    Class<?> result = null;

    try {
    char dummy = instance.setIndex( -1 );
    } catch( IllegalArgumentException ex ) {
    result = ex.getClass();
    }
    assertSame( expResult, result );
    }
    }
     
    markspace, Dec 10, 2010
    #7
  8. Ronny Mandal

    Tom Anderson Guest

    On Fri, 10 Dec 2010, markspace wrote:

    > On 12/10/2010 6:38 AM, John wrote:
    >
    >> JUnit has a built in way to tell whether an exception was thrown when
    >> it should have been:
    >>
    >> @Test(expected=IllegalArgumentException.class)

    >
    > That is very cool, thanks for pointing that out. I've always just caught the
    > exception like Lew.


    I used to catch exceptions, then i switched to expected=, but now i catch
    exceptions again. The expected= method is very easy to go wrong with,
    because it necessarily covers the whole method. Consider:

    @Test(expected=SecurityException.class)
    public void processingAListOfForbiddenFilesThrowsAnException() throws IOException {
    File list = File.createTempFile("test", ".txt");
    Writer out = new FileWriter(list);
    out.write("/root/forbiddenfile.txt\n");
    out.close();
    new FileProcessor().processFilesInListFile(list); // should throw SecurityException
    }

    If one of the first four lines manages to throw a SecurityException, this
    test will pass, when really, we want it to be an error. If we write it
    like this:

    @Test
    public void processingAListOfForbiddenFilesThrowsAnException() throws IOException {
    File list = File.createTempFile("test", ".txt");
    Writer out = new FileWriter(list);
    out.write("/root/forbiddenfile.txt\n");
    out.close();
    try {
    new FileProcessor().processFilesInListFile(list); // should throw SecurityException
    fail("processFilesInListFile should have thrown a SecurityException");
    }
    catch(SecurityException e) {
    // success!
    }
    }

    We don't have that problem.

    Admittedly, this problem only manifests itself when your tests include
    multiple calls to 'real' code. If you're making zealous use of mocking,
    then it is much less of a problem.

    Still, i like the fact that the explicit try-catch makes it obvious
    exactly where the exception is expected to be thrown. It's one less thing
    you have to know about tests, since the testing is handled using a normal
    java structure.

    > One advantage of the latter method is I think that you can test a large
    > number of vectors in a loop. Whereas using the built in feature
    > necessitates that you exit the test immediately.
    >
    > Here's an actual live example:
    >
    > for( String test: testVectors ) {
    > ByteArrayInputStream ins = new ByteArrayInputStream(
    > test.getBytes() );
    > MultiBufferCharSeq instance = new MultiBufferCharSeq( ins, null );
    >
    > Class<IllegalArgumentException> expResult =
    > IllegalArgumentException.class;
    > Class<?> result = null;
    >
    > try {
    > char dummy = instance.setIndex( -1 );
    > } catch( IllegalArgumentException ex ) {
    > result = ex.getClass();
    > }
    > assertSame( expResult, result );
    > }
    > }


    I don't see why you couldn't do this with expected=.

    tom

    --
    william gibson said that the future has already happened, it just isn't
    evenly distributed. he was talking specifically about finsbury park. --
    andy
     
    Tom Anderson, Dec 11, 2010
    #8
  9. Ronny Mandal

    Tom Anderson Guest

    On Fri, 10 Dec 2010, Ronny Mandal wrote:

    > On 10 Des, 10:14, Ronny Mandal <> wrote:
    >
    >> Hello, I am relatively new to JUnit. I am instrumenting already
    >> existing tests, what I want to do is to record which exceptions that
    >> (could) occure when running the tests. One sometimes expected exception
    >> is java.lang.SecurityException, i.e. it might be thrown, but not
    >> necessarily. The other does not matter, all that I am interested in is
    >> that an exception has occured.


    Firstly, this doesn't sound like a good test at all. An exception 'might
    be' thrown, and that's okay? Why? Why is it okay for the test to do
    different things at different times?

    >> So my results would be Pass / Fail / SecurityException (Other exception
    >> would normally be recorded as an error. I cannot use
    >> @Test(expected=...), because the test would fail if no exception was
    >> thrown. It is preferrable not to duplicate the test methods, where one
    >> has an expected exception.
    >>
    >> Any suggestions is greatly appreciated.

    >
    > Strange thing, the answer always emerges when the question is asked.
    > WHen an exception occurs, a stack trace is yielded. Capture this,
    > parse it and suddenly the task is done!


    That doesn't sound like a good solution at all. Where are you doing the
    capturing and parsing? In the test? In general, you can't really rely on
    the stack trace in an exception, since it can vary at the whim of the
    optimiser. Plus, parsing stack traces is just a hacky thing to do.

    Now, in advance of an answer to my question above, i'm going to make an
    assumption about what's going on here. I think that you're running these
    tests in different contexts; sometimes, the test can work, but sometimes,
    something it needs to do is blocked by the security manager, and you get
    the SecurityException. The reason you don't consider this a failure is
    that it means the test could not be carried out, not that the test failed.
    In that case, you can make use of a somewhat obscure JUnit feature called
    'assumptions': an assumpion is just like an assertion, except that if it
    fails, then rather than causing the test to fail, it causes it to be
    ignored. See:

    http://junit.sourceforge.net/javadoc/org/junit/Assume.html

    You can write your test like this:

    import static org.junit.Assume.assumeNoException;

    @Test
    public void testStuff() {
    try {
    // whatever
    }
    catch (SecurityException e) {
    assumeNoException(e);
    }
    }

    If you get a SecurityException, the test will be ignored.

    A warning: be careful with these. Remember that an ignored test is not
    testing anything. Every test that ends up being ignored is something about
    your system that is not being tested. Ignores are in a way more dangerous
    than failures, because they're so easy to overlook: if you have ninety
    passing tests and ten ignored, with no failures, you tend to think your
    system is okay, whereas in reality, the features covered by the ten
    ignored tests could be disastrously buggy. In general, i think it's better
    to have test which cannot be carried out fail rather than be ignored,
    because it makes you aware of this problem. Consequently, i'd say the only
    time ignored tests are useful is if you're monitoring their number; if an
    ignored tests is a cause for concern in the same way as a failed test,
    then you can make good use of them. But for most people, ignored tests are
    just ignored.

    tom

    --
    william gibson said that the future has already happened, it just isn't
    evenly distributed. he was talking specifically about finsbury park. --
    andy
     
    Tom Anderson, Dec 11, 2010
    #9
  10. Ronny Mandal

    markspace Guest

    On 12/11/2010 6:13 AM, Tom Anderson wrote:

    > I don't see why you couldn't do this with expected=.



    Because I need to test that ALL of the vectors throw the exception, and
    the FIRST one will terminate the loop if I don't catch the exception.
     
    markspace, Dec 11, 2010
    #10
  11. Ronny Mandal

    Tom Anderson Guest

    On Sat, 11 Dec 2010, markspace wrote:

    > On 12/11/2010 6:13 AM, Tom Anderson wrote:
    >
    >> I don't see why you couldn't do this with expected=.

    >
    > Because I need to test that ALL of the vectors throw the exception, and
    > the FIRST one will terminate the loop if I don't catch the exception.


    I completely missed that when i read your code. Apologies.

    You could still use expected= if you made the test parameterized:

    http://www.junit.org/apidocs/org/junit/runners/Parameterized.html

    Which would give you a separate test execution per vector; you might find
    that gave you more detailed output as well. However, parameterized tests
    involve rather a lot of boilerplate, and i very rarely find they're worth
    it.

    tom

    --
    Know who said that? Fucking Terrorvision, that's who. -- D
     
    Tom Anderson, Dec 11, 2010
    #11
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Ahmed Moustafa
    Replies:
    5
    Views:
    30,129
    Chris Smith
    Jul 14, 2004
  2. Paul Miller
    Replies:
    3
    Views:
    1,070
    Alex Martelli
    Nov 12, 2003
  3. Replies:
    3
    Views:
    637
    Sherm Pendley
    Apr 16, 2007
  4. swetha
    Replies:
    11
    Views:
    950
    swetha
    Oct 24, 2007
  5. Lie
    Replies:
    3
    Views:
    715
Loading...

Share This Page