Monday, July 15, 2013

3 ways of handling exceptions in JUnit. Which one to choose?

In JUnit there are many ways of handling exceptions in your test code:
  • try-catch idiom
  • With JUnit rule
  • With @Test annotation
  • With catch-exception library
  • With custom annotation
Which one should we use and when?

try-catch idiom

This idiom is one of the most popular one, because it was used already in JUnit 3.

    @Test
    public void throwsExceptionWhenNegativeNumbersAreGiven() {
        try {
            calculator.add("-1,-2,3");
            fail("Should throw an exception if one or more of given numbers are negative");
        } catch (Exception e) {
            assertThat(e)
                    .isInstanceOf(IllegalArgumentException.class)
                    .hasMessage("negatives not allowed: [-1, -2]");
        }
    }

The above approach is a common pattern. The test will fail when no exception is thrown and the exception itself is verified in a catch clause (in the above example I used the FEST Fluent Assertions) and although it is perfectly fine I prefer the approach with ExpectedException rule.

With JUnit rule

The same example can be created using ExceptedException rule. The rule must be a public field marked with @Rule annotation. Please note that the "thrown" rule may be reused in many tests.

    @Rule
    public ExpectedException thrown = ExpectedException.none();

    @Test
    public void throwsExceptionWhenNegativeNumbersAreGiven() {
        // arrange
        thrown.expect(IllegalArgumentException.class);
        thrown.expectMessage(equalTo("negatives not allowed: [-1, -2]"));
        // act
        calculator.add("-1,-2,3");
    }

In general, I find the above code more readable hence I use this approach in my projects.

When the exception isn't thrown you will get the following message: java.lang.AssertionError: Expected test to throw (an instance of java.lang.IllegalArgumentException and exception with message "negatives not allowed: [-1, -2]"). Pretty nice.

But not all exceptions I check with the above approach. Sometimes I need to check only the type of the exception thrown and then I use @Test annotation.

With annotation


    @Test (expected = IllegalArgumentException.class)
    public void throwsExceptionWhenNegativeNumbersAreGiven() {
        // act
        calculator.add("-1,-2,3");
    }

When the exception wasn't thrown you will get the following message: java.lang.AssertionError: Expected exception: java.lang.IllegalArgumentException

With this approach you need to be careful though. Sometimes it is tempting to expect general Exception, RuntimeException or even a Throwable. And this is considered as a bad practice, because your code may throw exception in other place than you actually expected and your test will still pass!

To sum up, in my code I use two approaches: with JUnit rule and with annotation. The advantages are:

  • Error messages when the code does not throw an exception are automagically handled
  • The readability is improved
  • There is less code to be created

catch-exceptions

In short, catch-exception is a library that catches exceptions in a single line of code and makes them available for further analysis.

Please read a separate post I created about it here: Yet another way to handle exceptions in JUnit: catch-exception

Custom annotation

I have heard of another way of handling the exception (one of my colleagues suggested it after reading this post) - use custom annotation.

Actually the solution seems nice at first glance, but it requires your own JUnit runner hence it has disadvantage: you cannot use this annotation with e.g. Mockito runner.

As a coding practice I have created such an annotation, so maybe someone finds it useful

The usage

@RunWith(ExpectsExceptionRunner.class)
public class StringCalculatorTest {
    @Test
    @ExpectsException(type = IllegalArgumentException.class, message = "negatives not allowed: [-1]")
    public void throwsExceptionWhenNegativeNumbersAreGiven() throws Exception {
        // act
        calculator.add("-1,-2,3");
    }
  
}

The above test will fail with a message: java.lang.Exception: Unexpected exception message, expected but was

An annotation


@Retention(RetentionPolicy.RUNTIME)
@Target({ElementType.METHOD})
public @interface ExpectsException {
    Class<? extends Throwable> type();

    String message() default "";
}


A runner with some copy & paste code


public class ExpectsExceptionRunner extends BlockJUnit4ClassRunner {
    public ExpectsExceptionRunner(Class<?> klass) throws InitializationError {
        super(klass);
    }

    @Override
    protected Statement possiblyExpectingExceptions(FrameworkMethod method, Object test, Statement next) {
        ExpectsException annotation = method.getAnnotation(ExpectsException.class);
        if (annotation == null) {
            return next;
        }
        return new ExpectExceptionWithMessage(next, annotation.type(), annotation.message());
    }

    class ExpectExceptionWithMessage extends Statement {

        private final Statement next;
        private final Class<? extends Throwable> expected;
        private final String expectedMessage;

        public ExpectExceptionWithMessage(Statement next, Class<? extends Throwable> expected, String expectedMessage) {
            this.next = next;
            this.expected = expected;
            this.expectedMessage = expectedMessage;
        }

        @Override
        public void evaluate() throws Exception {
            boolean complete = false;
            try {
                next.evaluate();
                complete = true;
            } catch (AssumptionViolatedException e) {
                throw e;
            } catch (Throwable e) {
                if (!expected.isAssignableFrom(e.getClass())) {
                    String message = "Unexpected exception, expected<"
                            + expected.getName() + "> but was <"
                            + e.getClass().getName() + ">";
                    throw new Exception(message, e);
                }

                if (isNotNull(expectedMessage) && !expectedMessage.equals(e.getMessage())) {
                    String message = "Unexpected exception message, expected<"
                            + expectedMessage + "> but was<"
                            + e.getMessage() + ">";
                    throw new Exception(message, e);
                }
            }
            if (complete) {
                throw new AssertionError("Expected exception: "
                        + expected.getName());
            }
        }

        private boolean isNotNull(String s) {
            return s != null && !s.isEmpty();
        }
    }

}

And what is your preference?

2 comments:

  1. You should check out the following cool project that simplifies exception handling in unit tests

    http://code.google.com/p/catch-exception/

    ReplyDelete
    Replies
    1. Quite late, but thanks for the suggestion. I had a look at this library. Looks really nice!

      Posted shortly about it: http://blog.codeleak.pl/2014/04/yet-another-way-to-handle-exceptions-in.html

      Thanks!

      Delete