Conditionally Run or Ignore Tests in JUnit 4

1. Overview

Let’s imagine we have a test for some code that depends on the Operating System and should run only if our test machine is running on Linux. If it’s running on any other OS, we want the test not to fail, but to be ignored at runtime.

A first approach could be using a couple of if statements to check for this condition using System class properties. This works, of course, but JUnit has a cleaner, more elegant method.

In this short tutorial, we’re going to look at how we can conditionally run or ignore tests in JUnit 4 using the Assume class.

2. The Assume Class

This class provides a set of methods to support conditional test execution based on certain conditions. Our test will only run if all these conditions are met. If not, JUnit will just skip its execution and mark it as passed in the test report. The latter is the main difference with the Assert class, in which a failing condition leads the test to end as failing.

An important thing to note is that the behavior we described for the Assume class is exclusive to the default JUnit runner. With custom runners, things may be different.

Finally, in the same way as with Assert, we can call the Assume methods either in the @Before or @BeforeClass annotated methods or within the @Test method itself.

Let’s now go through the most useful methods of the Assume class by showing some examples. For all the following examples, let’s assume getOsName() returns Linux.

2.1. Using assumeThat

The assumeThat() method checks that the state – in this case, getOsName() – satisfies the conditions of the matcher passed in:

@Test
public void whenAssumeThatAndOSIsLinux_thenRunTest() {
    assumeThat(getOsName(), is("Linux"));

    assertEquals("run", "RUN".toLowerCase());
}

In this example, we checked whether getOsName() equals to Linux. As getOsName() returns Linux, the test will be run. Note, we’re using the Hamcrest matcher method is(T) as the matcher here.

2.2. Using assumeTrue

Similarly, we can use the assumeTrue() method to specify a boolean expression that must evaluate to true in order for the test to run. If it evaluates to false, the test will be ignored:

private boolean isExpectedOS(String osName) {
    return "Linux".equals(osName);
}

@Test 
public void whenAssumeTrueAndOSIsLinux_thenRunTest() {
    assumeTrue(isExpectedOS(getOsName()));
 
    assertEquals("run", "RUN".toLowerCase());
}

In this case, isExpectedOs() returns true. Therefore, the conditions for the test to run have been met, and the test will be run.

2.3. Using assumeFalse

Finally, we can use the opposite assumeFalse() method to specify a boolean expression that must evaluate to false in order for the test to run. If it evaluates to true, the test will be ignored:

@Test
public void whenAssumeFalseAndOSIsLinux_thenIgnore() {
    assumeFalse(isExpectedOS(getOsName()));

    assertEquals("run", "RUN".toLowerCase());
}

In this case, as isExpectedOs() also returns true, the conditions for the test to run have not been met, and the test will be ignored.

2.4. Using assumeNotNull

When we want to ignore a test if some expression is null, we can use the assumeNotNull() method:

@Test
public void whenAssumeNotNullAndNotNullOSVersion_thenRun() {
    assumeNotNull(getOsName());

    assertEquals("run", "RUN".toLowerCase());
}

As getOsName() is returning a non-null value, the condition for the test to run has been satisfied and the test will run.

2.5. Using assumeNoException

Finally, we could want to ignore a test if an exception is thrown. We can use assumeNoException() for this purpose:

@Test
public void whenAssumeNoExceptionAndExceptionThrown_thenIgnore() {
    assertEquals("everything ok", "EVERYTHING OK".toLowerCase());
    String t=null;
    try {
        t.charAt(0);
    } catch(NullPointerException npe){
        assumeNoException(npe);
    }
    assertEquals("run", "RUN".toLowerCase());
}

In this example, as t is null, a NullPointerException exception is thrown, therefore the conditions for the test to run have not been met, and the test will be ignored.

3. Where Should We Put the assumeXXX Call?

It’s important to note that the behavior of the assumeXXX methods depends on where we put them in our tests.

Let’s slightly modify our assumeThat example so the assertEquals() call goes first. Also, let’s make the assertEquals() fail:

@Test
public void whenAssumeFalseAndOSIsLinux_thenIgnore() {
    assertEquals("run", "RUN");
    assumeFalse(isExpectedOS(getOsName()));
}

When we run this example, we’ll have:

org.junit.ComparisonFailure: 
Expected :run
Actual   :RUN

In this case, our test is not ignored because it has failed before we reached the assumeThat() call. The same happens with all of the assumeXXX methods. So, we need to make sure we put them in the right place inside our test method.

4. Conclusion

In this short tutorial, we’ve seen how we can conditionally decide whether or not a test should run, using the Assume class in JUnit 4. In case we are using JUnit 5, it’s also available in version 5.4 or later.