Course – LS – All

Get started with Spring and Spring Boot, through the Learn Spring course:

>> CHECK OUT THE COURSE

1. Overview

In this quick article, we are going to look at the @RepeatedTest annotation introduced in JUnit 5. It provides us a powerful way to write any test that we want to repeat several times.

If you want to learn more about JUnit 5, please check our other articles explaining the basics and guide to JUnit 5.

2. Maven Dependencies and Setup

The first thing to note is that JUnit 5 needs Java 8 to run. Having said that, let’s have a look at the Maven dependency:

<dependency>
    <groupId>org.junit.jupiter</groupId>
    <artifactId>junit-jupiter-engine</artifactId>
    <version>5.9.2</version>
    <scope>test</scope>
</dependency>

This is the main JUnit 5 dependency that we need to add to write our tests. Check out the latest version of the artifact here.

3. A Simple @RepeatedTest Example

Creating a repeated test is simple – just add the @RepeatedTest annotation on top of the test method:

@RepeatedTest(3)
void repeatedTest(TestInfo testInfo) {
    System.out.println("Executing repeated test");
 
    assertEquals(2, Math.addExact(1, 1), "1 + 1 should equal 2");
}

Note that instead of standard @Test annotation, we are using @RepeatedTest for our unit test. The above test will be executed three times as if the same test was written three times.

The test reports (the report files or the results in the JUnit tab of your IDE) will display all the executions:

repetition 1 of 3(repeatedTest(TestInfo))
repetition 2 of 3(repeatedTest(TestInfo))
repetition 3 of 3(repeatedTest(TestInfo))

4. Setting the failureThreshold

JUnit Jupiter version 5.10 introduces the failureThreshold as an attribute of the @RepeatedTest annotation. It helps us set the number of times a test repetition can fail before the remaining repetitions are skipped automatically.

Notably, the default value is Integer.MAX_VALUE. Also, the failure threshold must be a positive integer and be less than the repetition value.

Furthermore, repeated tests are useful in a scenario where a test may occasionally fail due to randomness or flakiness of the system under test. In the case of a flaky test, a single failure is enough to skip the remaining repetitions.

Here’s an example code that tests for randomness and skips the remaining repetition if the test fails twice:

@RepeatedTest(value = 10, failureThreshold = 2)
void whenGeneratingRandomNumber_thenNumberShouldBeWithinRange() {
    int number = random.nextInt(10);
    assertTrue(number < 8);
}

In the code above, we randomly generate an integer between zero and nine. Then, we assert that the generated number is less than eight. Also, we set the repetition value to ten and the failureThreshold to two. That means that our test will be repeated at most ten times and will stop as soon as two repetitions have failed:

random unit test with failure threshold

As for a repeated test without a threshold, the test as a whole fails if at least one repetition fails. The threshold makes sure that the test fails early and does not execute additional repetitions when we already know that the overall test will fail.

5. Lifecycle Support for @RepeatedTest

Each execution of the @RepeatedTest will behave like a regular @Test having full JUnit test life cycle support. Meaning that, during each execution, the @BeforeEach and @AfterEach methods will be called. To demonstrate this, just add the appropriate methods in the test class:

@BeforeEach
void beforeEachTest() {
    System.out.println("Before Each Test");
}

@AfterEach
void afterEachTest() {
    System.out.println("After Each Test");
    System.out.println("=====================");
}

If we run our previous test, the results will be displayed on the console:

Before Each Test
Executing repeated test
After Each Test
=====================
Before Each Test
Executing repeated test
After Each Test
=====================
Before Each Test
Executing repeated test
After Each Test
=====================

As we can see, the @BeforeEach and @AfterEach methods are called around each execution.

6. Configuring the Test Name

In the first example, we have observed that the output of the test report does not contain any identifiers. This can be configured further using the name attribute:

@RepeatedTest(value = 3, name = RepeatedTest.LONG_DISPLAY_NAME)
void repeatedTestWithLongName() {
    System.out.println("Executing repeated test with long name");
 
    assertEquals(2, Math.addExact(1, 1), "1 + 1 should equal 2");
}

The output will now contain the method name along with the repetition index:

repeatedTestWithLongName() :: repetition 1 of 3(repeatedTestWithLongName())
repeatedTestWithLongName() :: repetition 2 of 3(repeatedTestWithLongName())
repeatedTestWithLongName() :: repetition 3 of 3(repeatedTestWithLongName())

Another option is to use RepeatedTest.SHORT_DISPLAY_NAME which will produce the short name of the test:

repetition 1 of 3(repeatedTestWithShortName())
repetition 2 of 3(repeatedTestWithShortName())
repetition 3 of 3(repeatedTestWithShortName())

If however, we need to use our customized name, it is very much possible:

@RepeatedTest(value = 3, name = "Custom name {currentRepetition}/{totalRepetitions}")
void repeatedTestWithCustomDisplayName(TestInfo testInfo) {
    assertEquals(2, Math.addExact(1, 1), "1 + 1 should equal 2");
}

The {currentRepetition} and {totalRepetitions} are the placeholders for the current repetition and the total number of repetitions. These values are automatically provided by JUnit at the runtime, and no additional configuration is required. The output is pretty much what we expected:

Custom name 1/3(repeatedTestWithCustomDisplayName())
Custom name 2/3(repeatedTestWithCustomDisplayName())
Custom name 3/3(repeatedTestWithCustomDisplayName())

7. Accessing the RepetitionInfo

Apart from the name attribute, JUnit provides access to the repetition metadata in our test code as well. This is achieved by adding a RepetitionInfo parameter to our test method:

@RepeatedTest(3)
void repeatedTestWithRepetitionInfo(RepetitionInfo repetitionInfo) {
    System.out.println("Repetition #" + repetitionInfo.getCurrentRepetition());
 
    assertEquals(3, repetitionInfo.getTotalRepetitions());
}

The output will contain the current repetition index for each of the execution:

Repetition #1
Repetition #2
Repetition #3

The RepetitionInfo is provided by RepetitionInfoParameterResolver and is available only in the context of @RepeatedTest.

8. Conclusion

In this quick tutorial, we explored the @RepeatedTest annotation provided by JUnit and learned different ways of configuring it.

Don’t forget to check out the full source code for this article over on GitHub.

Course – LS – All

Get started with Spring and Spring Boot, through the Learn Spring course:

>> CHECK OUT THE COURSE
res – REST with Spring (eBook) (everywhere)
Comments are closed on this article!