A Guide to @RepeatedTest in JUnit 5

1. Overview

In this quick article, we will examine the @RepeatedTest annotation introduced in JUnit 5. It provides a powerful way to write any test that we want to repeat several times.

If you want to learn more about JUnit 5, please check our other articles explaining the basics and guide to JUnit 5.

2. Maven Dependencies and Setup

The first thing to note is that JUnit 5 needs Java 8 to run. Let’s have a look at the Maven dependency:

<dependency>
    <groupId>org.junit.jupiter</groupId>
    <artifactId>junit-jupiter-engine</artifactId>
    <version>5.11.0-M2</version>
    <scope>test</scope>
</dependency>

This is the main JUnit 5 dependency that we need to add to write our tests. Check out the latest version of the artifact here.

3. A Simple @RepeatedTest Example

Creating a repeated test is simple – just add the @RepeatedTest annotation on top of the test method:

@RepeatedTest(3)
void repeatedTest(TestInfo testInfo) {
    System.out.println("Executing repeated test");
 
    assertEquals(2, Math.addExact(1, 1), "1 + 1 should equal 2");
}

Note that instead of standard @Test annotation, we are using @RepeatedTest for our unit test. The above test will be executed three times as if the same test was written three times.

The test reports (the report files or the results in the JUnit tab of your IDE) will display all the executions:

repetition 1 of 3(repeatedTest(TestInfo))
repetition 2 of 3(repeatedTest(TestInfo))
repetition 3 of 3(repeatedTest(TestInfo))

4. Setting the failureThreshold

JUnit Jupiter version 5.10 introduces the failureThreshold as an attribute of the @RepeatedTest annotation. It helps us set the number of times a test repetition can fail before the remaining repetitions are skipped automatically.

Notably, the default value is Integer.MAX_VALUE. Also, the failure threshold must be a positive integer and be less than the repetition value.

Furthermore, repeated tests are useful when a test may occasionally fail due to randomness or flakiness of the system under test. In the case of a flaky test, a single failure is enough to skip the remaining repetitions.

Here’s an example code that tests for randomness and skips the remaining repetition if the test fails twice:

@RepeatedTest(value = 10, failureThreshold = 2)
void whenGeneratingRandomNumber_thenNumberShouldBeWithinRange() {
    int number = random.nextInt(10);
    assertTrue(number < 8);
}

We randomly generate an integer between zero and nine in the code above. Then, we assert that the generated number is less than eight. Also, we set the repetition value to ten and the failureThreshold to two. That means that our test will be repeated at most ten times and will stop as soon as two repetitions have failed:

A repeated test without a threshold fails if at least one repetition fails. The threshold ensures that the test fails early and does not execute additional repetitions when we already know that the overall test will fail.

5. Lifecycle Support for @RepeatedTest

Each execution of the @RepeatedTest will behave like a regular @Test with full JUnit test life cycle support. This means that, during each execution, the @BeforeEach and @AfterEach methods will be called. To demonstrate this, add the appropriate methods in the test class:

@BeforeEach
void beforeEachTest() {
    System.out.println("Before Each Test");
}

@AfterEach
void afterEachTest() {
    System.out.println("After Each Test");
    System.out.println("=====================");
}

If we run our previous test, the results will be displayed on the console:

Before Each Test
Executing repeated test
After Each Test
=====================
Before Each Test
Executing repeated test
After Each Test
=====================
Before Each Test
Executing repeated test
After Each Test
=====================

As we can see, the @BeforeEach and @AfterEach methods are called around each execution.

6. Configuring the Test Name

In the first example, we have observed that the output of the test report does not contain any identifiers. This can be configured further using the name attribute:

@RepeatedTest(value = 3, name = RepeatedTest.LONG_DISPLAY_NAME)
void repeatedTestWithLongName() {
    System.out.println("Executing repeated test with long name");
 
    assertEquals(2, Math.addExact(1, 1), "1 + 1 should equal 2");
}

The output will now contain the method name along with the repetition index:

repeatedTestWithLongName() :: repetition 1 of 3(repeatedTestWithLongName())
repeatedTestWithLongName() :: repetition 2 of 3(repeatedTestWithLongName())
repeatedTestWithLongName() :: repetition 3 of 3(repeatedTestWithLongName())

Another option is to use RepeatedTest.SHORT_DISPLAY_NAME, which will produce the short name of the test:

repetition 1 of 3(repeatedTestWithShortName())
repetition 2 of 3(repeatedTestWithShortName())
repetition 3 of 3(repeatedTestWithShortName())

If however, we need to use our customized name, it is very much possible:

@RepeatedTest(value = 3, name = "Custom name {currentRepetition}/{totalRepetitions}")
void repeatedTestWithCustomDisplayName(TestInfo testInfo) {
    assertEquals(2, Math.addExact(1, 1), "1 + 1 should equal 2");
}

The {currentRepetition} and {totalRepetitions} are the placeholders for the current repetition and the total number of repetitions. These values are automatically provided by JUnit at the runtime, and no additional configuration is required. The output is pretty much what we expected:

Custom name 1/3(repeatedTestWithCustomDisplayName())
Custom name 2/3(repeatedTestWithCustomDisplayName())
Custom name 3/3(repeatedTestWithCustomDisplayName())

7. Accessing the RepetitionInfo

Apart from the name attribute, JUnit also provides access to the repetition metadata in our test code. This is achieved by adding a RepetitionInfo parameter to our test method:

@RepeatedTest(3)
void repeatedTestWithRepetitionInfo(RepetitionInfo repetitionInfo) {
    System.out.println("Repetition #" + repetitionInfo.getCurrentRepetition());
 
    assertEquals(3, repetitionInfo.getTotalRepetitions());
}

The output will contain the current repetition index for each of the execution:

Repetition #1
Repetition #2
Repetition #3

The RepetitionInfo is provided by RepetitionInfoParameterResolver and is available only in the context of @RepeatedTest.

8. Conclusion

In this quick tutorial, we explored the @RepeatedTest annotation provided by JUnit and learned different ways of configuring it.