I just announced the new Spring Boot 2 material, coming in REST With Spring:

>> CHECK OUT THE COURSE

1. Introduction

Although executing tests serially works just fine most of the times. Sometimes we may need to speed things up. In these situations, parallel tests can help.

JUnit 4.7 and later versions make it possible to execute tests in parallel using Maven’s Surefire Plugin. In a nutshell, Surefire provides two ways of executing tests in parallel:

  • The first approach uses multithreading inside a single JVM process
  • While the second approach uses multiple JVM processes

In this tutorial, we’ll cover how to configure Surefire to run JUnit tests in parallel inside a single JVM process. Then we’ll see how to run tests in parallel within a Maven multimodule project.

2. Maven Dependencies

Let’s begin by importing the required dependencies. We’ll need to use JUnit 4.7 or later along with Surefire 2.16 or later:

<dependency>
    <groupId>junit</groupId>
    <artifactId>junit</artifactId>
    <version>4.12</version>
    <scope>test</scope>
</dependency>
<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-surefire-plugin</artifactId>
    <version>2.22.0</version>
</plugin>

3. Running Parallel Tests

To run a test in parallel we should use a test runner that extends org.junit.runners.ParentRunner.

However, even tests that don’t declare an explicit test runner work, as the default runner extends this class.

Next, to demonstrate parallel test execution, we’ll use a test suite with two test classes each having a few methods. In fact, any standard implementation of a JUnit test suite would do.

3.1. Using Parallel Parameter

First, let’s enable parallel behavior in Surefire using the parallel parameter. It states the level of granularity at which we would like to apply parallelism.

The possible values are:

  • methods – runs test methods in separate threads
  • classes – runs test classes in separate threads
  • classesAndMethods – runs classes and methods in separate threads
  • suites – runs suites in parallel
  • suitesAndClasses – runs suites and classes in separate threads
  • suitesAndMethods – creates separate threads for classes and for methods
  • all – runs suites, classes as well as methods in separate threads

In our example, we use all:

<configuration>
    <parallel>all</parallel>
</configuration>

Second, let’s define the total number of threads we want Surefire to create. We can do that in two ways:

Using threadCount which defines the maximum number of threads Surefire will create:

<threadCount>10</threadCount>

Or using useUnlimitedThreads parameter where one thread is created per CPU core:

<useUnlimitedThreads>true</useUnlimitedThreads>

By default, threadCount is per CPU core. We can use the parameter perCoreThreadCount to enable or disable this behavior:

<perCoreThreadCount>true</perCoreThreadCount>

3.2. Using Thread-Count Limitations

Now, let’s say we want to define the number of threads to create at the method, class and suite level. We can do this with the threadCountMethods, threadCountClasses and threadCountSuites parameters.

Let’s combine these parameters with threadCount from the previous configuration: 

<threadCountSuites>2</threadCountSuites>
<threadCountClasses>2</threadCountClasses>
<threadCountMethods>6</threadCountMethods>

Since we used all in parallel, we’ve defined the thread counts for methods, suites as well as classes. However, it isn’t mandatory to define the leaf parameter. Surefire deduces the number of threads to use in case leaf parameters are omitted.

For example, if threadCountMethods is omitted, then we just need to make sure threadCount > threadCountClasses threadCountSuites.

Sometimes we may want to limit the number of threads created for classes or suites or methods even while we are using an unlimited number of threads.

We can apply thread-count limitations in such cases as well:

<useUnlimitedThreads>true</useUnlimitedThreads>
<threadCountClasses>2</threadCountClasses>

3.3. Setting Timeouts

Sometimes we may need to ensure that test execution is time-bounded.

To do that we can use the parallelTestTimeoutForcedInSeconds parameter. This will interrupt currently running threads and will not execute any of the queued threads after the timeout has elapsed:

<parallelTestTimeoutForcedInSeconds>5</parallelTestTimeoutForcedInSeconds>

Another option is to use parallelTestTimeoutInSeconds.

In this case, only the queued threads will be stopped from executing:

<parallelTestTimeoutInSeconds>3.5</parallelTestTimeoutInSeconds>

Nevertheless, with both options, the tests will end with an error message when the timeout has elapsed.

3.4. Caveats

Surefire calls static methods annotated with @Parameters, @BeforeClass, and @AfterClass in the parent thread. Thus make sure to check for potential memory inconsistencies or race conditions before running tests in parallel.

Also, tests that mutate shared state are definitely not good candidates for running in parallel.

4. Test Execution in Multi-Module Maven Projects

Till now, we have focused on running tests in parallel within a Maven module.

But let’s say we have multiple modules in a Maven project. Since these modules are built sequentially, the tests for each module are also executed sequentially.

We can change this default behavior by using Maven’s -T parameter which builds modules in parallel. This can be done in two ways.

We can either specify the exact number of threads to use while building the project:

mvn -T 4 surefire:test

Or use the portable version and specify the number of threads to create per CPU core:

mvn -T 1C surefire:test

Either way we can speed up tests as well as build execution times.

5. Conclusion

To sum up, we started off by enabling multi-threaded behavior and defining the degree of parallelism using the parallel parameter. Subsequently, we applied limitations on the number of threads Surefire should create. Later, we set timeout parameters to control test execution times.

Finally, we looked at how we can reduce build execution times and therefore test execution times in multi-module Maven projects.

As always, the code presented here is available on GitHub.

I just announced the new Spring Boot 2 material, coming in REST With Spring:

>> CHECK OUT THE LESSONS

newest oldest most voted
Notify of
Jan
Guest
Jan

I cloned this code and ran mvn test and the tests did not run in parallel. I added sleeps to make sure that all tests should be started before any finishes. Can you confirm parallel running?

Loredana Crusoveanu
Editor

Hey Jan,

Yes, this works for me. I added sleeps of 4 seconds in each method in ArithmeticFunctionTest and the total time of the test was still aprox 4 seconds.

For the multi-module run, I can confirm the same thing.

As for “all tests should be started before any finishes” – I don’t think that’s a guarantee. Each test starts when there is an available thread.