Using the ScalaTest Runner

1. Introduction

ScalaTest is one of the most popular testing frameworks in the Scala ecosystem. It provides a tool called Runner to execute the tests from the command line. Additionally, the ScalaTest Runner provides a lot of flexibility for specifying which of our tests to run.

In this tutorial, we’ll look at different popular options to run the tests using Runner.

2. Setup

Let’s first add the relevant dependency to the build.sbt file:

libraryDependencies += "org.scalatest" %% "scalatest" % "3.2.15" % Test

Next, let’s create some sample tests to try out the various Runner options:

class ScalaTestRunnerTests extends AnyWordSpec with Matchers {
  "Scalatest runnner" should {
    "convert string true to boolean true" in {
      "true".toBoolean shouldBe true
    }
    "convert string false to boolean false" in {
      "false".toBoolean shouldBe false
    }
    "trim the spaces to the right and left" in {
      " with spaces ".trim() shouldBe "with spaces"
    }
  }
}

3. Specifying the Tests to Run

ScalaTest provides many different command-line flags to run a specific set of tests instead of executing everything. In this section, we’ll explore some of the common and useful arguments to run a subset of tests.

3.1. Run All Tests

We can run all the tests in the project using the command:

sbt test

3.2. Run a Test Class by Full Name

We can run all the tests in a test class by providing the full package path:

sbt 'test:testOnly com.baeldung.scalatest.runner.ScalaTestRunnerTests'

We should note that we need to wrap the sbt commands in quotes to pass the entire set of arguments to sbt. 

3.3. Run Test Class by Wildcard Match

We can use the wildcard format with the class name instead of the full package path:

sbt 'test:testOnly *ScalaTestRunnerTests'

This runs all the tests with class names ending with ScalaTestRunnerTests across all the packages.

Similarly, we can apply the wildcard on any part of the class name to run all the matching tests. We can run all the tests with class names containing the word Runner using the command:

sbt 'test:testOnly *Runner*'

3.4. Run Tests by Wildcard Matching the Testcase Name

We can run only a few of the tests within a class by providing the wildcard match. To provide additional flags, we need to separate the class name and the arguments using —. After that, we can run the tests that contain a specific substring using the flag -z :

sbt 'test:testOnly *ScalaTestRunnerTests -- -z "convert"'

As a result, this executes all the tests within ScalaTestRunnerTests that contain the text convert.

3.5. Run Tests by Tag

ScalaTest provides the flexibility to tag tests with some specific values. We can create a tag by creating an object and extending it with the class Tag:

object BooleanTests extends Tag("BooleanTests")

Then, we can mark the required tests with this tag using the method taggedAs():

"convert string true to boolean true" taggedAs (BooleanTests) in {}

After that, we can run all the tests tagged with this particular tag using the flag -n:

sbt 'test:testOnly *ScalaTestRunnerTests -- -n "BooleanTests"'

3.6. Excluding Tests by Tag

We can also exclude a particular set of tests by using tags. Instead of using -n, we can use -l to exclude the tests with the specified tag:

sbt 'test:testOnly *ScalaTestRunnerTests -- -l "BooleanTests"'

4. Reporting

Apart from just executing the tests, ScalaTest Runner also allows reporting the failures in a more understandable way.

4.1. Showing Consolidated Failed Tests at the End

Generally, ScalaTest shows all the test statuses in the order of execution. Runner provides an additional flag to show a consolidated list of all the failed tests at the end of the execution:

sbt 'test:testOnly *ScalaTestRunnerTests -- -eI'

We can use -eT and -eG flags instead of -eI. These two flags add more stack trace details to the error messages in the test, whereas -eI only shows the failed test name at the end.

5. Configuring Runner Flags in build.sbt

Instead of always providing the flags explicitly when running the tests, we can also provide the default configurations within build.sbt. We can provide the options using sbt method testOptions:

testOptions in Test += Tests.Argument(TestFrameworks.ScalaTest, "-eG")

6. Conclusion

In this article, we looked at some of the most common and useful configuration options provided by the ScalaTest Runner.

As always, the sample code used in this article is available over on GitHub.