Introduction to Testing With ScalaTest

Last updated: March 18, 2024

Reviewed by: Michal Aibin

 1. Overview

ScalaTest is one of the most popular, complete and easy-to-use testing frameworks in the Scala ecosystem.

Where ScalaTest differs from other testing tools is its ability to support a number of different testing styles such as XUnit and BDD out of the box.

In this introductory tutorial, we’ll start by creating our first test before examining its support for XUnit and BDD. We’ll also explore several other key features including matchers and mocking.

Along the way, we’ll demonstrate how concise and understandable our resulting code can be.

2. Dependencies 

Setting up ScalaTest is pretty straightforward; we only need to add the scalatest dependency to our build.sbt file:

libraryDependencies += "org.scalatest" %% "scalatest" % "3.2.15" % "test"

Alternatively, if we happen to have a Maven project, we can add ScalaTest to our pom.xml:

<dependency>
  <groupId>org.scalatest</groupId>
  <artifactId>scalatest_3</artifactId>
  <version>3.2.15</version>
  <scope>test</scope>
</dependency>

As always, we can get the latest version from Maven Central.

3. Key Concepts

To help give us a bit of context and before we dive into some real examples, let’s highlight a few of the key concepts of ScalaTest:

  • At the heart of ScalaTest is the concept of a suite, which is simply a collection of zero or more tests
  • In fact, the Scala trait Suite declares several lifecycle methods that define the way we can write and run tests
  • Thankfully ScalaTest already offers many style traits that extend Suite to support different testing styles
  • On top of this, we can mix these style traits together with any number of what ScalaTest terms as stackable traits, such as Matchers or BeforeAndAfter
  • Using this approach, we can quickly build up coherent, readable and focussed tests
  • Finally, it is of course perfectly possible for us to build on top of any of ScalaTest’s style or stackable traits outside of ScalaTest

4. Creating Our First Unit Test

Now that we have ScalaTest configured and understand some of the philosophy behind the framework, let’s start by defining a simple unit test for testing a List:

class ListFunSuite extends AnyFunSuite {

  test("An empty List should have size 0") {
    assert(List.empty.size == 0)
  }

}

Here, our test ListFunSuite extends the AnyFunSuite trait. This trait is typically aimed at people with xUnit experience and lets us organize our tests into easy to understand test() blocks with descriptive test names.

In the above example, we’re simply asserting that an empty List has a size of zero. Now let’s run our test using sbt:

sbt:scala-tutorials> testOnly com.baeldung.scala.scalatest.ListFunSuite
...
[info] Done compiling.
[info] ListFunSuite:
[info] - An empty List should have size 0
[info] ScalaTest
[info] Run completed in 87 milliseconds.
[info] Total number of tests run: 1
[info] Suites: completed 1, aborted 0
[info] Tests: succeeded 1, failed 0, canceled 0, ignored 0, pending 0
[info] All tests passed.
[info] Passed: Total 1, Failed 0, Errors 0, Passed 1
[success] Total time: 1 s, completed Mar 25, 2020 4:32:17 PM

As expected, our test passes and sbt prints a brief test report summary.

Now we can go ahead and populate our test suite with more advanced tests.

For example, sometimes we also need to test whether a method throws an expected exception. We can do this with assertThrows:

test("Accessing invalid index should throw IndexOutOfBoundsException") {
  val fruit = List("Banana", "Pineapple", "Apple");
  assert(fruit.head == "Banana")
  assertThrows[IndexOutOfBoundsException] {
    fruit(5)
  }
}

In this test, we’re first checking the first item in our fruit list is a Banana and then we check that that we receive an IndexOutOfBoundsException when we try to access an invalid index in our list of fruits.

5. Using Different Testing Styles

As we’ve previously mentioned, ScalaTest supports several different testing styles out of the box. In the last section, we saw how to use the FunSuite trait. In this section, we’ll take a look at a few of the other more popular styles.

5.1. Employing AnyFlatSpec

The main premise behind the AnyFlatSpec trait is to help facilitate a BDD style of development. It’s named flat because the structure of the tests we write is unnested in nature. In addition, this trait tries to guide us into writing more focused tests with descriptive, specification-style names.

In actual fact, the ScalaTest project recommends using AnyFlatSpec as the default choice for writing unit tests. With that in mind, let’s reimplement our simple List example, but this time, extending from AnyFlatSpec:

class ListFlatSpec extends AnyFlatSpec {

  "An empty List" should "have size 0" in {
    assert(List.empty.size == 0)
  }

  it should "throw an IndexOutOfBoundsException when trying to access any element" in {
    val emptyList = List();
    assertThrows[IndexOutOfBoundsException] {
      emptyList(1)
    }
  }
}

Our tests now read more like a specification, that is, “X should Y”. And, arguably the descriptions have a more natural flow.

One important difference with respect to our AnyFunSuite example is that when we use it – this is an alias to the previous name (in this case, “An empty List”)

When we run our ListFlatSpec test, we’ll see this more obviously in the output:

...
[info] ListFlatSpec:
[info] An empty List
[info] - should have size 0
[info] - should throw an IndexOutOfBoundsException when trying to access any element
...

5.2. Working With AnyFunSpec

We can take this BDD style of testing a step further using AnyFunSpec:

class ListFunSpec extends FunSpec {

  describe("A List") {
    describe("when empty") {
      it("should have size 0") {
        assert(List.empty.size == 0)
      }

      it("should throw an IndexOutOfBoundsException when to access an element") {
        val emptyList = List();
        assertThrows[IndexOutOfBoundsException] {
          emptyList(1)
        }
      }
    }
  }
}

This kind of syntax should feel familiar to people working with RSpec or Javascript. The main difference here is that the structure of our tests is provided by nesting and written using describe and it.

We can see the nesting more evidently when we run our ListFunSpec test:

[info] ListFunSpec:
[info] A List
[info]   when empty
[info]   - should have size 0
[info]   - should throw an IndexOutOfBoundsException when to access an element

To quickly summarise, we can use AnyFunSpec for BDD-style testing. This trait provides an excellent general-purpose choice for writing specification-style tests.

For the rest of our examples unless explicitly stated we’ll stick with AnyFlatSpec.

6. Before and After

More often than not, we’ll want to execute some common code before and after each test. This helps us minimize duplicate code in our tests and allows us to set up resources such as files or database connections. Or, we may wish to simply share some variables between our test methods.

In ScalaTest, we can mix in the BeforeAndAfter trait and then simply register the code we want to run before each test. To achieve this, we simply declare a before clause and an after clause that will register code to be run:

class StringFlatSpecWithBeforeAndAfter extends AnyFlatSpec with BeforeAndAfter {

  val builder = new StringBuilder;

  before {
    builder.append("Baeldung ")
  }

  after {
    builder.clear()
  }
  
  // ...

In this simple example, we create a StringBuilder instance val which is shared between our tests and before each test, we simply append some text, and after each test, we clear the builder variable.

Then we can populate our tests making use of the builder variable directly:

"Baeldung" should "be interesting" in {
  assert(builder.toString === "Baeldung ")

  builder.append("is very interesting!")
  assert(builder.toString === "Baeldung is very interesting!")
}

it should "have great tutorials" in {
  assert(builder.toString === "Baeldung ")

  builder.append("has great tutorials!")
  assert(builder.toString === "Baeldung has great tutorials!")
}

We should note that the only way before and after code can communicate with test code is via some side-effecting mechanism. For example, we just saw that we can do it by changing the state of mutable objects held from instance vals.

7. Working With Matchers

Up until now, we’ve used standard assertions in our tests which are available by default in any style trait. However, ScalaTest provides a powerful domain-specific language (DSL) for expressing assertions in tests using the word should. 

To use this, we can simply mix in the Matchers trait:

import org.scalatest.flatspec.AnyFlatSpec
import scalatest.matchers.should.Matchers

class ExampleFlatSpecWithMatchers extends AnyFlatSpec with Matchers {
...

7.1. Basic Matchers

Let’s now see some basic matchers in practice. We’ll start with some simple equality checks:

"A matcher" should "let us check equality" in {
  val number = 25
  number should equal (25)
  number shouldEqual 25
}

it should "also let us check equality using be" in {
  val number = 25
  number should be (25)
  number shouldBe 25
}

Generally speaking, we have two main ways we can check for equality.

In the first test above, we use equal and shouldEqual. The only difference here being that shouldEqual does not require parentheses for the argument value. Likewise, in the second test, we see the be and shouldBe matcher, using an argument with and without parentheses.

However, there is a subtle difference between the first test and the second test. When we use the *equal variation, we have the ability to customize the equality check as these constructs take an implicit Equality[T] to verify the computed value with the expected value.

Let’s see a simple example to understand what we mean:

it should "also let us customize equality" in {
  " baeldung  " should equal("baeldung")(after being trimmed)
}

The after being trimmed expression results in an Equality[String], which is then passed explicitly as the second parameter to equal.

Using the second variation with be, we cannot customize the equality check. As a rule of thumb, if we simply want to compare values, then should be is sufficient. It’s also the fastest to compile.

We’ll also frequently need to check the size and length when testing, only where it makes sense, of course:

it should "let us check the length of strings" in {
  val result = "baeldung"
  result should have length 8
}

We can also check the size of collections:

it should "let us check the size of collections" in {
  val days = List("Monday", "Tuesday", "Wednesday", "Thursday", "Friday", "Saturday", "Sunday")
  days should have size 7
}

Although not a concrete rule, a good way to decide whether to use length or size is to think that any object that has a field or method named length or a method named getLength is applicable for ‘should have length‘. Similarly, the same rule can be adapted for the size syntax.

The type of a length or size field, or return type of a method, must be either Int or Long.

7.2. Checking Strings

At some point, almost certainly we’ll want to check string values from within our tests. Using the Matchers mixin, we have support for a number of operations.

Let’s start with some simple string checks:

it should "let us check different parts of a string" in {
  val headline = "Baeldung is really cool!"
  headline should startWith("Baeldung")
  headline should endWith("cool!")
  headline should include("really")
}

Now let’s imagine we want to check an email address is valid:

it should "let us check that an email is valid" in {
  "[email protected]" should fullyMatch regex """[^@]+@[^\.]+\..+"""
}

As we can see, we can effortlessly make stricter checks using regular expressions.

7.3. Other Useful Matchers

In this section, we’ll take a very quick look at a few more valuable features of the Matchers trait. Sometimes, we might want to check for emptiness.

For this, ScalaTest provides the empty token:

it should "let us check a list is empty" in {
  List.empty shouldBe empty
}

Likewise, we can also check for the inverse using the not predicate together with empty:

it should "let us check a list is NOT empty" in {
  List(1, 2, 3) should not be empty
}

In our last couple of examples, we’ll see how we can check if a collection contains a given element, and afterward, we’ll see how to check the type of an object which can occasionally be required.

it should "let us check a map contains a given key and value" in {
  Map('x' -> 10, 'y' -> 20, 'z' -> 30) should contain('y' -> 20)
}

To conclude this section, let’s see how to check the type of an object:

it should "let us check the type of an object" in {
  List(1, 2, 3) shouldBe a[List[_]]
  List(1, 2, 3) should not be a[Map[_, _]]
}

In this section, we’ve seen how we can incorporate the impressive should DSL using Matchers. This style has an informal feel and makes our code easier to read, flowing almost like a conversation.

In this tutorial, we’ve only touched the surface of this important trait. For more information and examples, please see the full documentation.

8. Tagging Tests

Something that we all need to do from time to time is switch off a unit test temporarily, perhaps when diagnosing an intermittent failure.

By default, ScalaTest supports one tag ignore which we can use to accomplish precisely this. All we need to do is simply replace the it in our test case definition:

ignore should "let us check a list is empty" in {
  List.empty shouldBe empty
}

When we run our test suite now we’ll clearly see this in the output:

sbt:scala-tutorials> testOnly com.baeldung.scala.scalatest.ExampleFlatSpecWithMatchers
...
[info] - should let us check that an email is valid
[info] - should let us check a list is empty !!! IGNORED !!!
[info] - should let us check a list is NOT empty
...

Furthermore, we also have the possibility to tag our tests in a different way using taggedAs:

object BaeldungJavaTag extends Tag("com.baeldung.scala.scalatest.BaeldungJavaTag")

class TaggedFlatSpec extends AnyFlatSpec with Matchers {

  "Baeldung" should "be interesting" taggedAs (BaeldungJavaTag) in {
    "Baeldung has articles about Java" should include("Java")
  }
}

In this short example, we have defined our own BaeldungJavaTag, which we then use in our test definition after the taggedBy clause.

Then we have the possibility of running only test cases tagged with BaeldungJavaTag:

sbt "testOnly -- -n com.baeldung.scala.scalatest.BaeldungJavaTag"
...
[info] All tests passed.
[info] Passed: Total 1, Failed 0, Errors 0, Passed 1

Typically we might want to tag slow tests or tests which we might not want to run as frequently as others. ScalaTest makes this easy using the above approach.

9. Mocking

ScalaTest permits the usage of any Java mocking framework, or we can use ScalaMock which is a native Scala mocking library. Here, we’ll take a quick look at how to use ScalaMock from ScalaTest.

First, of all, we need to add the dependency to our build.sbt:

libraryDependencies += "org.scalamock" %% "scalamock" % "5.2.0" % Test

Then to get started, we need to mix in the trait MockFactory:

class ScalaMockFlatSpec extends AnyFlatSpec with MockFactory with Matchers {

Then we can go ahead and work with ScalaMock:

"A mocked Foo" should "return a mocked bar value" in {
  val mockFoo = mock[Foo]
  (mockFoo.bar _).expects().returning(6)

  mockFoo.bar should be(6)
}

class Foo {
  def bar = 100
}

In this very simple example, we see how to create a mock object and set an expectation. We can then check the value returned by the bar method is as expected.

10. Conclusion 

To summarize, in this tutorial, we’ve taken a first look at ScalaTest, a comprehensive testing framework for Scala.

First, we started by explaining some of the key concepts and saw how to write our first test. Next, we looked at some of the different testing styles and how we can mix in some nice matchers to our tests.

Finally, we took a quick look at how we can use tags and mocks from our tests.

As always, the full source code of the article is available over on GitHub.

2 Comments
Oldest
Newest
Inline Feedbacks
View all comments
Comments are open for 30 days after publishing a post. For any issues past this date, use the Contact form on the site.