Test Aspects in ZIO Test

1. Introduction

ZIO Test is a testing library specifically designed to test ZIO-based applications. The core idea behind it is that tests are first-class objects. This means they can be composed and decorated, similar to other ZIO values. This solution leads to increased flexibility, as we can express all the additional logic around tests (such as “before all” and “after all” operations) as a decoration instead of as a callback.

Aspects in ZIO Test serve this purpose. We can think of aspects as test transformations, aka functions to turn a ZIO Spec (basically a test suite) into another.

In this tutorial, we’ll briefly examine ZIO Specs and then see how to decorate them using ZIO Aspects.

2. Setup

To use ZIO Test to unit test our applications, we have to add the zio and zio-test libraries to our list of dependencies in build.sbt:

libraryDependencies += "dev.zio" %% "zio" % zioVersion 
libraryDependencies += "dev.zio" %% "zio-test" % "2.0.21" % Test

The zio-test dependency is only required for testing; we added Test at the end to ensure it’s not included in the built JAR for our project.

To run our tests in SBT, we’ll have to tell the build system to use ZIO’s test framework. First, let’s add the zio-test-sbt dependency:

libraryDependencies += "dev.zio" %% "zio-test-sbt" % zioVersion % Test

Secondly, let’s set zio.test.sbt.ZTestFramework as an SBT testFramework:

testFrameworks += new TestFramework("zio.test.sbt.ZTestFramework")

We can now run our ZIO-based tests as usual. Specifically, we can use the sbt test and sbt testOnly commands to run the tests in sbt. In addition, our tests should run using the tooling available in IDEs such as JetBrains IntelliJ.

3. ZIO Specs

In ZIO Test, any Scala object implementing the ZIOSpecDefault Scala trait is a runnable test. This requires our object to implement the spec method, defining a test specification. In a few words, we can imagine a ZIO Spec as a test suite, a collection of tests to run. Let’s take a look at a very simple Spec:

object ZAspectsSpec extends ZIOSpecDefault {
  override def spec = suite("ZAspectSpec")(
    test("Tautology test") {
      assertTrue(true)
    }
  )
}

In the example above, we create a new test suite with a single dummy test asserting that true equals true.

4. ZIO Aspects

As we saw above, ZIO aspects are simply functions that transform Specs (or single tests), effectively decorating them. As such, they’re composable, and we can chain them together. Let’s take a look at the syntax to apply an aspect to our test class:

object ZAspectsSpec extends ZIOSpecDefault {
  override def spec = suite("ZAspectSpec")(
    test("Tautology test") {
      assertTrue(true)
    } @@ testAspect
  ) @@ suiteAspect
}

All we need to do is to use the @@ operator, followed by the name of the aspect we want to apply. In particular, the framework applies testAspect only to the “Tautology test” test case. This means all the other tests in the suite wonìt be affected by the aspect. suiteAspect, on the other hand, applies to all tests in the suite.

We can also chain multiple aspects together: @@ testAspect1 @@ testAspect2. For example, the chain timeout(1.second) @@ ignore sets a timeout on a given test suite or test case but also ignores it. Therefore, the test(s) will not fail even if the timeout triggers. In general, the chaining ordering matters: the same aspects applied in different orders might behave differently.

To ignore a given test, we can use the ignore aspect:

test("Tautology test") {
  assertTrue(true)
} @@ ignore

If we run our suite, the output will be:

+ ZAspectSpec
  - Tautology test - ignored: 1
0 tests passed. 0 tests failed. 1 tests ignored.

4.1. Running ZIO Effects Before and After a Test

We can use these modifiers to run a ZIO effect before, after, and around a given test:

• before runs an effect before the test;
• after, afterSuccess, and afterFailure run an effect after a test, possibly only if the test succeeded or failed;
• around combines before and after. There is also an aroundWith aspect, making the result of the before effect available in the after one.

All the aspects mentioned above come with variants to apply the effect to all tests. For example, beforeAll runs the effect before all tests.

Let’s see how to set up an environment variable before a test and print it after the test, using around:

test("Run effects around a test") {
  for {
    plt <- System.env("PLATFORM")
  } yield assertTrue(plt.contains("Baeldung"))
} @@ aroundWith {
  val platform = "Baeldung"
  TestSystem.putEnv("PLATFORM", platform) *> ZIO.succeed(platform)
}(envVar => ZIO.debug(s"Platform: $envVar"))

In the code snippet above, the before effect sets the Platform environment variable to Baeldung and returns it. Then, the testing code gets the variable and verifies it contains “Baeldung”. Lastly, the after effect prints the value returned by the before one. Note that if we hadn’t specified *> ZIO.succeed(platform) in before, after would receive an instance of Unit, as TestSystem.putEnv() returns a UIO[Unit].

When we run the test above, we’ll get the following output:

+ ZAspectSpec
Platform: Baeldung
  + Run effects around a test
1 tests passed. 0 tests failed. 0 tests ignored.

4.2. Flaky and Non-Flaky Tests

Flaky tests are test cases that sometimes pass and sometimes fail, with no changes to the code. They’re usually indicators of concurrency issues in our code. ZIO provides us with two aspects: nonFlaky and flaky.

The former runs a test many times (100 by default), succeeding only if there are no failures. The latter, on the other hand, retries a test until it succeeds. This is useful, for example, if we generate random data for our test and are ok with the test failing sometimes.

Let’s see how we can use nonFlaky to repeat a test 10 times:

test("Non flaky") {
   assertTrue(true)
} @@ nonFlaky(10)

If we run the test above, we’ll get the following output, confirming it was executed ten times:

+ ZAspectSpec
  + Non flaky - repeated: 10
1 tests passed. 0 tests failed. 0 tests ignored.

4.3. Timing Out Tests

Sometimes, setting a timeout for a test is helpful to avoid having it run for a long time. Use cases for this include tests interacting with other components that have to fail if the response doesn’t arrive within a given time window or tests for time-sensitive components, where we want to ensure that the code works and that it can terminate within a predictable deadline. The timeout aspect is perfect for this:

test("timeouts") {
  for {
    _ <- ZIO.succeed("Baeldung").forever
  } yield assertTrue(true)
} @@ timeout(1.second)

In the snippet above, we used ZIO::forever to repeat, indefinitely and without an end, a successful ZIO effect, with Bealdung as a value. This way, the test would never terminate if it wasn’t for timeout, which aborts the execution after one second.

If we run this code, we’ll get a failure:

+ ZAspectSpec
  - timeouts
Timeout of 1 s exceeded.
0 tests passed. 1 tests failed. 0 tests ignored.

  - ZAspectSpec - timeouts
Timeout of 1 s exceeded.

5. Conclusion

In this article, we scratched the surface of ZIO Test Aspects. First, we looked at how to write tests in the ZIO ecosystem. Then, we saw what aspects are and how we can leverage them to modify the behavior of our tests. For example, we saw how to expose concurrency issues, set time limits, and run other ZIO effects before or after executing our test cases.

As usual, you can find the code over on GitHub.