Cancellation in Cats Effect

1. Introduction

In this article, we’ll explore the cancellation of computations in Cats Effect. Сancellation becomes necessary when some results are no longer needed, the user cancels the operation, or the programmatic flow triggers it. We’ll briefly review the features of Cats Effect runtime that shape the cancellation properties. Later we’ll review low-level and high-level capabilities for cancellation in Cats Effect.

2. Concurrency and Cancellation in Cats Effect

The cancellation features of the library are largely a derivative of its concurrency capabilities since cancellation involves operations with the underlying concurrency primitives.

The main building block of concurrency in Cats Effect is Fiber, which is essentially a lightweight logical thread. Cats Effect runtime governs the lifecycle of the fibers, making the cancellation of fiber much cheaper than the cancellation (i.e., interruption) of the JVM thread, which maps directly onto a system thread.

There are a few properties of cancellation in Cats Effect we need to be aware of. First, cancellation in Cats Effect is cooperative: Cats Effect runtime doesn’t imperatively shut down fibers. Instead, calling cancel on a fiber conveys a request for cancellation. The fiber employs its finalizers to cancel itself and then returns control to the calling fiber. If the fiber can’t be canceled at the moment, then the calling fiber will asynchronously wait for it.

Second, the Cats Effect provides uncancelable regions: within certain areas, a fiber ignores cancellation requests (atomic computations utilize this feature a lot). Additionally, a fiber performs cancellation checks at every step of its execution, which provides better accuracy and predictability of the cancellation result.

3. Direct Interaction With Fiber Data Type

3.1. Adding Project Dependency

In this tutorial, we’ll leverage only Cats Effect library:

libraryDependencies ++= Seq(
  "org.typelevel" %% "cats-effect" % "3.5.2"
)

3.2. Meeting the Fundamental Fiber Operations

We can use the Fiber data type as a handle for managing running computation, including start, join, and cancellation of the Fiber:

for {
  fiber <- IO.pure(3).onCancel(IO.println("cancellation signal received for fiber")).start
  _     <- fiber.cancel
  _     <- fiber.join
} yield ()

Fiber’s API is pretty low-level for the typical development tasks, and it’s also can be challenging to implement some fucntionalities directly with fibers.

We’ll demonstrate this through an example of running two effects in parallel and canceling one fiber if the other fails.

3.2.  Execution of the Effects in Parallel With Fibers

The first implementation may resemble this:

def naiveParIO[A, B, C](ioa: IO[A], iob: IO[B], onCancelA: IO[A], onCancelB: IO[B], f: (A, B) => C) =
    for {
      fiberA <- ioa.start
      fiberB <- iob.start
      a      <- fiberA.joinWith(onCancelA).onError(_ => fiberB.cancel)
      b      <- fiberB.joinWith(onCancelB).onError(_ => fiberA.cancel)
    } yield f(a, b)

The problem here is that the call onError() is itself an effect, so if fiberB fails, we won’t be able to invoke its onError handler until the termination of fiberA. We can solve this problem by employing another pair of fibers:

def parIO[A, B, C](ioa: IO[A], iob: IO[B], onCancelA: IO[A], onCancelB: IO[B], f: (A, B) => C): IO[C] =
    for {
      fiberA  <- ioa.start
      fiberB  <- iob.start
      fiberAj = fiberA.joinWith(onCancelA).onError(_ => fiberB.cancel)
      fiberBj = fiberB.joinWith(onCancelB).onError(_ => fiberA.cancel)
      regA    <- fiberAj.start
      regB    <- fiberBj.start
      a       <- regA.joinWith(onCancelA)
      b       <- regB.joinWith(onCancelB)
    } yield f(a, b)

As we can see, the function for running fibers in parallel with proper handling of errors and cancellation turns out to be quite bulky. Fortunately, it’s rarely required, and further, we’ll focus on high-level constructs for cancellation in Cats Effect.

4. High-Level Mechanisms for Cancellation

4.1. Racing of the IOs

The first combinator from IO we’ll review is race:

def race[A, B](left: IO[A], right: IO[B]): IO[Either[A, B]]

This method returns the first task to finish either in success or error, and the “loser” task is canceled. In this context, Either[A, B] is used to express equal alternatives, not an error and success flow.

If we would like to gain more control over the losing fiber instead of simply canceling it, we should employ racePair:

def racePair[A, B](
      left: IO[A],
      right: IO[B]): IO[Either[(OutcomeIO[A], FiberIO[B]), (FiberIO[A], OutcomeIO[B])]]

The implementation of race in the Cats Effect relies on racePair. While the underlying code is pretty complex, the intuition behind it is straightforward: cancel the “loser” fiber and discard its result in favor of the “winner” fiber through as(a) or as(b):

IO.racePair(ioA, ioB).flatMap {
  case Left((a, fiberB)) => fiberB.cancel.as(a)
  case Right((fiberA, b)) => fiberA.cancel.as(b)
}

Additionally, there’s a raceOutcome method which essentially just runs two fibers in parallel because it returns a pair of outcomes of terminated fibers:

def raceOutcome[B](that: IO[B]): IO[Either[OutcomeIO[A @uncheckedVariance], OutcomeIO[B]]]

4.2. Running of the Effects in Parallel

Cancellation of a running fiber can occur even if the computation itself runs without any issues, but another computation running in parallel fails. In such scenarios, both fibers fail.  Let’s demonstrate this behavior with parTupled method of IO, which runs a tuple of IOs in parallel:

(IO.sleep(1.seconds) >> IO.raiseError(new RuntimeException("Boom!")),
 IO.sleep(2.seconds) >> IO.println("Hey there")).parTupled

The second fiber will be canceled, and this behavior is expected by design: if one of the parallel effects throws an exception, the other is marked for cancellation. This shows the reaching of the cancellation boundary for the second fiber.

5. Uncancellable and Cancelable Regions

5.1. IO.uncancelable() in Action

As developers, we should maintain the state of our application consistent and cancellation is one of the threats to this consistency. Pretty often, certain sections of a program must execute completely, even in the presence of a cancellation signal.

Let’s imagine the service of notifications that holds a list of student emails to notify them about particular events, such as the rescheduling of an exam. For illustration, we’re adding IO.sleep(2.seconds) in the send method below. The first implementation may resemble this:

def sendEmails(message: String): IO[Unit] = {
    def getEmails(): IO[List[String]] = IO(List("[email protected]", "[email protected]", "[email protected]"))

    def send(email: String): IO[Unit] = IO.sleep(2.seconds) >> IO.println(s"the email to $email with $message was sent")

    for {
      emails <- getEmails()
      _      <- IO.println("ready to send emails")
      _      <- emails.traverse(send)
      _      <- IO.println("emails are sent")
    } yield ()
}

Now let’s suppose that we closed the mailing program accidentally. We can model this scenario using IO.race, where cancellation occurs in the middle of sending the second email:

val sendEmails: IO[Unit] = Cancellation
  .sendEmails("Exam on FP is shifted to Jan 11, 2024")
  .onCancel(IO.println("cancellation signal received"))
IO.race(IO.sleep(3.seconds), sendEmails).unsafeRunSync()

After running this code, we noticed that one email was eventually sent:

ready to send emails
the email to [email protected] with Exam on FP is postponed to Jan 11, 2024 was sent
cancellation signal received

Obviously, we aim to send either all messages or none of them. This is where the Cats Effect fine-grained control over cancellation comes into play. Specifically, we can impose certain steps of the fiber’s execution to ignore the cancellation altogether. In this case, emails.traverse(send) should be uncancellable:

for {
  emails <- getEmails()
  _      <- IO.println("ready to send emails")
  _      <- IO.uncancelable(_ => emails.traverse(send))
  _      <- IO.println("emails are sent")
} yield ()

IO.uncancellable is used to safeguard particular effects from cancellation. The fiber receives a cancellation signal during the execution of IO.uncancelable(_ => emails.traverse(send)), but the cancellation doesn’t act on this effect. Instead, the cancellation will occur at the next cancellable step, i.e., IO.println(“emails are sent”).

And ultimately, we can ensure that all emails are sent:

ready to send emails
sending email to [email protected] with Exam on FP is shifted to Jan 11, 2024
sending email to [email protected] with Exam on FP is shifted to Jan 11, 2024
sending email to [email protected] with Exam on FP is shifted to Jan 11, 2024
cancellation signal received

5.2. Cancellation in Case of Errors

Recalling our example with parTupled, we can now defer the cancellation of the second fiber using IO.uncancelable:

(IO.sleep(1.seconds) >> IO.raiseError(new RuntimeException("Boom!")),
 IO.uncancelable(_ => IO.sleep(2.seconds) >> IO.println("Hey there"))).parTupled

Although the resulting effect will fail, we’ll see the message “Hey there” in the console because the second fiber’s cancellation has been deferred.

6. Cleanup in Case of Cancellation With Resource and bracket()

Cats Effect Resource is a data structure that allows us to acquire and release resources in a controlled and effectful fashion. Resource has a very important property: if the effect is canceled during the acquisition or usage stages, the finalizer will be executed.

For simplicity, let’s consider that the previously used function sendEmails() leverages Resource:

Resource.make(getEmails())(emails => IO.println(s"emails $emails are released"))
  .use(emails => 
    IO.println("ready to send emails") >>
    IO.uncancelable(_ => emails.traverse(send)) >>
    IO.println("emails are sent"))

If we cancel its execution during the acquisition or usage, the finalization logic will still execute:

ready to send emails
...
emails List([email protected], [email protected], [email protected]) are released
cancellation signal received

The bracket() method of IO will behave identically in this case:

getEmails().bracket(emails => IO.println("ready to send emails") >>
  IO.uncancelable(_ => emails.traverse(send(message))) >>
  IO.println("emails are sent")
)(emails => IO.println(s"emails $emails are released"))

According to the Resource’s ScalaDocs, the actions provided to acquire and release of the method make aren’t interruptible. The release will run whether the action passed to use succeeds, fails, or is interrupted.

In this context, cancellation also fits the scenario of interruption. The crucial difference between cancellation and interruption is that cancellation acts on the fiber itself, managed by Cats Effect runtime, whereas interruption pertains directly to the JVM thread lifecycle.

7. Blocking and Cancellation

7.1. Cancellation of Blocking and Interruptible Effects

The Cats Effect runtime dispatches blocking operations to a separate thread pool for optimization purposes, and by design, these operations are uncancelable. In the following example, the blocking effect terminates in 1 second disregarding the timeout of 100 milliseconds:

IO.blocking(Thread.sleep(1000)).timeout(100.millis)

It’s possible to solve such issues by employing IO.interruptible in place of IO.blocking:

IO.interruptible(Thread.sleep(1000)).timeout(100.millis)

In this scenario, the Cats Effect runtime should take care of the interruption of the underlying thread in the blocking thread pool. Not only does it impose a performance penalty, but it may also lead to unexpected outcomes since some effects may ignore interruption, such as java.io calls. Furthermore, the authors of the Cats Effect reasonably warn us that interruption may result in resource leaks or invalid states.

7.2. Partial Control Over Cancelation With IO’s cancelable()

As we’ve shown, using IO.interruptible can be questionable, and it’s sometimes not even feasible. Thankfully, we have an alternative tool to gain partial control over the blocking effect’s cancellation if we can’t or don’t want to use IO.interruptible.

Let’s analyze the slightly modified example from the Cats Effect ScalaDoc for IO method cancelable:

val flag = new AtomicBoolean(false)
val ioa = IO blocking {
  while (!flag.get()) {
    Thread.sleep(1_000)
    println(s"counter = $counter")
  }
}
ioa.cancelable(IO.sleep(5.seconds) >> IO.println("executing the finalizer...") >> IO.delay(flag.set(true)))

In this case, ioa executes in the blocking execution context, and the Cats Effect runtime won’t cancel the underlying fibers. The last line ioa.cancelable(IO.delay(flag.set(true))) signifies that the cancellation signal will trigger execution of the passed finalizer IO.delay(flag.set(true)). Then the finalizer changes the AtomicBoolean and thus affects the fiber execution in a way very similar to the actual cancellation.

As a test, execute this application using a console command:

sbt run com.baeldung.scala.catseffects.CancellationApp

and cancel its execution by Ctrl+C. We’ll observe the message “executing the finalizer…” 5 seconds after cancellation, and immediately after that, the execution stops. Without adding cancelable, the program will keep flooding the console every second, disregarding the cancellation signals.

It’s important to realize the limitations of cancelable: it requires external state to interleave with the effect’s execution. Otherwise, the finalizer won’t help in cancellation. In the example above, we had the option to utilize AtomicBoolean. However, sometimes the finalizer provided to cancelable will never even run because the effect is inherently non-terminating, and there’s no way to intervene in its lifecycle:

val trullyUncancellable = IO.uncancelable(_ => IO.never)
trullyUncancellable.cancelable(IO.delay(cancelWhateverPeopleSay()))

8. Conclusion

In this article, we explored the fundamentals of cancellation support in Cats Effect, starting from the underlying runtime design decisions. We investigated the fiber’s cancellation process in action, and we figured out what are the uncancelable regions where the fiber suspends cancellation. We learned how Cats Effect constructs like Resource and bracket approach cancellation, and we examined aspects of working with blocking effects and interruptions. Finally, we met cancelable as a mechanism to gain partial control over the cancellation of the blocking effect.

To sum up, we demonstrated that developers shouldn’t only consider how an application addresses errors but also how it handles cancellations. Fortunately, Cats Effect provides us with effective mechanisms to manage both.

As usual, the full code for this article is available over on GitHub.