Handling YAML in Scala 3

Last updated: December 15, 2023

Reviewed by: Dominique Plante

1. Introduction

YAML stands for YAML Ain’t a Markup Language – we use it for data, not documents. It’s a human-readable data serialization format that we often use for writing configuration files with a design that makes it easy to read and write while also being expressive enough to represent complex data structures.

YAML is widely used and is compatible with most common programming languages. In this tutorial, we’ll learn how to read YAML from a string or file and write to a YAML file.

2. Circe-yaml

When dealing with data in Scala, two formats come to mind, JSON and YAML. JSON excels due to its simplicity and compatibility with web APIs, while YAML shines regarding human-readable configuration files.

But what if we need to work with both formats? Enter Circe-yaml, a small Scala library for parsing YAML into Circe’s JSON AST. It bridges the gap between JSON and YAML in a type-safe and idiomatic way.

Here are some good reasons to use Circe-yaml:

  • Simple API: Circe-yaml provides a simple API for working with JSON and YAML since it uses the same functional programming techniques one would use with Circe for JSON when working with YAML data.
  • Interoperability: It also allows us to switch between JSON and YAML representations of our data seamlessly. This flexibility Circe-Yaml brings is valuable when dealing with configurations that may be authored in either format.
  • Circe Eco-system: Projects that already use Circe for JSON will find that adding Circe-yaml comes as a natural extension since it integrates smoothly with the existing ecosystem making it easy to adopt.

3. Setting Up

We’ll need to add the following dependencies to our build to follow along with this article.sbt file:

val scala3Version = "3.3.0"

lazy val root = project
  .in(file("."))
  .settings(
    name := "Scala3Features",
    version := "0.1.0-SNAPSHOT",
    scalaVersion := scala3Version,
    libraryDependencies += "io.circe" %% "circe-yaml" % "0.14.2",
    libraryDependencies += "io.circe" %% "circe-generic" % "0.14.6",
    libraryDependencies += "io.circe" %% "circe-parser" % "0.14.6"
  )

In addition to Circe-yaml, we’ll also use Circe-generic for automatic conversions between case classes and Circe’s JSON format. We’ll need Circe-parser, which is a wrapper around the Jawn JSON parser.

4. Reading YAML

In this section, we’ll cover how to read data from a YAML string and a YAML file with single and multiple documents.

4.1. Reading From a YAML String

Let’s create a YAML string with a sample configuration:

val ordersYamlConfig: String =
  """
    name: Orders String
    server:
        host: localhost
        port: 8080
    serverType: 
        - Http
        - Grpc
  """

Here, we create our YAML string and assign it to ordersYamlConfig. To parse this, we’ll need to import the yaml package from circe:

import io.circe.yaml
import io.circe.*

val ordersStringConfig: Either[ParsingFailure, Json] =
  yaml.parser.parse(ordersYamlConfig)

The parse() function from Circe-yaml takes a string and returns an Either[ParsingFailure, Json]. To manage this data into our Scala project, we’ll need to define case classes to hold it:

case class Server(host: String, port: Int)
case class OrdersConfig(app: String, server: Server, serverType: List[String])

OrdersConfig is where all the data will be stored. It takes app of type String, server of type Server, and serverType of type List[String]. Since the server key in ordersYamlConfig has nested values, we give it its own Server case class with a host value of type String and a port value of type Int. Let’s define a function to process the Either[ParsingFailure, Json] value type:

import cats.syntax.either.*
import io.circe.generic.auto.*

def processJson(
    json: Either[ParsingFailure, Json]
): Either[Error, OrdersConfig] =
  json
    .leftMap(err => err: Error)
    .flatMap(_.as[OrdersConfig])

We define processJson(), a function that converts the YAML parsed values to Either[Error, OrdersConfig]. This is done by calling leftMap() on our Either[ParsingFailure, Json] to convert the ParsingFailure to Circe’s Error type, after which we flatMap() on the resulting Either to convert the Json value to an OrdersConfig using the as[OrdersConfig] method.

This implementation is made possible with the help of the io.circe.generic.auto.* import which automatically provides a given of Decoder[OrdersConfig] from Circe in scope. Let’s create another utility function for printing OrdersConfig to the console:

def printValue(value: Either[Error, OrdersConfig]) =
  value match
    case Right(v)  => println(v)
    case Left(err) => println(err.getMessage)

The printValue() function takes our Either[Error, OrdersConfig], performs a pattern match, and either prints OrdersConfig in case of success or print the error message in case of failure:

printValue(ordersFileConfig)

/** OrdersConfig(Orders File,Server(localhost,8080),List(Http, Grpc))
  */

This shows that our YAML string has been converted to a Scala case class.

4.2. Reading From a YAML File

Let’s create an orders.yaml file in our projects resources folder with the following contents:

name: Orders File
server:
    host: localhost
    port: 8080
serverType: 
    - Http
    - Grpc

To read this file, we’ll need to use FileReader from java.io:

import java.io.FileReader

val yamlFileReader: Either[Throwable, FileReader] =
  Try {
    new FileReader(
      "src/main/scala/resources/orders.yaml"
    )
  }.toEither

val ordersFileConfig: Either[Throwable, OrdersConfig] =
  yamlFileReader
    .map(fileReader => processJson(yaml.parser.parse(fileReader)))
    .flatten

Here we create a new FileReader class and pass the path as a string to the constructor. We then wrap it in a Try since it could fail attempting to read our file. Calling toEither on Try gives us an Either[Throwable, FileReader], which we assign to yamlFileReader.

To read this file, we call map on yamlFileReader and pass the FileReader to Circe-yaml’s parser() method to give us an Either[ParsingFailure, Json], which in turn is passed to processJson(). At this point, this whole expression returns an Either[Throwable, Either[circe.Error, OrdersConfig]].  By calling flatten, we reduce this to Either[Throwable, OrdersConfig].

However, the printValue() function only works with Either[Error, OrdersConfig]. To make this work with Throwable as well, we’ll need to add a small modification:

def printValue(value: Either[Error | Throwable, OrdersConfig]) = ???

This makes use of Scala 3 union types to handle both Error and Throwable.
Let’s print this to the console and see the result:

printValue(ordersFileConfig)

/** OrdersConfig(Orders File,Server(localhost,8080),List(Http, Grpc))
  */

4.3. Reading From a Yaml File With Multiple Documents

Let’s create a service.yaml file in the resources folder with the following contents:

name: Orders
server:
    host: localhost
    port: 8080
serverType: 
    - Http
    - Grpc
---
name: Test
server:
    host: localhost
    port: 9999
serverType: 
    - Http
    - Grpc

It’s possible to store multiple YAML documents in a single file by separating them with triple dashes, as shown in the code above.

Circe-yaml also provides a method for reading such documents:

val yamlFileReader2: Either[Throwable, FileReader] =
  Try {
    new FileReader(
      "src/main/scala/resources/service.yaml"
    )
  }.toEither

We still use FileReader to read the file as we did in the previous section:

val ordersFileConfig2: Either[Throwable, List[Either[Error, OrdersConfig]]] =
  yamlFileReader2
    .map(fileReader => yaml.parser.parseDocuments(fileReader).toList)
    .map(_.map(processJson))

Here we also pass the fileReader to Circe-yaml’s parser; however, this time, we use the parseDocuments() method, which returns a Stream[Either[ParsingFailure, Json]] that we convert to a List by calling toList.

Lastly, we call map to access the list on which we apply the processJson() function to all elements giving us an Either[Throwable, List[Either[Error, OrdersConfig]]]. Let’s see how we can print this to the console:

ordersFileConfig2 match
  case Right(lst) =>
    lst.foreach(printValue)
  case Left(err) => println(err.getMessage)

/** OrdersConfig(Orders,Server(localhost,8080),List(Http, Grpc)) 
  * OrdersConfig(Test,Server(localhost,9999),List(Http, Grpc))
  */

Here we pattern match on ordersFileConfig2; in case of success, we call foreach() on our list and sequentially call printValue on every element. In case of an error, we print the error message.

5. Writing YAML

In this section, we’ll look at two different scenarios, how to write a JSON String to a YAML file and how to write a case class to a YAML file.

Before we dive in, we’ll need some utility functions. Let’s define a function that gives us access to Java’s FileWriter:

import java.io.FileWriter
import java.io.File

def fileWriter(path: String): Either[Throwable, FileWriter] =
  Try {
    new FileWriter(new File(path))
  }.toEither

The fileWriter() function takes a path and passes it to a new instance of FileWriter. We wrap it in a File and pass it as an argument. This expression is nested in a Try, which we transform into an Either since it may fail.

Now let’s define a function to make use of FIleWriter to write our YAML files:

import io.circe.yaml.syntax.*
def writeYaml(jsnValue: Json, fw: FileWriter, path: String): String =
    Try {
      fw.write(jsnValue.asYaml.spaces2)
      fw.close()
    }.fold(
      e => e.getMessage(),
      _ => s"${Paths.get(path).getFileName().toString()} has been written"
    )

The writeYaml function takes a value of type Json, a FileWriter, and a path as arguments. Within the Try, we call fw.write to write the YAML string and then close the connection by calling fw.close(). Lastly, we call fold on Try and either get an error message in case the write process fails or a message telling the user that the file has been written.

The .asYaml.spaces2 method comes from circe.yaml.syntax.*, this converts our  JSON value to YAML and formats it before being written to a file.

5.1. Writing a JSON String to a YAML File

Let’s define our sample JSON configuration string:

import io.circe.parser.*

val jsonString =
  """
    {
      "name": "Orders Json",
      "server":
        {
          "host": "localhost",
          "port": 8080
        },
      "serverType": ["Http", "Grpc"]
    }
  """

Now let’s define a function to parse this string:

import io.circe.parser.*
def writeJsonStr(path: String, jsonStr: String): Either[Throwable, String] =
  for
    jsnValue <- parse(jsonString)
    fw <- fileWriter(path)
  yield writeYaml(jsnValue, fw, path)

The writeJsonStr function takes a path and a JSON string as arguments. It makes use of Scala’s for comprehension, which is syntactic sugar to flatMap and then map.

We pass the jsonStr to the parse() function from circe.parser, this returns a Either[ParsingFailure, Json] which we flatMap on to get our Json value (jsnValue), we also flatMap on fileWriter(path) to retrieve our FileWriter and finally map these values to writeYaml to write to a file:

writeJsonStr("src/main/scala/resources/sample.yaml", jsonString) match
    case Right(v)  => println(v)
    case Left(err) => println(err.getMessage)

If calling the writeJsonStr() function is successful, pattern matching on it should yield the following result:

/** sample.yaml has been written
  */

We should also have a sample.yaml file in the resources folder of our project with the following contents:

name: Orders Json
server:
  host: localhost
  port: 8080
serverType:
- Http
- Grpc

5.2. Writing a Case Class to a YAML File

Let’s define the case class that we’ll write:

val myCaseClass =
  OrdersConfig("Orders", Server("localhost", 8080), List("Http", "Grpc"))

Here we define a case class with similar configuration values to our jsonString. Let’s define our write function:

import io.circe.syntax.*
def writeOrdersConfig(path: String, oc: OrdersConfig): String =
  fileWriter(path) match
    case Right(fw) => writeYaml(oc.asJson, fw, path)
    case Left(err) => err.getMessage

The writeOrdersConfig() function takes a path of type String and an OrdersConfig. We pattern match on fileWrtier(path) to get access to the FileWrtier which we pass to the wrtieYaml() function along with the path and value of type Json.

This function returns a string in case of success or an error message in case of failure. The asJson method on oc comes from io.circe.syntax.*, it converts the case class to circe’s Json type.

Calling this function should return a similar this in case of success:

println(
  writeOrdersConfig("src/main/scala/resources/sample2.yaml", myCaseClass)
)

/**sample2.yaml has been written
  */

If we check the supplied path, we should find sample2.yaml with the following contents:

name: Orders
server:
  host: localhost
  port: 8080
serverType:
- Http
- Grpc

6. Conclusion

Circe-yaml is a very good solution that bridges the gap between JSON and YAML in our project.

In this article, we learned at how to use Circe-yaml to handle YAML in a Scala project. We also discussed how the library makes it easy to read either single or multiple document YAML files. Finally, we saw how to write to a YAML file.

As usual, the code for this article can be found over on GitHub.

Comments are open for 30 days after publishing a post. For any issues past this date, use the Contact form on the site.