Parsing JSON with Circe

1. Introduction

Circe is a Scala library that simplifies working with JSON, allowing us to easily decode a JSON string into a Scala object or convert a Scala object to JSON. The library automatically generates the object encoders and decoders, thereby reducing the lines of code we need to work with JSON in Scala.

2. Installation

To install the library, we’ll add a few lines to our build.sbt file:

val circeVersion = "0.14.3"

libraryDependencies ++= Seq(
  "io.circe" %% "circe-core",
  "io.circe" %% "circe-generic",
  "io.circe" %% "circe-parser"
).map(_ % circeVersion)

3. Validating JSON

Let’s start by validating a JSON string to check whether it contains a valid JSON object. First, let’s define a String that contains a valid JSON object and pass it to the parse function:

import io.circe._, io.circe.parser._

val jsonString =
"""
|{
| "textField": "textContent",
| "numericField": 123,
| "booleanField": true,
| "nestedObject": {
| "arrayField": [1, 2, 3]
| }
|}
|""".stripMargin

val parseResult: Either[ParsingFailure, Json] = parse(jsonString)

As a result, we get either a ParsingError or a Json object. We’ll then use the match statement to distinguish between the returned values:

parseResult match {
  case Left(parsingError) =>
    throw new IllegalArgumentException(s"Invalid JSON object: ${parsingError.message}")
  case Right(json) => // here we use the JSON object
}

4. Accessing JSON Properties the Long Way

It’s possible to extract field values from the parsed Json object by searching for field names and converting their values to expected data types. However, such a method is not recommended because it requires writing verbose and error-prone code.

Expanding on the match statement we defined earlier, let’s extract the value of numericField:

case Right(json) => 
  val numbers = json \\ "numericField"
  val firstNumber: Option[Option[JsonNumber]] =
    numbers.collectFirst{ case field => field.asNumber }
  val singleOption: Option[Int] = firstNumber.flatten.flatMap(_.toInt)

Note that the \\ notation is an alias to the findAllByKey function, which returns a list of Json objects.

Afterward, we’re using the collectFirst function to get an Option that may contain the first element of the list. Because of that, we end up with an Option nested in another Option. To get the underlying numeric value, we’ll flatMap the Option and convert the JsonNumber to an Option[Int].

Obviously, we don’t need to use this overcomplicated method because Circe offers a simplified API that hides all of the complexity.

5. Converting Scala Objects to JSON

Rather than extracting the fields manually and converting them to the expected formats, we can use the Circe codecs to convert JSON to and from a Scala object.

Codecs, however, require creating a case class that matches the fields of the parsed JSON string:

import io.circe._, io.circe.generic.semiauto._

case class Nested(arrayField: List[Int])

case class OurJson(
  textField: String,
  numericField: Int,
  booleanField: Boolean,
  nestedObject: Nested
)

When the case classes are defined, we can derive a Decoder from the class and use it to parse a JSON string. Note that we’ll define a Decoder of the Nested class first:

implicit val nestedDecoder: Decoder[Nested] = deriveDecoder[Nested]
implicit val jsonDecoder: Decoder[OurJson] = deriveDecoder[OurJson]

val decoded = decode[OurJson](jsonString)

We need the Nested class because, in our JSON string, the nestedObject field contains another JSON object. Therefore, we’ll deserialize every JSON object to a separate Scala object and define individual classes.

6. Converting JSON to Scala Objects

Similarly, we can derive class encoders and convert a Scala object into a Json object, and later, into a JSON string. Let’s convert the decoded case class back into a JSON string:

implicit val nestedEncoder: Encoder[Nested] = deriveEncoder[Nested]
implicit val jsonEncoder: Encoder[OurJson] = deriveEncoder[OurJson]

decoded match {
  case Right(decodedJson) =>
    val jsonObject: Json = decodedJson.asJson
    val newJsonString = jsonObject.spaces2
}

7. Other Data Type Conversions

Whenever we need to convert JSON to arbitrary types, we can use Encoder and Decoder.

7.1. Encoders

Encoders are the types that transform any type into JSON.

For example, let’s look at how to convert a List of Integers to a JSON:

import io.circe.syntax._

val otherJson = List(1, 2, 3).asJson

The asJson method will return a representation of the JSON below in the Circe data structure:

{
  "value": [1, 2, 3]
}

That’s possible because Circe has implemented multiple standard library data structure conversions.

7.2. Decoders

Decoders are the types that transform a JSON into another data structure.

Let’s keep with the example above, but this time, implement a Json conversion to List:

val jsonAsList: Result[List[Int]] = otherJson.as[List[Int]]

When successful, it returns the original Integer List.

7.3. Convert JSON to a Map

Circe features predefined Encoder and Decoder implementations for Map.

Let’s start with a simple String to JSON conversion with Circe:

import io.circe._, io.circe.parser._

val stringJson: String = """
{
  "name": "Baeldung",
  "language": "Scala"
}
"""

val parseResult = parse(stringJson)

And use the getOrElse method to extract the result:

val json: Json = parseResult.getOrElse(null)

Finally, let’s see a simple conversion from this JSON to a Map:

val jsonAsMap = json.as[Map[String, String]]

8. Working with Optional Fields

If some of the JSON fields are optional or have missing values, we’ll change the class definition and use the Option class as the field type. Let’s take a look at a JSON string with missing numericField, booleanField, and arrayField:

val jsonStringWithMissingFields =
"""{
| "textField" : "textContent",
| "nestedObject" : {
| "arrayField" : null
| }
|}""".stripMargin

An attempt to decode it without changing the class definition will end up with a “Left(DecodingFailure(Attempt to decode value on failed cursor, List(DownField(numericField))))” error. Therefore, we must modify the class definitions:

case class Nested(arrayField: Option[List[Int]])

case class OurJson(
  textField: String,
  numericField: Option[Int],
  booleanField: Option[Boolean],
  nestedObject: Nested
)

Now, the decoder can deal with missing values without any problems:

implicit val nestedDecoder: Decoder[Nested] = deriveDecoder[Nested]
implicit val jsonDecoder: Decoder[OurJson] = deriveDecoder[OurJson]

decode[OurJson](jsonStringWithMissingFields)

9. Writing a Custom Decoder

What if we wanted to turn a missing arrayField into an empty list instead of having an Option of List? Such situations require writing a custom Decoder. To write a custom decoder, we must implement the Decoder type.

In this example, we’ll check whether the arrayField is null and return Nil (an empty list) instead of None. If the field exists and contains an array, we return it without making any changes:

implicit val decodeNested: Decoder[Nested] = (c: HCursor) => for {
  arrayField <- c.downField("arrayField").as[Option[List[Int]]]
} yield {
  val flattenedArray = arrayField.getOrElse(Nil)
  Nested(flattenedArray)
}

10. Testing a Custom Decoder

When we customize the Circe decoders and encoders, we must test our code to make sure that it works correctly. For our examples, we’ll use the ScalaTest library for testing the custom code.

We’ll define a specification class in the test directory, prepare the input JSON strings, define the case classes, and implement the decoders (both the custom decoder and the automatically generated one).

After that, we’ll write four tests for every possible case — an existing array with values, an empty array, a null field, and a missing field:

"A custom decoder" should "decode a JSON with a null array value" in {
  decode[OurJson](jsonStringWithNullArray) shouldEqual Right(OurJson("textContent", None, None, Nested(Nil)))
}

it should "decode a JSON with a missing field" in {
  decode[OurJson](jsonStringWithMissingArray) shouldEqual Right(OurJson("textContent", None, None, Nested(Nil)))
}

it should "decode a JSON with an existing array value" in {
  decode[OurJson](jsonStringWithArray) shouldEqual Right(OurJson("textContent", None, None, Nested(List(1, 2))))
}

it should "decode a JSON with an empty array" in {
  decode[OurJson](jsonStringWithEmptyArray) shouldEqual Right(OurJson("textContent", None, None, Nested(Nil)))
}

11. Automatically Generated Encoders

When we don’t need to customize the decoder, we can use the automatic decoder derivation and shorten the code even more.

If we import the io.circe.generic.auto._ package, we get an automatic derivation of decoders (and encoders) for all of the existing types, so we can use the JSON parser without creating a Decoder instance:

import io.circe.generic.auto._, io.circe.parser

parser.decode[OurJson](jsonString)

12. Conclusion

Circe is a Scala library that simplifies working with JSON by hiding the implementation details in a simple API. However, we can always modify its behavior by creating a custom encoder or a custom decoder or using the field extraction code directly.

As always, the code for these examples is available over on GitHub.