Custom JSON Deserialization Using Spring WebClient

1. Overview

In this article, we’ll explore the need for custom deserialization and how this can be implemented using Spring WebClient.

2. Why Do We Need Custom Deserialization?

Spring WebClient in the Spring WebFlux module handles serialization and deserialization through Encoder and Decoder components. The Encoder and Decoder exist as an interface representing the contracts to read and write content. By default, The spring-core module provides byte[], ByteBuffer, DataBuffer, Resource, and String encoder and decoder implementations.

Jackson is a library that exposes helper utilities using ObjectMapper to serialize Java objects into JSON and deserialize JSON strings into Java objects. ObjectMapper contains built-in configurations that can be turned on/off using the deserialization feature.

Customizing the deserialization process becomes necessary when the default behavior offered by the Jackson Library proves inadequate for our specific requirements. To modify the behavior during serialization/deserialization, ObjectMapper provides a range of configurations that we can set. Consequently, we must register this custom ObjectMapper with Spring WebClient for use in serialization and deserialization.

3. How to Customize Object Mappers?

A custom ObjectMapper can be linked with WebClient at the global application level or can be associated with a specific request.

Let’s explore a simple API that provides a GET endpoint for customer order details. In this article, we’ll consider some of the attributes in the order response that require custom deserialization for our application’s specific functionality.

Let’s have a look at the OrderResponse model:

{
  "orderId": "a1b2c3d4-e5f6-4a5b-8c9d-0123456789ab",
  "address": [
    "123 Main St",
    "Apt 456",
    "Cityville"
  ],
  "orderNotes": [
    "Special request: Handle with care",
    "Gift wrapping required"
  ],
  "orderDateTime": "2024-01-20T12:34:56"
}

Some of the deserialization rules for the above customer response would be:

• If the customer order response contains unknown properties, we should make the deserialization fail. We’ll set the FAIL_ON_UNKNOWN_PROPERTIES property to true in ObjectMapper.
• We’ll also add the JavaTimeModule to the mapper for deserialization purposes since OrderDateTime is a LocalDateTime object.

4. Custom Deserialization Using Global Config

To deserialize using Global Config, we need to register the custom ObjectMapper bean:

@Bean
public ObjectMapper objectMapper() {
    return new ObjectMapper()
      .configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, true)
      .registerModule(new JavaTimeModule());
}

This ObjectMapper bean, upon registration, will be automatically linked with CodecCustomizer to customize the encoder and decoder associated with the application WebClient. Consequently, it ensures that any request or response at the application level is serialized and deserialized accordingly.

Let’s define a controller with a GET endpoint that invokes an external service to retrieve order details:

@GetMapping(value = "v1/order/{id}", produces = MediaType.APPLICATION_JSON_VALUE)
public Mono<OrderResponse> searchOrderV1(@PathVariable(value = "id") int id) {
    return externalServiceV1.findById(id)
      .bodyToMono(OrderResponse.class);
}

The external service that retrieves the order details will use the WebClient.Builder:

public ExternalServiceV1(WebClient.Builder webclientBuilder) {
    this.webclientBuilder = webclientBuilder;
}

public WebClient.ResponseSpec findById(int id) {
    return webclientBuilder.baseUrl("http://localhost:8090/")
      .build()
      .get()
      .uri("external/order/" + id)
      .retrieve();
}

Spring reactive automatically uses the custom ObjectMapper to parse the retrieved JSON response.

Let’s add a simple test that uses  MockWebServer to mock the external service response with additional attributes, and this should cause the request to fail:

@Test
void givenMockedExternalResponse_whenSearchByIdV1_thenOrderResponseShouldFailBecauseOfUnknownProperty() {

    mockExternalService.enqueue(new MockResponse().addHeader("Content-Type", "application/json; charset=utf-8")
      .setBody("""
        {
          "orderId": "a1b2c3d4-e5f6-4a5b-8c9d-0123456789ab",
          "orderDateTime": "2024-01-20T12:34:56",
          "address": [
            "123 Main St",
            "Apt 456",
            "Cityville"
          ],
          "orderNotes": [
            "Special request: Handle with care",
            "Gift wrapping required"
          ],
          "customerName": "John Doe",
          "totalAmount": 99.99,
          "paymentMethod": "Credit Card"
        }
        """)
      .setResponseCode(HttpStatus.OK.value()));

    webTestClient.get()
      .uri("v1/order/1")
      .exchange()
      .expectStatus()
      .is5xxServerError();
}

The response from the external service contains additional attributes (customerName,  totalAmount, paymentMethod) which causes the test to fail.

5. Custom Deserialization Using WebClient Exchange Strategies Config

In certain situations, we might want to configure an ObjectMapper only for specific requests, and in that case, we need to register the mapper with ExchangeStrategies.

Let’s assume that the date format received is different in the above example and includes an offset.

We’ll add a CustomDeserializer, which will parse the received OffsetDateTime and convert it to the model LocalDateTime in UTC:

public class CustomDeserializer extends LocalDateTimeDeserializer {
    @Override
    public LocalDateTime deserialize(JsonParser jsonParser, DeserializationContext ctxt) throws IOException {
      try {
        return OffsetDateTime.parse(jsonParser.getText())
        .atZoneSameInstant(ZoneOffset.UTC)
        .toLocalDateTime();
      } catch (Exception e) {
          return super.deserialize(jsonParser, ctxt);
      }
    }
}

In a new implementation of ExternalServiceV2, let’s declare a new ObjectMapper that links with the above CustomDeserializer and register it with a new WebClient using ExchangeStrategies:

public WebClient.ResponseSpec findById(int id) {

    ObjectMapper objectMapper = new ObjectMapper().registerModule(new SimpleModule().addDeserializer(LocalDateTime.class, new CustomDeserializer()));

    WebClient webClient = WebClient.builder()
      .baseUrl("http://localhost:8090/")
      .exchangeStrategies(ExchangeStrategies.builder()
      .codecs(clientDefaultCodecsConfigurer -> {
        clientDefaultCodecsConfigurer.defaultCodecs()
        .jackson2JsonEncoder(new Jackson2JsonEncoder(objectMapper, MediaType.APPLICATION_JSON));
        clientDefaultCodecsConfigurer.defaultCodecs()
        .jackson2JsonDecoder(new Jackson2JsonDecoder(objectMapper, MediaType.APPLICATION_JSON));
      })
      .build())
    .build();

    return webClient.get().uri("external/order/" + id).retrieve();
}

We have linked this ObjectMapper exclusively with a specific API request, and it will not apply to any other requests within the application. Next, let’s add a GET /v2 endpoint that will invoke an external service using the above findById implementation along with a specific ObjectMapper:

@GetMapping(value = "v2/order/{id}", produces = MediaType.APPLICATION_JSON_VALUE)
public final Mono<OrderResponse> searchOrderV2(@PathVariable(value = "id") int id) {
    return externalServiceV2.findById(id)
      .bodyToMono(OrderResponse.class);
}

Finally, we’ll add a quick test where we pass a mocked orderDateTime with an offset and validate if it uses the CustomDeserializer to convert it to UTC:

@Test
void givenMockedExternalResponse_whenSearchByIdV2_thenOrderResponseShouldBeReceivedSuccessfully() {

    mockExternalService.enqueue(new MockResponse().addHeader("Content-Type", "application/json; charset=utf-8")
      .setBody("""
      {
        "orderId": "a1b2c3d4-e5f6-4a5b-8c9d-0123456789ab",
        "orderDateTime": "2024-01-20T14:34:56+01:00",
        "address": [
          "123 Main St",
          "Apt 456",
          "Cityville"
        ],
        "orderNotes": [
          "Special request: Handle with care",
          "Gift wrapping required"
        ]
      }
      """)
      .setResponseCode(HttpStatus.OK.value()));

    OrderResponse orderResponse = webTestClient.get()
      .uri("v2/order/1")
      .exchange()
      .expectStatus()
      .isOk()
      .expectBody(OrderResponse.class)
      .returnResult()
      .getResponseBody();
    assertEquals(UUID.fromString("a1b2c3d4-e5f6-4a5b-8c9d-0123456789ab"), orderResponse.getOrderId());
    assertEquals(LocalDateTime.of(2024, 1, 20, 13, 34, 56), orderResponse.getOrderDateTime());
    assertThat(orderResponse.getAddress()).hasSize(3);
    assertThat(orderResponse.getOrderNotes()).hasSize(2);
}

This test invokes the /v2 endpoint, which uses a specific ObjectMapper with CustomDeserializer to parse the order details response received from an external service.

6. Deserialize a JSON Array to List<T>

Moving on, we can deserialize a JSON array into a List by using a ParameterizedTypeReference. This is necessary because Java removes generic type information during compilation.

Suppose we have an external service that returns the following JSON array:

["123 Main St", "456 Oak Ave", "789 Pine Rd"]

Next, let’s write a service method that calls the external endpoint:

public WebClient.ResponseSpec orderAddress(List<String> address) {
    WebClient webClient = WebClient.builder()
      .baseUrl("http://localhost:8090/")
      .build();

    return webClient.get()
      .uri(uriBuilder -> uriBuilder.path("/external/order")
        .queryParam("address", address.toArray())
        .build())
      .retrieve();
}

Here, the address list is converted into multiple query parameters.

Next, let’s define the controller method:

@GetMapping(value = "v3/order", produces = MediaType.APPLICATION_JSON_VALUE)
public final Mono<List<String>> searchOrderV3(@RequestParam(value = "address") List<String> address) {
    return externalServiceV3.orderAddress(address)
      .bodyToMono(new ParameterizedTypeReference<List<String>>() {
      })
      .log();
}

Here, we use ParameterizedTypeReference<List<String>> to preserve the generic type information at runtime. Without it, due to type erasure, the WebClient wouldn’t be able to correctly deserialize the JSON array into a List<String>.

Finally, let’s write a unit test to verify this behavior:

@Test
void givenMockedExternalResponse_whenSearchByMultipleAddress_thenAddressShouldBeReceivedSuccessfully() {
        
    mockExternalService.enqueue(new MockResponse().addHeader("Content-Type", "application/json; charset=utf-8")
      .setBody("[\"123 Main St\", \"456 Oak Ave\", \"789 Pine Rd\"]")
      .setResponseCode(HttpStatus.OK.value()));
        
    List<String> address = webTestClient.get()
      .uri(uriBuilder -> uriBuilder.path("/v3/order")
        .queryParam("address", "123 Main St", "456 Oak Ave", "789 Pine Rd")
        .build())
      .exchange()
      .expectStatus()
      .isOk()
      .expectBody(new ParameterizedTypeReference<List<String>>() {
      })
      .returnResult()
      .getResponseBody();
    assertThat(address).isNotNull();
    assertThat(address).hasSize(3);
    assertThat(address).containsExactly("123 Main St", "456 Oak Ave", "789 Pine Rd");
}

The test above confirms that the JSON array is correctly deserialized into a List<String>.

7. Conclusion

In this article, we explored the need for custom deserialization and different ways to implement it. We first looked at registering a mapper for the entire application and also for specific requests. We can also use the same configurations to implement a custom serializer.