Binary Data Formats in a Spring REST API

1. Overview

While JSON and XML are widely popular data transfer formats when it comes to REST APIs, they’re not the only options available.

There exist many other formats with varying degree of serialization speed and serialized data size.

In this article we explore how to configure a Spring REST mechanism to use binary data formats – which we illustrate with Kryo.

Moreover we show how to support multiple data formats by adding support for Google Protocol buffers.

2. HttpMessageConverter

HttpMessageConverter interface is basically Spring’s public API for the conversion of REST data formats.

There are different ways to specify the desired converters. Here we implement WebMvcConfigurer and explicitly provide the converters we want to use in the overridden configureMessageConverters method:

@Configuration
@EnableWebMvc
@ComponentScan({ "com.baeldung.web" })
public class WebConfig implements WebMvcConfigurer {
    @Override
    public void configureMessageConverters(List<HttpMessageConverter<?>> messageConverters) {
        //...
    }
}

3. Kryo

3.1. Kryo Overview and Maven

Kryo is a binary encoding format that provides good serialization and deserialization speed and smaller transferred data size compared to text-based formats.

While in theory it can be used to transfer data between different kinds of systems, it is primarily designed to work with Java components.

We add the necessary Kryo libraries with the following Maven dependency:

<dependency>
    <groupId>com.esotericsoftware</groupId>
    <artifactId>kryo</artifactId>
    <version>4.0.0</version>
</dependency>

To check the latest version of kryo you can have a look here.

3.2. Kryo in Spring REST

In order to utilize Kryo as data transfer format, we create a custom HttpMessageConverter and implement the necessary serialization and deserialization logic. Also, we define our custom HTTP header for Kryo: application/x-kryo. Here is a full simplified working example which we use for demonstration purposes:

public class KryoHttpMessageConverter extends AbstractHttpMessageConverter<Object> {

    public static final MediaType KRYO = new MediaType("application", "x-kryo");

    private static final ThreadLocal<Kryo> kryoThreadLocal = new ThreadLocal<Kryo>() {
        @Override
        protected Kryo initialValue() {
            Kryo kryo = new Kryo();
            kryo.register(Foo.class, 1);
            return kryo;
        }
    };

    public KryoHttpMessageConverter() {
        super(KRYO);
    }

    @Override
    protected boolean supports(Class<?> clazz) {
        return Object.class.isAssignableFrom(clazz);
    }

    @Override
    protected Object readInternal(
      Class<? extends Object> clazz, HttpInputMessage inputMessage) throws IOException {
        Input input = new Input(inputMessage.getBody());
        return kryoThreadLocal.get().readClassAndObject(input);
    }

    @Override
    protected void writeInternal(
      Object object, HttpOutputMessage outputMessage) throws IOException {
        Output output = new Output(outputMessage.getBody());
        kryoThreadLocal.get().writeClassAndObject(output, object);
        output.flush();
    }

    @Override
    protected MediaType getDefaultContentType(Object object) {
        return KRYO;
    }
}

Notice we’re using a ThreadLocal here simply because the creation of Kryo instances can get expensive, and we want to re-utilize these as much as we can.

The controller method is straightforward (note there is no need for any custom protocol-specific data types, we use plain Foo DTO):

@RequestMapping(method = RequestMethod.GET, value = "/foos/{id}")
@ResponseBody
public Foo findById(@PathVariable long id) {
    return fooRepository.findById(id);
}

And a quick test to prove that we have wired everything together correctly:

RestTemplate restTemplate = new RestTemplate();
restTemplate.setMessageConverters(Arrays.asList(new KryoHttpMessageConverter()));

HttpHeaders headers = new HttpHeaders();
headers.setAccept(Arrays.asList(KryoHttpMessageConverter.KRYO));
HttpEntity<String> entity = new HttpEntity<String>(headers);

ResponseEntity<Foo> response = restTemplate.exchange("http://localhost:8080/spring-rest/foos/{id}",
  HttpMethod.GET, entity, Foo.class, "1");
Foo resource = response.getBody();

assertThat(resource, notNullValue());

4. Supporting Multiple Data Formats

Often you would want to provide support for multiple data formats for the same service. The clients specify the desired data formats in the Accept HTTP header, and the corresponding message converter is invoked to serialize the data.

Usually, you just have to register another converter for things to work out of the box. Spring picks the appropriate converter automatically based on the value in the Accept header and the supported media types declared in the converters.

For example, to add support for both JSON and Kryo, register both KryoHttpMessageConverter and MappingJackson2HttpMessageConverter:

@Override
public void configureMessageConverters(List<HttpMessageConverter<?>> messageConverters) {
    messageConverters.add(new MappingJackson2HttpMessageConverter());
    messageConverters.add(new KryoHttpMessageConverter());
    super.configureMessageConverters(messageConverters);
}

Now, let’s suppose that we want to add Google Protocol Buffer to the list as well. For this example, we assume there is a class FooProtos.Foo generated with the protoc compiler based on the following proto file:

package baeldung;
option java_package = "com.baeldung.web.dto";
option java_outer_classname = "FooProtos";
message Foo {
    required int64 id = 1;
    required string name = 2;
}

Spring comes with some built-in support for Protocol Buffer. All we need to make it work is to include ProtobufHttpMessageConverter in the list of supported converters:

@Override
public void configureMessageConverters(List<HttpMessageConverter<?>> messageConverters) {
    messageConverters.add(new MappingJackson2HttpMessageConverter());
    messageConverters.add(new KryoHttpMessageConverter());
    messageConverters.add(new ProtobufHttpMessageConverter());
}

However, we have to define a separate controller method that returns FooProtos.Foo instances (JSON and Kryo both deal with Foos, so no changes are needed in the controller to distinguish the two).

There are two ways to resolve the ambiguity about which method gets called. The first approach is to use different URLs for protobuf and other formats. For example, for protobuf:

@RequestMapping(method = RequestMethod.GET, value = "/fooprotos/{id}")
@ResponseBody
public FooProtos.Foo findProtoById(@PathVariable long id) { … }

and for the others:

@RequestMapping(method = RequestMethod.GET, value = "/foos/{id}")
@ResponseBody
public Foo findById(@PathVariable long id) { … }

Notice that for protobuf we use value = “/fooprotos/{id}” and for the other formats value = “/foos/{id}”.

The second – and better approach is to use the same URL, but to explicitly specify the produced data format in the request mapping for protobuf:

@RequestMapping(
  method = RequestMethod.GET, 
  value = "/foos/{id}", 
  produces = { "application/x-protobuf" })
@ResponseBody
public FooProtos.Foo findProtoById(@PathVariable long id) { … }

Note that by specifying the media type in the produces annotation attribute we give a hint to the underlying Spring mechanism about which mapping needs to be used based on the value in the Accept header provided by clients, so there is no ambiguity about which method needs to be invoked for the “foos/{id}” URL.

The second approach enables us to provide a uniform and consistent REST API to the clients for all data formats.

Finally, if you’re interested in going deeper into using Protocol Buffers with a Spring REST API, have a look at the reference article.

5. Registering Extra Message Converters

It is very important to note that you lose all of the default message converters when you override the configureMessageConverters method. Only the ones you provide will be used.

While sometimes this is exactly what you want, in many cases you just want to add new converters, while still keeping the default ones which already take care of standard data formats like JSON. To achieve this, override the extendMessageConverters method:

@Configuration
@EnableWebMvc
@ComponentScan({ "com.baeldung.web" })
public class WebConfig implements WebMvcConfigurer {
    @Override
    public void extendMessageConverters(List<HttpMessageConverter<?>> messageConverters) {
        messageConverters.add(new ProtobufHttpMessageConverter());
        messageConverters.add(new KryoHttpMessageConverter());
    }
}

6. Conclusion

In this tutorial, we looked at how easy it is to use any data transfer format in Spring MVC, and we examined this by using Kryo as an example.

We also showed how to add support for multiple formats so that different clients are able to use different formats.