ETags for REST with Spring

1. Overview

This article will focus on working with ETags in Spring, integration testing of the REST API and consumption scenarios with curl.

2. REST and ETags

From the official Spring documentation on ETag support:

We can use ETags for two things – caching and conditional requests. The ETag value can be thought of as a hash computed out of the bytes of the Response body. Because the service likely uses a cryptographic hash function, even the smallest modification of the body will drastically change the output and thus the value of the ETag. This is only true for strong ETags – the protocol does provide a weak Etag as well.

Using an If-* header turns a standard GET request into a conditional GET. The two If-* headers that are using with ETags are “If-None-Match” and “If-Match” – each with its own semantics as discussed later in this article.

3. Client-Server Communication With curl

We can break down a simple Client-Server communication involving ETags into the steps:

First, the Client makes a REST API call – the Response includes the ETag header that will be stored for further use:

curl -H "Accept: application/json" -i http://localhost:8080/spring-boot-rest/foos/1

HTTP/1.1 200 OK
ETag: "f88dd058fe004909615a64f01be66a7"
Content-Type: application/json;charset=UTF-8
Content-Length: 52

For the next request, the Client will include the If-None-Match request header with the ETag value from the previous step. If the Resource hasn’t changed on the Server, the Response will contain no body and a status code of 304 – Not Modified:

curl -H "Accept: application/json" -H 'If-None-Match: "f88dd058fe004909615a64f01be66a7"'
 -i http://localhost:8080/spring-boot-rest/foos/1

HTTP/1.1 304 Not Modified
ETag: "f88dd058fe004909615a64f01be66a7"

Now, before retrieving the Resource again, let’s change it by performing an update:

curl -H "Content-Type: application/json" -i 
  -X PUT --data '{ "id":1, "name":"Transformers2"}' 
    http://localhost:8080/spring-boot-rest/foos/1

HTTP/1.1 200 OK
ETag: "d41d8cd98f00b204e9800998ecf8427e" 
Content-Length: 0

Finally, we send out the last request to retrieve the Foo again. Keep in mind that we’ve updated it since the last time we requested it, so the previous ETag value should no longer work. The response will contain the new data and a new ETag which, again, can be stored for further use:

curl -H "Accept: application/json" -H 'If-None-Match: "f88dd058fe004909615a64f01be66a7"' -i 
  http://localhost:8080/spring-boot-rest/foos/1

HTTP/1.1 200 OK
ETag: "03cb37ca667706c68c0aad4cb04c3a211"
Content-Type: application/json;charset=UTF-8
Content-Length: 56

And there you have it – ETags in the wild and saving bandwidth.

4. ETag Support in Spring

On to the Spring support: using ETag in Spring is extremely easy to set up and completely transparent for the application. We can enable the support by adding a simple Filter in the web.xml:

<filter>
   <filter-name>etagFilter</filter-name>
   <filter-class>org.springframework.web.filter.ShallowEtagHeaderFilter</filter-class>
</filter>
<filter-mapping>
   <filter-name>etagFilter</filter-name>
   <url-pattern>/foos/*</url-pattern>
</filter-mapping>

We’re mapping the filter on the same URI pattern as the RESTful API itself. The filter itself is the standard implementation of ETag functionality since Spring 3.0.

The implementation is a shallow one – the application calculates the ETag based on the response, which will save bandwidth but not server performance.

So, a request that will benefit from the ETag support will still be processed as a standard request, consume any resource that it would normally consume (database connections, etc) and only before having its response returned back to the client will the ETag support kick in.

At that point the ETag will be calculated out of the Response body and set on the Resource itself; also, if the If-None-Match header was set on the Request, it will be handled as well.

A deeper implementation of the ETag mechanism could potentially provide much greater benefits – such as serving some requests from the cache and not having to perform the computation at all – but the implementation would most definitely not be as simple, nor as pluggable as the shallow approach described here.

4.1. Java Based Configuration

Let’s see how the Java-based configuration would look like by declaring a ShallowEtagHeaderFilter bean in our Spring context:

@Bean
public ShallowEtagHeaderFilter shallowEtagHeaderFilter() {
    return new ShallowEtagHeaderFilter();
}

Keep in mind that if we need to provide further filter configurations, we can instead declare a FilterRegistrationBean instance:

@Bean
public FilterRegistrationBean<ShallowEtagHeaderFilter> shallowEtagHeaderFilter() {
    FilterRegistrationBean<ShallowEtagHeaderFilter> filterRegistrationBean
      = new FilterRegistrationBean<>( new ShallowEtagHeaderFilter());
    filterRegistrationBean.addUrlPatterns("/foos/*");
    filterRegistrationBean.setName("etagFilter");
    return filterRegistrationBean;
}

Finally, if we’re not using Spring Boot we can set up the filter using the AbstractAnnotationConfigDispatcherServletInitializer‘s getServletFilters method.

4.2. Using the ResponseEntity’s eTag() Method

This method was introduced in Spring framework 4.1, and we can use it to control the ETag value that a single endpoint retrieves.

For instance, imagine we’re using versioned entities as an Optimist Locking mechanism to access our database information.

We can use the version itself as the ETag to indicate if the entity has been modified:

@GetMapping(value = "/{id}/custom-etag")
public ResponseEntity<Foo>
  findByIdWithCustomEtag(@PathVariable("id") final Long id) {

    // ...Foo foo = ...

    return ResponseEntity.ok()
      .eTag(Long.toString(foo.getVersion()))
      .body(foo);
}

The service will retrieve the corresponding 304-Not Modified state if the request’s conditional header matches the caching data.

5. Testing ETags

Let’s start simple – we need to verify that the response of a simple request retrieving a single Resource will actually return the “ETag” header:

@Test
public void givenResourceExists_whenRetrievingResource_thenEtagIsAlsoReturned() {
    // Given
    String uriOfResource = createAsUri();

    // When
    Response findOneResponse = RestAssured.given().
      header("Accept", "application/json").get(uriOfResource);

    // Then
    assertNotNull(findOneResponse.getHeader("ETag"));
}

Next, we verify the happy path of the ETag behavior. If the Request to retrieve the Resource from the server uses the correct ETag value, then the server doesn’t retrieve the Resource:

@Test
public void givenResourceWasRetrieved_whenRetrievingAgainWithEtag_thenNotModifiedReturned() {
    // Given
    String uriOfResource = createAsUri();
    Response findOneResponse = RestAssured.given().
      header("Accept", "application/json").get(uriOfResource);
    String etagValue = findOneResponse.getHeader(HttpHeaders.ETAG);

    // When
    Response secondFindOneResponse= RestAssured.given().
      header("Accept", "application/json").headers("If-None-Match", etagValue)
      .get(uriOfResource);

    // Then
    assertTrue(secondFindOneResponse.getStatusCode() == 304);
}

Step by step:

• we create and retrieve a Resource, storing the ETag value
• send a new retrieve request, this time with the “If-None-Match” header specifying the ETag value previously stored
• on this second request, the server simply returns a 304 Not Modified, since the Resource itself has indeed not beeing modified between the two retrieval operations

Finally, we verify the case where the Resource is changed between the first and the second retrieval requests:

@Test
public void 
  givenResourceWasRetrievedThenModified_whenRetrievingAgainWithEtag_thenResourceIsReturned() {
    // Given
    String uriOfResource = createAsUri();
    Response findOneResponse = RestAssured.given().
      header("Accept", "application/json").get(uriOfResource);
    String etagValue = findOneResponse.getHeader(HttpHeaders.ETAG);

    existingResource.setName(randomAlphabetic(6));
    update(existingResource);

    // When
    Response secondFindOneResponse= RestAssured.given().
      header("Accept", "application/json").headers("If-None-Match", etagValue)
      .get(uriOfResource);

    // Then
    assertTrue(secondFindOneResponse.getStatusCode() == 200);
}

Step by step:

• we first create and retrieve a Resource – and store the ETag value for further use
• then we update the same Resource
• send a new GET request, this time with the “If-None-Match” header specifying the ETag that we previously stored
• on this second request, the server will return a 200 OK along with the full Resource, since the ETag value is no longer correct, as we updated the Resource in the meantime

Finally, the last test – which is not going to work because the functionality has not yet been implemented in Spring – is the support for the If-Match HTTP header:

@Test
public void givenResourceExists_whenRetrievedWithIfMatchIncorrectEtag_then412IsReceived() {
    // Given
    T existingResource = getApi().create(createNewEntity());

    // When
    String uriOfResource = baseUri + "/" + existingResource.getId();
    Response findOneResponse = RestAssured.given().header("Accept", "application/json").
      headers("If-Match", randomAlphabetic(8)).get(uriOfResource);

    // Then
    assertTrue(findOneResponse.getStatusCode() == 412);
}

Step by step:

• we create a Resource
• then retrieve it using the “If-Match” header specifying an incorrect ETag value – this is a conditional GET request
• the server should return a 412 Precondition Failed

6. ETags Are Big

We have only used ETags for read operations. An RFC exists trying to clarify how implementations should deal with ETags on write operations – this is not standard, but is an interesting read.

There are of course other possible uses of the ETag mechanism, such as for an Optimistic Locking Mechanism as well as dealing with the related “Lost Update Problem”.

There are also several known potential pitfalls and caveats to be aware of when using ETags.

7. Conclusion

This article only scratched the surface with what’s possible with Spring and ETags.