ETags for REST with Spring

Table of Contents

1. Overview

This article will focus on working with ETags in Spring, integration testing of the REST API and consumption scenarios with curl.

2. REST and ETags

From the official Spring documentation on ETag support:

An ETag (entity tag) is an HTTP response header returned by an HTTP/1.1 compliant web server used to determine change in content at a given URL.

ETags are used for two things – caching and conditional requests. The ETag value can be though as a hash computed out of the bytes of the Response body. Because a cryptographic hash function is likely used, even the smallest modification of the body will drastically change the output and thus the value of the ETag. This is only true for strong ETags – the protocol does provide a weak Etag as well.

Using an If-* header turns a standard GET request into a conditional GET. The two If-* headers that are using with ETags are “If-None-Match” and “If-Match” – each with it’s own semantics as discussed later in this article.

3. Client-Server communication with curl

A simple Client-Server communication involving ETags can be broken down into the steps:

- first, the Client makes a REST API call – the Response includes the ETag header to be stored for further use:

curl -H "Accept: application/json" -i http://localhost:8080/rest-sec/api/resources/1
HTTP/1.1 200 OK
ETag: "f88dd058fe004909615a64f01be66a7"
Content-Type: application/json;charset=UTF-8
Content-Length: 52

- next request the Client makes to the REST API includes the If-None-Match request header with the ETag value from the previous step; if the Resource has not changed on the Server, the Response will contain no body and a status code of 304 – Not Modified:

curl -H "Accept: application/json" -H 'If-None-Match: "f88dd058fe004909615a64f01be66a7"'
 -i http://localhost:8080/rest-sec/api/resources/1
HTTP/1.1 304 Not Modified
ETag: "f88dd058fe004909615a64f01be66a7"

- now, before retrieving the Resource again, we will change it by performing an update:

curl --user admin@fake.com:adminpass -H "Content-Type: application/json" -i 
  -X PUT --data '{ "id":1, "name":"newRoleName2", "description":"theNewDescription" }' 

http://localhost:8080/rest-sec/api/resources/1
HTTP/1.1 200 OK
ETag: "d41d8cd98f00b204e9800998ecf8427e" 
Content-Length: 0

- finally, we send out the the last request to retrieve the Privilege again; keep in mind that it has been updated since the last time it was retrieved, so the previous ETag value should no longer work – the response will contain the new data and a new ETag which, again, can be stored for further use:

curl -H "Accept: application/json" -H 'If-None-Match: "f88dd058fe004909615a64f01be66a7"' -i 

http://localhost:8080/rest-sec/api/resources/1
HTTP/1.1 200 OK
ETag: "03cb37ca667706c68c0aad4cb04c3a211"
Content-Type: application/json;charset=UTF-8
Content-Length: 56

And there you have it – ETags in the wild and saving bandwidth.

4. ETag support in Spring

On to the Spring support – to use ETag in Spring is extremely easy to set up and completely transparent for the application. The support is enabled by adding a simple Filter in the web.xml:

<filter>
   <filter-name>etagFilter</filter-name>
   <filter-class>org.springframework.web.filter.ShallowEtagHeaderFilter</filter-class>
</filter>
<filter-mapping>
   <filter-name>etagFilter</filter-name>
   <url-pattern>/api/*</url-pattern>
</filter-mapping>

The filter is mapped on the same URI pattern as the RESTful API itself. The filter itself is the standard implementation of ETag functionality since Spring 3.0.

The implementation is a shallow one – the ETag is calculated based on the response, which will save bandwidth but not server performance. So, a request that will benefit from the ETag support will still be processed as a standard request, consume any resource that it would normally consume (database connections, etc) and only before having it’s response returned back to the client will the ETag support kick in.

At that point the ETag will be calculated out of the Response body and set on the Resource itself; also, if the If-None-Match header was set on the Request, it will be handled as well.

A deeper implementation of the ETag mechanism could potentially provide much greater benefits – such as serving some requests from the cache and not having to perform the computation at all – but the implementation would most definitely not be as simple, nor as pluggable as the shallow approach described here.

5. Testing ETags

Let’s start simple – we need to verify that the response of a simple request retrieving a single Resource will actually return the “ETag” header:

@Test
public void givenResourceExists_whenRetrievingResource_thenEtagIsAlsoReturned() {
    // Given
    String uriOfResource = createAsUri();

    // When
    Response findOneResponse = RestAssured.given().header("Accept", "application/json").get(uriOfResource);

    // Then
    assertNotNull(findOneResponse.getHeader("ETag"));
}

Next, we verify the happy path of the ETag behaviour – if the Request to retrieve the Resource from the server uses the correct ETag value, then the Resource is no longer returned.

@Test
public void givenResourceWasRetrieved_whenRetrievingAgainWithEtag_thenNotModifiedReturned() {
    // Given
    String uriOfResource = createAsUri();
    Response findOneResponse = RestAssured.given().
      header("Accept", "application/json").get(uriOfResource);
    String etagValue = findOneResponse.getHeader(HttpHeaders.ETAG);

    // When
    Response secondFindOneResponse= RestAssured.given().
      header("Accept", "application/json").headers("If-None-Match", etagValue)
      .get(uriOfResource);

    // Then
    assertTrue(secondFindOneResponse.getStatusCode() == 304);
}

Step by step:

  • a Resource is first created and then retrieved – the ETag value is stored for further use
  • a new retrieve request is sent, this time with the “If-None-Match” header specifying the ETag value previously stored
  • on this second request, the server simply returns a 304 Not Modified, since the Resource itself has indeed not beeing modified between the two retrieval operations

Finally, we verify the case where the Resource is changed between the first and the second retrieval requests:

@Test
public void 
  givenResourceWasRetrievedThenModified_whenRetrievingAgainWithEtag_thenResourceIsReturned() {
    // Given
    String uriOfResource = createAsUri();
    Response findOneResponse = RestAssured.given().
      header("Accept", "application/json").get(uriOfResource);
    String etagValue = findOneResponse.getHeader(HttpHeaders.ETAG);

    existingResource.setName(randomAlphabetic(6));
    update(existingResource);

    // When
    Response secondFindOneResponse= RestAssured.given().
      header("Accept", "application/json").headers("If-None-Match", etagValue)
      .get(uriOfResource);

    // Then
    assertTrue(secondFindOneResponse.getStatusCode() == 200);
}

Step by step:

  • a Resource is first created and then retrieved – the ETag value is stored for further use
  • the same Resource is then updated
  • a new retrieve request is sent, this time with the “If-None-Match” header specifying the ETag value previously stored
  • on this second request, the server will returns a 200 OK along with the full Resource, since the ETag value is no longer correct, as the Resource has been updated in the meantime

Finally, the last test – which is not going to work because the functionality has not yet been implemented in Spring – is the support for the If-Match HTTP header:

@Test
public void givenResourceExists_whenRetrievedWithIfMatchIncorrectEtag_then412IsReceived() {
    // Given
    T existingResource = getApi().create(createNewEntity());

    // When
    String uriOfResource = baseUri + "/" + existingResource.getId();
    Response findOneResponse = RestAssured.given().header("Accept", "application/json").
      headers("If-Match", randomAlphabetic(8)).get(uriOfResource);

    // Then
    assertTrue(findOneResponse.getStatusCode() == 412);
}

Step by step:

  • a Resource is first created
  • the Resource is then retrieved with the “If-Match” header specifying an incorrect ETag value – this is a conditional GET request
  • the server should return a 412 Precondition Failed

6. ETags are BIG

We have only used ETags for read operations – a RFC exists trying to clarify how implementations should deal with ETags on write operations – this is not standard, but is an interesting read.

There are of course other possible uses of the ETag mechanism, such an for an Optimistic Locking Mechanism using Spring 3.1 as well as dealing with the related “Lost Update Problem”.

There are also several known potential pitfalls and caveats to be aware of when using ETags.

7. Conclusion

This article only scratched the surface with what’s possible with Spring and ETags.

For a full implementation of an ETag enabled RESTful service, along with integration tests verifying the ETag behavior, check out the github project – this is an Eclipse based project, so it should be easy to import and run as it is.

I usually post about REST APIs and HTTP on Google+ - you can follow me there:

GET THE 3 EBOOKS >>
Download the 3 eBooks - Build Your App "The Right Way"
×
Build Your Web App with Spring (and quickly prototype it to 90%)

, ,

  • beleck

    Are you aware of any deep etag support in Spring?

    I am attempting to use the @Version tag in hibernate to create the etag…

    • baeldung

      There is no out of the box support for that unfortunatelly, at least none that I’m aware of, but I have seen custom implementations that did that well.