Spring Top

Finally announcing my next course. The intro price of the upcoming “Learn Spring” course will permanently increase by $50 on Friday:

>> HAVE A LOOK

REST Top

Finally announcing my next course. The intro price of the upcoming “Learn Spring” course will permanently increase by $50 on Friday:

>> HAVE A LOOK

1. Overview

This tutorial will focus on the implementation of pagination in a REST API, using Spring MVC and Spring Data.

Further reading:

Pagination with Spring REST and AngularJS table

An extensive look at how to implement a simple API with pagination with Spring and how to consume it with AngularJS and UI Grid.

Read more

JPA Pagination

Pagination in JPA - how to use JQL and the Criteria API to do pagination correctly.

Read more

REST API Discoverability and HATEOAS

HATEOAS and Discoverability of a REST Service - driven by tests.

Read more

2. Page as Resource vs Page as Representation

The first question when designing pagination in the context of a RESTful architecture is whether to consider the page an actual Resource or just a Representation of Resources.

Treating the page itself as a resource introduces a host of problems such as no longer being able to uniquely identify resources between calls. This, coupled with the fact that, in the persistence layer, the page is not a proper entity but a holder that is constructed when necessary, makes the choice straightforward: the page is part of the representation.

The next question in the pagination design in the context of REST is where to include the paging information:

  • in the URI path: /foo/page/1
  • the URI query: /foo?page=1

Keeping in mind that a page is not a Resource, encoding the page information in the URI is no longer an option.

We’re going to use the standard way of solving this problem by encoding the paging information in a URI query.

3. The Controller

Now, for the implementation – the Spring MVC Controller for pagination is straightforward:

@GetMapping(params = { "page", "size" })
public List<Foo> findPaginated(@RequestParam("page") int page, 
  @RequestParam("size") int size, UriComponentsBuilder uriBuilder,
  HttpServletResponse response) {
    Page<Foo> resultPage = service.findPaginated(page, size);
    if (page > resultPage.getTotalPages()) {
        throw new MyResourceNotFoundException();
    }
    eventPublisher.publishEvent(new PaginatedResultsRetrievedEvent<Foo>(
      Foo.class, uriBuilder, response, page, resultPage.getTotalPages(), size));

    return resultPage.getContent();
}

In this example, we’re injecting the two query parameters, size and page, in the Controller method via @RequestParam.

Alternatively, we could have used a Pageable object, which maps the pagesize, and sort parameters automatically. In addition, the PagingAndSortingRepository entity provides out-of-the-box methods that support using the Pageable as a parameter as well.

We’re also injecting both the Http Response and the UriComponentsBuilder to help with Discoverability – which we’re decoupling via a custom event. If that’s not a goal of the API, you can simply remove the custom event.

Finally – note that the focus of this article is only the REST and the web layer – to go deeper into the data access part of pagination you can check out this article about Pagination with Spring Data.

4. Discoverability for REST Pagination

Within the scope of pagination, satisfying the HATEOAS constraint of REST means enabling the client of the API to discover the next and previous pages based on the current page in the navigation. For this purpose, we’re going to use the Link HTTP header, coupled with the “next“, “prev“, “first” and “last” link relation types.

In REST, Discoverability is a cross-cutting concern, applicable not only to specific operations but to types of operations. For example, each time a Resource is created, the URI of that Resource should be discoverable by the client. Since this requirement is relevant for the creation of ANY Resource, we’ll handle it separately.

We’ll decouple these concerns using events, as we discussed in the previous article focusing on Discoverability of a REST Service. In the case of pagination, the event – PaginatedResultsRetrievedEvent – is fired in the controller layer. Then we’ll implement discoverability with a custom listener for this event.

In short, the listener will check if the navigation allows for a nextpreviousfirst and last pages. If it does – it will add the relevant URIs to the response as a ‘Link’ HTTP Header.

Let’s go step by step now. The UriComponentsBuilder passed from the controller contains only the base URL (the host, the port and the context path). Therefore, we’ll have to add the remaining sections:

void addLinkHeaderOnPagedResourceRetrieval(
 UriComponentsBuilder uriBuilder, HttpServletResponse response,
 Class clazz, int page, int totalPages, int size ){

   String resourceName = clazz.getSimpleName().toString().toLowerCase();
   uriBuilder.path( "/admin/" + resourceName );

    // ...
   
}

Next, we’ll use a StringJoiner to concatenate each link. We’ll use the uriBuilder to generate the URIs. Let’s see how we’d proceed with the link to the next page:

StringJoiner linkHeader = new StringJoiner(", ");
if (hasNextPage(page, totalPages)){
    String uriForNextPage = constructNextPageUri(uriBuilder, page, size);
    linkHeader.add(createLinkHeader(uriForNextPage, "next"));
}

Let’s have a look at the logic of the constructNextPageUri method:

String constructNextPageUri(UriComponentsBuilder uriBuilder, int page, int size) {
    return uriBuilder.replaceQueryParam(PAGE, page + 1)
      .replaceQueryParam("size", size)
      .build()
      .encode()
      .toUriString();
}

We’ll proceed similarly for the rest of the URIs that we want to include.

Finally, we’ll add the output as a response header:

response.addHeader("Link", linkHeader.toString());

Note that, for brevity, I included only a partial code sample and the full code here.

5. Test Driving Pagination

Both the main logic of pagination and discoverability are covered by small, focused integration tests. As in the previous article, we’ll use the REST-assured library to consume the REST service and to verify the results.

These are a few examples of pagination integration tests; for a full test suite, check out the GitHub project (link at the end of the article):

@Test
public void whenResourcesAreRetrievedPaged_then200IsReceived(){
    Response response = givenAuth().get(paths.getFooURL() + "?page=0&size=2");

    assertThat(response.getStatusCode(), is(200));
}
@Test
public void whenPageOfResourcesAreRetrievedOutOfBounds_then404IsReceived(){
    String url = getFooURL() + "?page=" + randomNumeric(5) + "&size=2";
    Response response = givenAuth().get(url);

    assertThat(response.getStatusCode(), is(404));
}
@Test
public void givenResourcesExist_whenFirstPageIsRetrieved_thenPageContainsResources(){
   createResource();
   Response response = givenAuth().get(paths.getFooURL() + "?page=0&size=2");

   assertFalse(response.body().as(List.class).isEmpty());
}

6. Test Driving Pagination Discoverability

Testing that pagination is discoverable by a client is relatively straightforward, although there is a lot of ground to cover.

The tests will focus on the position of the current page in navigation and the different URIs that should be discoverable from each position:

@Test
public void whenFirstPageOfResourcesAreRetrieved_thenSecondPageIsNext(){
   Response response = givenAuth().get(getFooURL()+"?page=0&size=2");

   String uriToNextPage = extractURIByRel(response.getHeader("Link"), "next");
   assertEquals(getFooURL()+"?page=1&size=2", uriToNextPage);
}
@Test
public void whenFirstPageOfResourcesAreRetrieved_thenNoPreviousPage(){
   Response response = givenAuth().get(getFooURL()+"?page=0&size=2");

   String uriToPrevPage = extractURIByRel(response.getHeader("Link"), "prev");
   assertNull(uriToPrevPage );
}
@Test
public void whenSecondPageOfResourcesAreRetrieved_thenFirstPageIsPrevious(){
   Response response = givenAuth().get(getFooURL()+"?page=1&size=2");

   String uriToPrevPage = extractURIByRel(response.getHeader("Link"), "prev");
   assertEquals(getFooURL()+"?page=0&size=2", uriToPrevPage);
}
@Test
public void whenLastPageOfResourcesIsRetrieved_thenNoNextPageIsDiscoverable(){
   Response first = givenAuth().get(getFooURL()+"?page=0&size=2");
   String uriToLastPage = extractURIByRel(first.getHeader("Link"), "last");

   Response response = givenAuth().get(uriToLastPage);

   String uriToNextPage = extractURIByRel(response.getHeader("Link"), "next");
   assertNull(uriToNextPage);
}

Note that the full low-level code for extractURIByRel – responsible for extracting the URIs by rel relation is here.

7. Getting All Resources

On the same topic of pagination and discoverability, the choice must be made if a client is allowed to retrieve all the Resources in the system at once, or if the client must ask for them paginated.

If the choice is made that the client cannot retrieve all Resources with a single request, and pagination is not optional but required, then several options are available for the response to a get all request. One option is to return a 404 (Not Found) and use the Link header to make the first page discoverable:

Link=<http://localhost:8080/rest/api/admin/foo?page=0&size=2>; rel=”first”, <http://localhost:8080/rest/api/admin/foo?page=103&size=2>; rel=”last”

Another option is to return redirect – 303 (See Other) – to the first page. A more conservative route would be to simply return to the client a 405 (Method Not Allowed) for the GET request.

8. REST Paging with Range HTTP Headers

A relatively different way of implementing pagination is to work with the HTTP Range headersRange, Content-Range, If-Range, Accept-Ranges – and HTTP status codes – 206 (Partial Content), 413 (Request Entity Too Large), 416 (Requested Range Not Satisfiable).

One view on this approach is that the HTTP Range extensions were not intended for pagination and that they should be managed by the Server, not by the Application. Implementing pagination based on the HTTP Range header extensions is nevertheless technically possible, although not nearly as common as the implementation discussed in this article.

9. Spring Data REST Pagination

In Spring Data, if we need to return a few results from the complete data set, we can use any Pageable repository method, as it will always return a Page. The results will be returned based on the page number, page size, and sorting direction.

Spring Data REST automatically recognizes URL parameters like page, size, sort etc.

To use paging methods of any repository we need to extend PagingAndSortingRepository:

public interface SubjectRepository extends PagingAndSortingRepository<Subject, Long>{}

If we call http://localhost:8080/subjects Spring automatically adds the page, size, sort parameters suggestions with the API:

"_links" : {
  "self" : {
    "href" : "http://localhost:8080/subjects{?page,size,sort}",
    "templated" : true
  }
}

By default, the page size is 20 but we can change it by calling something like http://localhost:8080/subjects?page=10.

If we want to implement paging into our own custom repository API we need to pass an additional Pageable parameter and make sure that API returns a Page:

@RestResource(path = "nameContains")
public Page<Subject> findByNameContaining(@Param("name") String name, Pageable p);

Whenever we add a custom API a /search endpoint gets added to the generated links. So if we call http://localhost:8080/subjects/search we will see a pagination capable endpoint:

"findByNameContaining" : {
  "href" : "http://localhost:8080/subjects/search/nameContains{?name,page,size,sort}",
  "templated" : true
}

All APIs that implement PagingAndSortingRepository will return a Page. If we need to return the list of the results from the Page, the getContent() API of Page provides the list of records fetched as a result of the Spring Data REST API.

The code in this section is available in the spring-data-rest project.

10. Conclusion

This article illustrated how to implement Pagination in a REST API using Spring, and discussed how to set up and test Discoverability.

If you want to go in depth on pagination in the persistence level, check out my JPA or Hibernate pagination tutorials.

The implementation of all these examples and code snippets can be found in the GitHub project – this is a Maven-based project, so it should be easy to import and run as it is.

Spring bottom

Finally announcing my next course. The intro price of the upcoming “Learn Spring” course will permanently increase by $50 on Friday:

>> HAVE A LOOK

REST bottom

Finally announcing my next course. The intro price of the upcoming “Learn Spring” course will permanently increase by $50 on Friday:

>> HAVE A LOOK

newest oldest most voted
Notify of
Guest
Guest
Guest

What the deal with those freaking hugeee methods name “whenSecondPageOfResourcesAreRetrieved_thenFirstPageIsPrevious”? You really use those for the real stuff? Damn,,,,,,,,,,,,

Tommy McGuire
Guest
Tommy McGuire

What do you mean by “This approach does however have one downside – it cuts into the query space for actual queries…”? Are you referring to the size of the query string on the URL, or the number of query parameters?

Eugen Paraschiv
Guest

I am mainly referring to the fact that it adds to the length of the query part of the URL – not a major problem.

John
Guest
John

Using the query string for paging can also cause problems with caching.

Eugen Paraschiv
Guest

Agreed – this may make some types of caching mechanism more difficult – but we are dealing with lists of resources here, so presumably these will change anyhow – this makes caching is a smaller concern.
Thanks. Eugen.

pik
Guest
pik

Hi, what about collections that can be seen from different point of views ? Eg: Event and Person both have Tickets{person,event,+data}. In this case also there is almost no sense in seeing all tickets of all events. When I start from a Ticket how can I link both the collections ? With a compound rel ? “collection-person” “collection-event” ? Also I had a problem with LINK HTTP header for its size limit..I found myself having to check and skip produced header when search forms are involved and query string gets long, also if tomcat has 8K as default limit I… Read more »

Eugen Paraschiv
Guest

Using custom rel types and defining your own custom media type is one way to go, and that would indeed enable you to use these types of link relations. Another is to just use the Link header – it’s not as semantically rich, but it’s standard, so any client will implicitly understand it. So – you could simply point to the URI of the list of tickets via a Link. As for loading up the header with so much information – it’s probably better to treat that as a code smell and try to improve the your responses instead of… Read more »

PA
Guest
PA

Hello, I’m running your “spring-security-rest-full” code on eclipse, but it gives me error:

You need to run build with JDK or have tools.jar on the classpath.If this occures during eclipse build make sure you run eclipse under JDK as well (com.mysema.maven:apt-maven-plugin:1.1.3:process:default:generate-sources).

I am looking to implement Pagination with HATEOAS and ResourceAssembler class. Please help me. Provide code.

Eugen Paraschiv
Guest

You’re probably running on a fresh workspace in Eclipse, and it picked up your JRE instead of your JDK. Go into Preferences -> Java -> Installed JREs and add your JDK in there (replacing the JRE). Hope that helps. Cheers,
Eugen.