Generic Top

I just announced the new Learn Spring course, focused on the fundamentals of Spring 5 and Spring Boot 2:

>> CHECK OUT THE COURSE

1. Overview

In this tutorial, we'll explore the basics of Spring Data Elasticsearch in a code-focused and practical manner.

We'll show how to index, search, and query Elasticsearch in a Spring application using Spring Data Elasticsearch. Spring Data Elasticseach is a Spring module that implements Spring Data thus offering a way to interact with the popular open-source, Lucene-based search engine.

While Elasticsearch can work with no hardly defined schema, it's a common practice to design one and create mappings specifying the type of data to be expected in certain fields. When a document is indexed, its fields are processed according to their types. For example, a text field will be tokenized and filtered according to mapping rules. We could also create filters and tokenizers of our own.

For the sake of simplicity, we'll use a docker image for our Elasticsearch instance, though any Elasticsearch instance listening on port 9200 will do.

We start by firing up our Elasticsearch instance:

docker run -d --name es762 -p 9200:9200 -e "discovery.type=single-node" elasticsearch:7.6.2

2. Spring Data

Spring Data helps avoid boilerplate code. For example, if we define a repository interface that extends the ElasticsearchRepository interface provided by Spring Data Elasticsearch, CRUD operations for the corresponding document class will be made available by default.

Additionally, simply by declaring methods with names in a predefined format, method implementations are generated for us – there is no need to write an implementation of the repository interface.

The Baeldung guides on Spring Data provide the essentials to get started on the topic.

2.1. Maven Dependency

Spring Data Elasticsearch provides a Java API for the search engine. In order to use it we need to add a new dependency to the pom.xml:

<dependency>
    <groupId>org.springframework.data</groupId>
    <artifactId>spring-data-elasticsearch</artifactId>
    <version>4.0.0.RELEASE</version>
</dependency>

2.2. Defining Repository Interfaces

In order to define new repositories, we extend one of the provided repository interfaces, replacing the generic types with our actual document and primary key types.

It's important to note that ElasticsearchRepository extends from PagingAndSortingRepository. This allows built-in support for pagination and sorting.

In our example, we will use the paging feature in our custom search methods:

public interface ArticleRepository extends ElasticsearchRepository<Article, String> {

    Page<Article> findByAuthorsName(String name, Pageable pageable);

    @Query("{\"bool\": {\"must\": [{\"match\": {\"authors.name\": \"?0\"}}]}}")
    Page<Article> findByAuthorsNameUsingCustomQuery(String name, Pageable pageable);
}

With the findByAuthorsName method, the repository proxy will create an implementation based on the method name. The resolution algorithm will determine that it needs to access the authors property and then search the name property of each item.

The second method, findByAuthorsNameUsingCustomQuery, uses a custom Elasticsearch boolean query, defined using the @Query annotation, which requires strict matching between the author's name and the provided name argument.

2.3. Java Configuration

When configuring Elasticsearch in our Java application, we need to define how we connect to the Elasticsearch instance. For that, we use a RestHighLevelClient that is offered by the Elasticsearch dependency:

@Configuration
@EnableElasticsearchRepositories(basePackages = "com.baeldung.spring.data.es.repository")
@ComponentScan(basePackages = { "com.baeldung.spring.data.es.service" })
public class Config {

    @Bean
    public RestHighLevelClient client() {
        ClientConfiguration clientConfiguration 
            = ClientConfiguration.builder()
                .connectedTo("localhost:9200")
                .build();

        return RestClients.create(clientConfiguration).rest();
    }

    @Bean
    public ElasticsearchOperations elasticsearchTemplate() {
        return new ElasticsearchRestTemplate(client());
    }
}

We're using a standard Spring-enabled style annotation. @EnableElasticsearchRepositories will make Spring Data Elasticsearch scan the provided package for Spring Data repositories.

In order to communicate with our Elasticsearch server, we use a simple RestHighLevelClient. While Elasticsearch provides multiple types of clients, using the RestHighLevelClient is a good way to future-proof the communication with the server.

Finally, we set up an ElasticsearchOperations bean to execute operations on our server. In this case, we instantiate an ElasticsearchRestTemplate.

3. Mappings

Mappings are used to define a schema for our documents. By defining a schema to our documents, we protect them from undesired outcomes such as mapping to a type we wouldn't want to.

Our entity is a simple document named Article where the id is of the type String. We also specify that such documents must be stored in an index named blog within the article type.

@Document(indexName = "blog", type = "article")
public class Article {

    @Id
    private String id;
    
    private String title;
    
    @Field(type = FieldType.Nested, includeInParent = true)
    private List<Author> authors;
    
    // standard getters and setters
}

Indexes can have several types. We can use that feature to implement hierarchies.

The authors field is marked as FieldType.Nested. This allows us to define the Author class separately, but have the individual instances of author embedded in an Article document when it is indexed in Elasticsearch.

4. Indexing Documents

Spring Data Elasticsearch generally auto-creates indexes based on the entities in the project. However, we can also create an index programmatically via the client template:

elasticsearchTemplate.indexOps(Article.class).create();

Then, we can add documents to the index:

Article article = new Article("Spring Data Elasticsearch");
article.setAuthors(asList(new Author("John Smith"), new Author("John Doe")));
articleRepository.save(article);

5. Querying

5.1. Method Name-Based Query

When we use the method name-based query, we write methods that define the query we want to perform. During the setup, Spring Data will parse the method signature and create the queries accordingly:

String nameToFind = "John Smith";
Page<Article> articleByAuthorName
  = articleRepository.findByAuthorsName(nameToFind, PageRequest.of(0, 10));

By calling findByAuthorsName with a PageRequest object, we obtain the first page of results (page numbering is zero-based), with that page containing at most 10 articles. The page object also provides the total number of hits for the query along with other handy pagination information.

5.2. A Custom Query

There are a couple of ways to define custom queries for Spring Data Elasticsearch repositories. One way is to use the @Query annotation, as demonstrated in section 2.2.

Another option is to use the query builder to create our custom query.

Having to search for articles that have the word “data” in the title, we could just create a NativeSearchQueryBuilder with a Filter on the title:

Query searchQuery = new NativeSearchQueryBuilder()
   .withFilter(regexpQuery("title", ".*data.*"))
   .build();
SearchHits<Article> articles = 
   elasticsearchTemplate.search(searchQuery, Article.class, IndexCoordinates.of("blog");

6. Updating and Deleting

In order to update a document, we first must retrieve it:

String articleTitle = "Spring Data Elasticsearch";
Query searchQuery = new NativeSearchQueryBuilder()
  .withQuery(matchQuery("title", articleTitle).minimumShouldMatch("75%"))
  .build();

SearchHits<Article> articles = 
   elasticsearchTemplate.search(searchQuery, Article.class, IndexCoordinates.of("blog");
Article article = articles.getSearchHit(0).getContent();

Then, we can make changes to the document just by editing the content of the object using its assessors:

article.setTitle("Getting started with Search Engines");
articleRepository.save(article);

As for deleting, there are several options. We can retrieve the document and delete it using the delete method:

articleRepository.delete(article);

We can also delete it by id when it's known:

articleRepository.deleteById("article_id");

It is also possible to create custom deleteBy queries and make use of the bulk delete feature offered by Elasticsearch:

articleRepository.deleteByTitle("title");

7. Conclusion

In this tutorial, we explored how to connect and make use of Spring Data Elasticsearch. We discussed how we can query, update and delete documents. Moreover, we also discussed how to create custom queries and they don't fit what's offered by Spring Data Elasticsearch.

As usual, the source code used throughout this tutorial can be found over on GitHub.

Generic bottom

I just announced the new Learn Spring course, focused on the fundamentals of Spring 5 and Spring Boot 2:

>> CHECK OUT THE COURSE
16 Comments
Oldest
Newest Most Voted
Inline Feedbacks
View all comments
nop
nop
4 years ago

Is it compatible with the current stable Elasticsearch version? => No

Eugen Paraschiv
4 years ago
Reply to  nop

That’s interesting – any link to support that claim? Cheers,
Eugen.

Kanagat
Kanagat
3 years ago

hello guys, i have question:
if my index have dynamic names like “index-2016.05.12”, “index-2016.04.12”, in this case:
is it possible use in @Document annotation, two or more indexNames?

@Document(indexName = “blog”, type = “article”)
public class Article {

Eugen Paraschiv
3 years ago
Reply to  Kanagat

Yes, it’s supported. Have a look at the JIRA.
Cheers,
Eugen.

David Morley
David Morley
4 years ago
Reply to  nop

It is not compatible with the latest version of Elasticsearch. It’s on their JIRA board for a future release, however. See: https://jira.spring.io/browse/DATAES-211

Jemli Fathi
Jemli Fathi
4 years ago

Hi, thank you for this tutorial, actually, i’m learning spring data elasticsearch and i have a very urgent use case on, first, how to make the searchquery covers all the document fields and next, about ES Fuzzy queries, especially on how to set the fuzziness level to be a maximum level to match a very large number of possible documents.

Do you have any idea about these two topics?

Thank you very much.

Eugen Paraschiv
4 years ago
Reply to  Jemli Fathi

Hey Jemli – these are interesting topics to add to the content calendar of the site, and I can certainly explore them in future articles, but if you need “urgent” help, I’d recommend the official channel for Spring Data Elasticsearch question – which is StackOverflow. And once you have your question there, feel free to email me the link to it and I’d be happy to have a look. Cheers,
Eugen.

Jemli Fathi
Jemli Fathi
4 years ago

Thank you very much sir, I appreciate it.

Cobra
3 years ago

Eugen, do you know if it’s possible make spring-data-elasticsearch @Document with compound key?
See, http://stackoverflow.com/questions/34469968/its-possible-make-spring-data-elasticsearch-document-with-compound-key

Thanks!

Eugen Paraschiv
3 years ago
Reply to  Cobra

That’s an interesting question. I remember a while back it wasn’t possible, but that was almost two years ago – so I’m not sure if that changed or not.

gaurav
gaurav
3 years ago

Hi,
Can you give any reallife scenario where data is kept in RDBMS or NoSQL and searching needs to be done using elasticsearch for faster retrival.

Could you demonstrate us .

Eugen Paraschiv
3 years ago
Reply to  gaurav

Hey Gaurav,
That’s an interesting topic suggestion, I’ll add it to the Content Calendar.
Cheers,
Eugen.

Zoltan Altfatter
Zoltan Altfatter
3 years ago

I have a badly designed document structure, like: { “_index”: “items”, “_type”: “item”, “_id”: “CD5D8F6516A88805FA826C10777B1750D9AAF5DA9CDD8E264757AB7EEC22B1EB”, “_score”: 1, “_source”: { “title”: “Textverständnis 5”, “active”: true, “successorId”: null, “metadata”: { “Fach”: “DE”, “Kompetenz”: “Les”, “code”: “C_SX_DE_Les_A0016_00149_V00”, … } } } I would like to retrieve the the title, Fach, and code from the above document. @Document(indexName = “items”, type = “item”) @Data public class Item { @Id private String id; private String title; private Metadata metadata; @Data static class Metadata { private String Fach; private String code; } } Retrieving the title, code are ok, but the “Fach” field returns null. Do… Read more »

Eugen Paraschiv
3 years ago

Hey Zoltan,
That’s a good question. Last time I checked, the @Field annotation didn’t have a good way to do this unfortunately – you’ll have to work around it.

That may mean doing query and then marshalling the response yourself with Jackson (at which point you have a lot more control over the process); or it may mean a number of other options. But, that’s what I’d look at first.

Hope that helps. Cheers,
Eugen.

Zoltan Altfatter
Zoltan Altfatter
3 years ago

Thanks Eugen!
In the meantime I figured it out, just using Jackson’s @JsonProperty.
http://stackoverflow.com/questions/41466561/spring-data-elasticsearch-field-mapping

Eugen Paraschiv
3 years ago

Yeah, Jackson makes it much easier. Cool, I’m glad you sorted it out.
Cheers,
Eugen.

Comments are closed on this article!