How to Implement Elasticsearch Health Check in Docker Compose

1. Overview

When working in Docker Compose, ensuring services start reliably and in the right order is crucial. For instance, this is true when we’re using a stack that includes Elasticsearch and other dependent services such as Kibana, Logstash, and Metricbeat. Without a proper health check, Docker may consider the Elasticsearch container unhealthy even if it’s functioning correctly, which can cause other services to fail during startup.

In this tutorial, we’ll explore why Elasticsearch health checks often fail in Docker Compose environments and how we can implement a reliable and practical solution.

2. Defining Health Check in Docker Compose

In Docker Compose, a health check enables us to define a way for Docker to check whether a container is healthy. To clarify, the health check command executes at intervals and returns a status:

• Healthy – indicates that the container works properly
• Unhealthy – indicates that Docker detected a problem based on the defined health check command

So, we can utilize this status to ensure that dependent services only start after the target service becomes healthy.

3. Elasticsearch Health Checks

Elasticsearch plays a key role in many monitoring, logging, and search platforms. For example, tools such as Kibana, Logstash, Metricbeat, and Filebeat depend on it being ready.

3.1. Elasticsearch vs. Normal Health Checks

Typically, health checks verify whether a service is listening on a port. For example, we may ping a basic HTTP service using:

test: ["CMD", "curl", "-f", "http://localhost"]

The above instruction works for simple web servers but not for Elasticsearch. This is because Elasticsearch responds to HTTP requests before it’s fully initialized, and during this time, the health endpoint can return a non-green (e.g., yellow) status even though the service is functioning correctly in development mode.

Without a proper health check, Docker Compose may continue marking Elasticsearch as unhealthy, stopping all dependent services from launching. As a result, this may lead to health checks passing too early or failing without indicating an actual problem.

3.2. How Elasticsearch Health Checks Are Different

With Elasticsearch, we need to verify internal readiness — not just whether the process is running.

Additionally, the health check needs to verify whether the cluster status is yellow or green and not just whether the container is alive. A yellow status means Elasticsearch has all the main data available, even though backup copies (replicas) aren’t assigned yet. This is normal and acceptable in single-node setups, where replicas aren’t required.

To manually check the cluster status, we can use the following command:

$ curl http://localhost:9200/_cluster/health?pretty
{
  "cluster_name" : "docker-cluster",
  "status" : "green",
  "timed_out" : false,
  "number_of_nodes" : 1,
  ...
}

In health checks, we’re interested in ensuring that the cluster status is at least yellow, indicating that Elasticsearch is initialized and ready for queries.

Furthermore, we need to parse JSON output or grep for cluster state.

4. Setting Up the Demo Project

To demonstrate, we’ll create a Docker Compose project. With this project, we aim to show a working health check that ensures services only start when Elasticsearch is truly ready.

Conveniently, Elasticsearch provides a special endpoint called _cluster/health that returns the current status of the cluster. Now, when we pass the parameter wait_for_status to this endpoint, we can block the request until the cluster reaches a minimum status of yellow. With the help of the wait_for_status parameter, we can wait for Elasticsearch to become fully operational before starting dependent services.

Let’s use the tree command to display our project’s structure:

$ tree elasticsearch-healthcheck-project
elasticsearch-healthcheck-project
└── docker-compose.yml

0 directories, 1 file

Now, in our docker-compose.yml file, let’s add the following content:

version: '3.7'

services:
  elasticsearch:
    image: elasticsearch:7.12.1
    container_name: elasticsearch
    healthcheck:
      test: ["CMD-SHELL", "curl -fs http://localhost:9200/_cluster/health?wait_for_status=yellow&timeout=5s || exit 1"]
      interval: 10s
      timeout: 5s
      retries: 5
      start_period: 20s
    environment:
      - discovery.type=single-node
      - xpack.security.enabled=false
      - ES_JAVA_OPTS=-Xms512m -Xmx512m
    ports:
      - 9200:9200
    networks:
      - elastic

  app:
    image: alpine
    container_name: dummy-app
    command: ["sh", "-c", "echo 'Elasticsearch is healthy and app is running'; sleep 3600"]
    depends_on:
      elasticsearch:
        condition: service_healthy
    networks:
      - elastic

networks:
  elastic:

The setup above ensures dummy-app only starts once Elasticsearch returns a healthy cluster status of at least yellow, indicating the service is initialized and functioning.

Now, let’s analyze the components of our docker-compose.yml file that contribute to a reliable Elasticsearch health check.

4.1. The elasticsearch Service

Here’s the image and container name we use:

elasticsearch:
  image: elasticsearch:7.12.1
  container_name: elasticsearch

So, we pull the official Elasticsearch image (version 7.12.1) and give the container the name elasticsearch.

4.2. Health Check Logic

Next, let’s discuss the health check logic:

healthcheck:
    test: ["CMD-SHELL", "curl -fs http://localhost:9200/_cluster/health?wait_for_status=yellow&timeout=5s || exit 1"]
    interval: 10s
    timeout: 5s
    retries: 5
    start_period: 20s

The curl command checks the cluster health endpoint and waits for the cluster to reach a yellow status.

Let’s analyze the health check options:

• interval – how often to run the check, in this case, every 10s
• timeout – how long to wait for a response, in this case, 5s
• retries – instructs Docker to mark the container as unhealthy after 5 failed checks
• start_period – wait 20 seconds before starting the first health check

Now, we can prevent services from starting while Elasticsearch is still booting.

4.3. Elasticsearch Environment Variables

We use environment variables to add extra configurations for Elasticsearch:

environment:
  - discovery.type=single-node
  - xpack.security.enabled=false
  - ES_JAVA_OPTS=-Xms512m -Xmx512m

In particular, we configure Elasticsearch to run in single-node mode, disable security, and assign manageable memory settings respectively.

4.4. Ports and Network

Next, let’s look at the ports and networks section:

ports:
  - 9200:9200
networks:
  - elastic

We expose port 9200 to allow HTTP access from the host system and include the container in the elastic network so that other services like our app service can resolve it via hostname.

4.5. Dependent app Service

The app service is a simple Alpine-based container that displays a success message and sleeps:

app:
  image: alpine
  container_name: dummy-app
  command: ["sh", "-c", "echo 'Elasticsearch is healthy and app is running'; sleep 3600"]
  depends_on:
    elasticsearch:
      condition: service_healthy
  networks:
    - elastic

So, we use this service to represent any real application that depends on Elasticsearch. Additionally, depends_on ensures that app waits until Elasticsearch reports a healthy status before launching.

5. Verifying the Setup

At this point, let’s start all services:

$ docker compose up -d
Creating network "elasticsearch-healthcheck-project_elastic" with the default driver
Pulling elasticsearch (elasticsearch:7.12.1)...
...
Creating elasticsearch ... done
Creating dummy-app     ... done

To verify the health status, we can use:

$ docker inspect --format='{{json .State.Health}}' elasticsearch | jq
{
  "Status": "healthy",
  ...
}

Alternatively, we can access the endpoint manually:

$ curl http://localhost:9200/_cluster/health?pretty
{
  "cluster_name" : "docker-cluster",
  "status" : "yellow",
}

Another option we can use is inspecting the logs of dummy-app:

$ docker logs dummy-app
Elasticsearch is healthy and app is running

The output confirms that our setup works as intended.

6. Conclusion

In this article, we’ve explored how to implement Elasticsearch health checks in Docker Compose.

When working in Docker Compose, health checks are crucial in multi-container setups involving Elasticsearch. Unlike standard services, Elasticsearch takes time to initialize and may seem unhealthy if we depend solely on basic port availability checks.

By using the _cluster/health endpoint with wait_for_status, we can delay the startup of dependent services until Elasticsearch is fully functional. This way, we can enhance predictability and reliability, reduce startup errors, and ensure services are ready for connection.