How to Get Docker-Compose to Always Use the Latest Image

1. Introduction

In our projects, we often use docker-compose to deploy our containerized applications. With CI and CD, code changes and deployments are very frequent nowadays. Hence it’s essential to ensure that docker-compose always uses the latest images of the applications.

In this tutorial, we’ll check out several options to achieve the same.

2. Pulling Images Explicitly

Let’s take a simple example of a docker-compose file:

version: '2.4'
services:
  db:
    image: postgres
  my_app:
    image: "eugen/test-app:latest"
    ports:
      - "8080:8080"

Here, we’ve used two services – one is the PostgreSQL database and another is a test application. We use the following command to deploy our applications:

$ docker-compose up -d

This pulls both the images from a remote repository (let’s say, dockerhub) and creates the corresponding containers.

The problem happens when we redeploy our application with the same command. As the images already exist in the local repository, it doesn’t check if there are any changes for these images in the remote repository. So, our option is to pull all the images in the docker-compose file beforehand:

$ docker-compose pull
Pulling db     ... done
Pulling my_app ... done

This command first checks if there are any updates available for both the images in dockerhub. Then, it downloads only the necessary layers to keep the image up-to-date with the remote.

However, we may not always prefer to pull the image of Postgres. If we use a stable version of the database, there will be no changes in the image. So, it doesn’t make sense to download that for every deployment. Sometimes, the existing data files are not compatible with the new version. This can break the deployment. In that case, we may use only the names of specific services in the pull command:

$ docker-compose pull my_app
Pulling my_app ... done

Now, if we execute the up command, this recreates the containers with the latest image for sure:

$ docker-compose up -d
Starting docker-test_db_1     ... done
Starting docker-test_my_app_1 ... done

We can, of course, combine both the commands to create a single-liner:

$ docker-compose pull && docker-compose up -d

3. Removing Local Images

Let’s consider the same example again. We already know that the main problem is the existence of the local images. So, the second option we have here is to stop all the containers and remove their images from the local repository:

$ docker-compose down --rmi all
Stopping docker-test_my_app_1 ... done
Removing docker-test_db_1     ... done
Removing docker-test_my_app_1 ... done
Removing network docker-test_default
Removing image postgres
Removing image eugen/test-app:latest

The down command stops and removes the containers. The –rmi option removes the images from the local repo. The rmi type local removes only those images which have been built locally. So, it’s better to use the rmi type all to ensure all the images used by the current configuration get removed.

Now, we start our container again with the up command:

$ docker-compose up -d
Creating network "docker-test_default" with the default driver
Pulling db (postgres:)...
latest: Pulling from library/postgres
7d63c13d9b9b: Pull complete
cad0f9d5f5fe: Pull complete
...
Digest: sha256:eb83331cc518946d8ee1b52e6d9e97d0cdef6195b7bf25323004f2968e91a825
Status: Downloaded newer image for postgres:latest
Pulling my_app (eugen/test-app:latest)...
latest: Pulling from eugen/test-app
df5590a8898b: Already exists
705bb4cb554e: Already exists
...
Digest: sha256:31c05c8245192b32b8b359fc58b5e45d8397674ccf41f5f17a7d3109772ab5c1
Status: Downloaded newer image for eugen/test-app:latest
Creating docker-test_db_1     ... done
Creating docker-test_my_app_1 ... done

We can see that this doesn’t find the image in the local repository anymore. So, it pulls the latest image from the remote repository to recreate the containers.

The downside with this approach is that you cannot choose specific images to remove. So, it always removes the postgres image and downloads the same image again which is not necessary. To resolve this, we can use docker rmi to remove the specific image:

$ docker rmi -f eugen/test-app:latest
Untagged: eugen/test-app:latest
Untagged: eugen/test-app@sha256:31c05c8245192b32b8b359fc58b5e45d8397674ccf41f5f17a7d3109772ab5c1
Deleted: sha256:7bc07b4eb1c23f7a91afeb7133f107e0a8318fb77655d7d5f2f395a035a13eb7

4. Rebuilding the Images

Let’s see the same example of the docker-compose file with a different flavor:

version: '2.4'
services:
  db:
    image: postgres
  my_app:
    build: ./test-app
    ports:
      - "8080:8080"

Here, we’ve used the same db service with Postgres. But, for the service my_app, we’ve given a build section instead of using a ready-made image. This section contains the build context of the test-app. The docker file which resides in the test-app directory is like this:

FROM openjdk:11
COPY target/test-app-0.0.1-SNAPSHOT.jar app.jar
ENTRYPOINT ["java","-jar","/app.jar"]

In this scenario, when we redeploy using the up command, docker-compose again reuses the local image if it exists. So, we need to make sure that docker-compose rebuilds the image each time we trigger a deployment. We can do this using the option –build:

$ docker-compose up -d --build
Building my_app
[+] Building 2.5s (8/8) FINISHED                                                                                            
 => [internal] load build definition from Dockerfile                                                                    0.0s
 => => transferring dockerfile: 41B                                                                                     0.0s
 => [internal] load .dockerignore                                                                                       0.0s
 => => transferring context: 2B                                                                                         0.0s
 => [internal] load metadata for docker.io/library/openjdk:11                                                           2.3s
 => [auth] library/openjdk:pull token for registry-1.docker.io                                                          0.0s
 => [1/2] FROM docker.io/library/openjdk:11@sha256:1b04b1958a4a61900feec994e3938a2a5d8f88db8ec9515f46a25cbe561b65d9     0.0s
 => [internal] load build context                                                                                       0.0s
 => => transferring context: 84B                                                                                        0.0s
 => CACHED [2/2] COPY target/test-app-0.0.1-SNAPSHOT.jar app.jar                                                        0.0s
 => exporting to image                                                                                                  0.0s
 => => exporting layers                                                                                                 0.0s
 => => writing image sha256:4867a3f0b0a043cd54e16086e2d3c81dbf4c418806399f60fc7d7ffc094c7159                            0.0s
 => => naming to docker.io/library/docker-test_my_app                                                                   0.0s

Starting docker-test_db_1 ... 
Starting docker-test_db_1 ... done

The alternate approach is to use the build command before running the up command:

$ docker-compose build --pull --no-cache
db uses an image, skipping
Building my_app
...                                                                                                           0.0s
 => => naming to docker.io/library/docker-test_my_app 

This command offers two more options. The –pull option asks to pull the base image of the Dockerfile from the remote repository while building the image. The –no-cache option denotes that the local cache is skipped. However, it skips building the db service because we have used the direct image there.

Now, if we restart our compose configuration, the containers will use the latest image:

$ docker-compose up -d
Starting docker-test_db_1       ... done
Recreating docker-test_my_app_1 ... done

5. Conclusion

In this article, we’ve seen why docker-compose may not use the latest image at the time of deployment. We’ve also learned several ways to make docker-compose use always the latest images to recreate the containers with that image. We also discussed the pitfalls of the approaches and how we can mitigate them.

The code related to the article is available over on GitHub.