In our projects, we often use docker-compose to deploy our containerized applications. With CI and CD, code changes and deployments are very frequent nowadays. Hence it's essential to ensure that docker-compose always uses the latest images of the applications.
In this tutorial, we'll check out several options to achieve the same.
2. Pulling Images Explicitly
Let's take a simple example of a docker-compose file:
version: '2.4' services: db: image: postgres my_app: image: "eugen/test-app:latest" ports: - "8080:8080"
Here, we've used two services – one is the PostgreSQL database and another is a test application. We use the following command to deploy our applications:
$ docker-compose up -d
This pulls both the images from a remote repository (let's say, dockerhub) and creates the corresponding containers.
The problem happens when we redeploy our application with the same command. As the images already exist in the local repository, it doesn't check if there are any changes for these images in the remote repository. So, our option is to pull all the images in the docker-compose file beforehand:
$ docker-compose pull Pulling db ... done Pulling my_app ... done
This command first checks if there are any updates available for both the images in dockerhub. Then, it downloads only the necessary layers to keep the image up-to-date with the remote.
However, we may not always prefer to pull the image of Postgres. If we use a stable version of the database, there will be no changes in the image. So, it doesn't make sense to download that for every deployment. Sometimes, the existing data files are not compatible with the new version. This can break the deployment. In that case, we may use only the names of specific services in the pull command:
$ docker-compose pull my_app Pulling my_app ... done
Now, if we execute the up command, this recreates the containers with the latest image for sure:
$ docker-compose up -d Starting docker-test_db_1 ... done Starting docker-test_my_app_1 ... done
We can, of course, combine both the commands to create a single-liner:
$ docker-compose pull && docker-compose up -d
3. Removing Local Images
Let's consider the same example again. We already know that the main problem is the existence of the local images. So, the second option we have here is to stop all the containers and remove their images from the local repository:
$ docker-compose down --rmi all Stopping docker-test_my_app_1 ... done Removing docker-test_db_1 ... done Removing docker-test_my_app_1 ... done Removing network docker-test_default Removing image postgres Removing image eugen/test-app:latest
The down command stops and removes the containers. The –rmi option removes the images from the local repo. The rmi type local removes only those images which have been built locally. So, it’s better to use the rmi type all to ensure all the images used by the current configuration get removed.
Now, we start our container again with the up command:
$ docker-compose up -d Creating network "docker-test_default" with the default driver Pulling db (postgres:)... latest: Pulling from library/postgres 7d63c13d9b9b: Pull complete cad0f9d5f5fe: Pull complete ... Digest: sha256:eb83331cc518946d8ee1b52e6d9e97d0cdef6195b7bf25323004f2968e91a825 Status: Downloaded newer image for postgres:latest Pulling my_app (eugen/test-app:latest)... latest: Pulling from eugen/test-app df5590a8898b: Already exists 705bb4cb554e: Already exists ... Digest: sha256:31c05c8245192b32b8b359fc58b5e45d8397674ccf41f5f17a7d3109772ab5c1 Status: Downloaded newer image for eugen/test-app:latest Creating docker-test_db_1 ... done Creating docker-test_my_app_1 ... done
We can see that this doesn’t find the image in the local repository anymore. So, it pulls the latest image from the remote repository to recreate the containers.
The downside with this approach is that you cannot choose specific images to remove. So, it always removes the postgres image and downloads the same image again which is not necessary. To resolve this, we can use docker rmi to remove the specific image:
$ docker rmi -f eugen/test-app:latest Untagged: eugen/test-app:latest Untagged: eugen/[email protected]:31c05c8245192b32b8b359fc58b5e45d8397674ccf41f5f17a7d3109772ab5c1 Deleted: sha256:7bc07b4eb1c23f7a91afeb7133f107e0a8318fb77655d7d5f2f395a035a13eb7
4. Rebuilding the Images
Let's see the same example of the docker-compose file with a different flavor:
version: '2.4' services: db: image: postgres my_app: build: ./test-app ports: - "8080:8080"
Here, we've used the same db service with Postgres. But, for the service my_app, we've given a build section instead of using a ready-made image. This section contains the build context of the test-app. The docker file which resides in the test-app directory is like this:
FROM openjdk:11 COPY target/test-app-0.0.1-SNAPSHOT.jar app.jar ENTRYPOINT ["java","-jar","/app.jar"]
In this scenario, when we redeploy using the up command, docker-compose again reuses the local image if it exists. So, we need to make sure that docker-compose rebuilds the image each time we trigger a deployment. We can do this using the option –build:
$ docker-compose up -d --build Building my_app [+] Building 2.5s (8/8) FINISHED => [internal] load build definition from Dockerfile 0.0s => => transferring dockerfile: 41B 0.0s => [internal] load .dockerignore 0.0s => => transferring context: 2B 0.0s => [internal] load metadata for docker.io/library/openjdk:11 2.3s => [auth] library/openjdk:pull token for registry-1.docker.io 0.0s => [1/2] FROM docker.io/library/openjdk:[email protected]:1b04b1958a4a61900feec994e3938a2a5d8f88db8ec9515f46a25cbe561b65d9 0.0s => [internal] load build context 0.0s => => transferring context: 84B 0.0s => CACHED [2/2] COPY target/test-app-0.0.1-SNAPSHOT.jar app.jar 0.0s => exporting to image 0.0s => => exporting layers 0.0s => => writing image sha256:4867a3f0b0a043cd54e16086e2d3c81dbf4c418806399f60fc7d7ffc094c7159 0.0s => => naming to docker.io/library/docker-test_my_app 0.0s Starting docker-test_db_1 ... Starting docker-test_db_1 ... done
The alternate approach is to use the build command before running the up command:
$ docker-compose build --pull --no-cache db uses an image, skipping Building my_app ... 0.0s => => naming to docker.io/library/docker-test_my_app
This command offers two more options. The –pull option asks to pull the base image of the Dockerfile from the remote repository while building the image. The –no-cache option denotes that the local cache is skipped. However, it skips building the db service because we have used the direct image there.
Now, if we restart our compose configuration, the containers will use the latest image:
$ docker-compose up -d Starting docker-test_db_1 ... done Recreating docker-test_my_app_1 ... done
In this article, we've seen why docker-compose may not use the latest image at the time of deployment. We've also learned several ways to make docker-compose use always the latest images to recreate the containers with that image. We also discussed the pitfalls of the approaches and how we can mitigate them.
The code related to the article is available over on GitHub.