Copy Variables Between Stages in a Multi Stage Docker Build

1. Overview

The Multi-stage builds feature in Docker allows developers to create a sequence of builds, each with its own set of instructions. This helps in optimizing the size of the resulting image. One of the challenges in multi-stage builds is copying variables between stages.

Copying variables between stages is important because it allows us to reuse values in the subsequent stages of the build process.

In this tutorial, we’ll explore three different methods for copying variables across stages in a multi-stage Docker build: using ARG, using ENV, and using a file. We’ll learn how each method works, along with an example to illustrate its usage.

2. Using ARG

The easiest way to copy variables between stages in a multi-stage Docker build is using ARG. We can use ARG variables during the build process and can pass them to the docker build command as build arguments.

Let’s now look into a Dockerfile to copy variables in a multi-stage build:

# Stage 1
FROM alpine AS stage1
ARG MY_VAR
RUN echo MY_VAR value is $MY_VAR

# Stage 2
FROM alpine AS stage2
ARG MY_VAR
RUN echo MY_VAR is still set to: $MY_VAR

In this example, we define a variable called MY_VAR using ARG in both stages. We then use the RUN command to print the value of MY_VAR in each stage.

To build the image and pass in the value of MY_VAR as a build argument, run the following command:

$ docker build --build-arg MY_VAR=somevalue -t my_image .

This will set the value of MY_VAR to “somevalue” during the build process:

$ docker build --build-arg MY_VAR=somevalue -t my_image .
Sending build context to Docker daemon  2.048kB
Step 1/6 : FROM alpine AS stage1
 ---> b2aa39c304c2
Step 2/6 : ARG MY_VAR
 ---> Running in 38b3ef3093c3
Removing intermediate container 38b3ef3093c3
 ---> 5619deec98e2
Step 3/6 : RUN echo MY_VAR value is $MY_VAR
 ---> Running in 1bf3db328de6
MY_VAR value is somevalue
Removing intermediate container 1bf3db328de6
 ---> 5e9f9bc1255a
Step 4/6 : FROM alpine AS stage2
 ---> b2aa39c304c2
Step 5/6 : ARG MY_VAR
 ---> Running in 19d18d18f055
Removing intermediate container 19d18d18f055
 ---> 842a58a44c8e
Step 6/6 : RUN echo MY_VAR is still set to: $MY_VAR
 ---> Running in 88ba4ea6188b
MY_VAR is still set to: somevalue
Removing intermediate container 88ba4ea6188b
 ---> 6b912736db48
Successfully built 6b912736db48
Successfully tagged my_image:latest

In the output above, we can clearly see that the value of ARG MY_VAR is preserved in both stages.

3. Using ENV

Another way to copy variables between stages in a multi-stage Docker build is by using ENV. We can use ENV to set variables that are available during the build process and also persist in the resulting image.

Let’s create a simple Dockerfile for illustration:

FROM alpine:latest as base
ENV MY_ENV="my_env"

FROM base
RUN echo ${MY_ENV}

Here, we created an environment variable, MY_ENV, in the first stage of the Dockerfile. This variable is now accessible to all the subsequent stages that are created using the first stage.

Let’s now build the Docker image using the following command:

$  docker build --no-cache  -t my_image .
Sending build context to Docker daemon   5.12kB
Step 1/4 : FROM alpine:latest as base
 ---> b2aa39c304c2
Step 2/4 : ENV MY_ENV="my_env"
 ---> Running in 7ffd89e261a2
Removing intermediate container 7ffd89e261a2
 ---> 17e87c9864c3
Step 3/4 : FROM base
 ---> 17e87c9864c3
Step 4/4 : RUN echo ${MY_ENV}
 ---> Running in 337470d2041d
my_env
Removing intermediate container 337470d2041d
 ---> 8e6abc31f316
Successfully built 8e6abc31f316
Successfully tagged my_image:latest

In the output above, the echo command in Step 4 prints the MY_ENV value from the first stage of the Dockerfile.

The main difference between using the ARG and ENV methods to copy variables between stages in a multi-stage Docker build is in how the variables are stored and accessed. ARG variables are passed in at build time and are only available during the build process, while ENV variables are set in the Dockerfile and can be accessed by subsequent stages.

4. Using File

The ARG approach is useful when we need to pass a variable’s value from one stage to another during the build process. It’s also useful when we want to make the value of the variable available during the build process, but not in the final image. This is because ARG variables are only available during the build process and are not persisted in the final image.

Another way to copy variables between stages in a multi-stage Docker build is to use a file. This method involves writing the variable value to a file in one stage and then reading the file in another stage.

Here’s a simple Dockerfile for illustration:

# Stage 1 - Build
FROM alpine AS build

# Write the variable value to a file
RUN echo "hello" > /tmp/myvar

# Stage 2 - Run
FROM alpine AS run

# Copy the variable value from the previous stage using a file
COPY --from=build /tmp/myvar /tmp/myvar
# Read the variable value from the file
RUN MY_VAR=$(cat /tmp/myvar) && echo "Running the app with MY_VAR=$MY_VAR"

In this example, we have two stages: “build” and “run”. The “build” stage writes the value “hello” to a file called /tmp/myvar. In the “run” stage, we copy the file from the previous stage and read its contents using the cat command. We then set the MY_VAR environment variable to the value of the file and print it out using echo.

$ docker build --no-cache -t my_image .
Sending build context to Docker daemon  7.168kB
Step 1/6 : FROM alpine AS build
 ---> b2aa39c304c2
Step 2/6 : RUN echo "hello" > /tmp/myvar
 ---> Running in db7cbe1a060f
Removing intermediate container db7cbe1a060f
 ---> 3c64d471fe3a
Step 3/6 : RUN echo "Building the app..."
 ---> Running in 08c4d8024c53
Building the app...
Removing intermediate container 08c4d8024c53
 ---> 4cc6f20768e5
Step 4/6 : FROM alpine AS run
 ---> b2aa39c304c2
Step 5/6 : COPY --from=build /tmp/myvar /tmp/myvar
 ---> 3dc259424cb6
Step 6/6 : RUN MY_VAR=$(cat /tmp/myvar) && echo "Running the app with MY_VAR=$MY_VAR"
 ---> Running in 961f02320a71
Running the app with MY_VAR=hello
Removing intermediate container 961f02320a71
 ---> 1a8c5349f42b
Successfully built 1a8c5349f42b
Successfully tagged my_image:latest

In the output above, we can see that the MY_VAR value (step 6) is accessible in the second stage. We can also verify the file content inside the docker container:

$ docker run --rm my_image /bin/sh -c "cat /tmp/myvar"
hello

This approach is useful when we need to make the variable available in the final image. We can write the variable’s value to a file in one stage and then copy it to another stage.

Furthermore, the file approach is useful when we want to copy too many variables across different stages. Adding all the variables to a file and sharing it across all the stages becomes easier.

5. Conclusion

In this article, we learned about three different ways to copy variables between stages in a multi-stage Docker build: using ARG, ENV, and a file.

By understanding the differences between these approaches, we can choose the most appropriate method.