Docker Build and Run Platforms and Resolving exec /usr/bin/sh: exec format error

1. Introduction

As a breakthrough containerization tool based on Linux features, Docker supports different architectures for both its core and containers. These are sometimes interchangeable, but could depend on certain conditions like executable file formats, layer abstractions, configuration, and others.

In this tutorial, we address platform management when building or running a Docker container, as well as the execution format error that may occur in certain scenarios. First, we build and run a sample Docker image via a Dockerfile to see each step. After that, we understand how and when the target or execution platform can be specified during this process. Next, we go through the consequences of mismatched build and run platforms. Finally, we delve into other reasons for getting an exec format error.

We tested the code in this tutorial on Debian 12 (Bookworm) with GNU Bash 5.2.15. Unless otherwise specified, it should work in most POSIX-compliant environments.

2. Docker Image Build and Run

To create a Docker container, one usually first fetches or builds a Docker image. For that, a Dockerfile is commonly the way to go.

2.1. Sample Dockerfile

Let’s create a simple Dockerfile for demonstration purposes:

$ cat Dockerfile
FROM debian:bookworm-slim
RUN apt-get update
RUN apt-get install -y curl
RUN curl https://gerganov.com/ > response
CMD ["sh"]

This image blueprint is based on the slim version of the current latest Debian distribution. It downloads the most recent curl package and dumps an HTTP response to a file.

Notably, CMD sets sh as the shell to use, i.e., runs that process if no other process is specified.

2.2. Image Build

After creating a Dockerfile, we can initiate a build.

To do so, let’s issue the docker command with the build or buildx build subcommand:

$ docker buildx build -t samplex-app .

In this case, we name (or [-t]ag) the image as samplex-app. The . period indicates that the Dockerfile is in the current directory.

At this point, we should have an image available in the local registry.

2.3. Run Container

Once we have an image, let’s run it via the run subcommand of the docker command:

$ docker run -it samplex-app

Here, we create an [-i]nteractive [-t]erminal session for communication with the container after it initiates. Thus, we usually see a prompt:

#

Of course, this series of steps occurs during a successful workflow. Sometimes, the resulting image can be incompatible with the run environment.

3. Container Architecture Platform Specification

Since Linux exists on many architectures, containers can be built for and run on any of them.

In fact, we can both build and use containers meant for one architecture on others. For that, we need to configure the binfmt_misc kernel feature. It enables the recognition and execution of binaries for different architectures via emulators such as QEMU. This way, we might be able to run a PowerPC or ARM container within an x86 environment.

3.1. The –platform Parameter

Although it works in different settings, the –platform parameter always has the same syntax for its options:

--platform <OS>/<ARCHITECTURE>[/<VARIANT>]

Let’s see a breakdown of each part:

• OS: linux, windows, darwin
• ARCHITECTURE: amd64, arm64, arm, 386, and others
• VARIANT (optional): v3, v5, and similar

For instance, linux/amd64 is perhaps one of the most common desktop or server platforms. Something like linux/arm64 indicates a 64-bit ARM architecture such as that of the Raspberry PI v4 (also known as aarch64). Older ARM platforms are linux/arm/v6 and linux/arm/v7.

3.2. Build-Time Platform

The default target platform for builds is usually the native one of the host.

To ensure we can specify a given platform during the build process, we must use buildx, because the regular build subcommand doesn’t have that option:

$ docker buildx build --platform [...]

In addition, the buildx build subcommand respects the DOCKER_DEFAULT_PLATFORM environment variable for targeted builds when the –platform parameter is omitted:

$ DOCKER_DEFAULT_PLATFORM='linux/arm' docker buildx build -t samplex-app .

Either way, we ensure that the platform for the resulting image is set accordingly. Notably, with the proper setup, we can build for platforms different from that of the host via emulators like QEMU.

3.3. Runtime Platform

When using the run subcommand of docker, we can indicate the platform we want the image to be fetched for, in case there are several:

$ docker run --platform [...]

Similar to buildx build, the run subcommand considers DOCKER_DEFAULT_PLATFORM if –platform isn’t explicitly defined:

$ DOCKER_DEFAULT_PLATFORM='linux/arm' docker run -it samplex-app

While the parameter may look obsolete if there’s only one version or platform, it forces the docker run command to error out:

no matching manifest for linux/arm64 in the manifest list entries

This can be critical to ensure we don’t attempt to run an unsupported image. Of course, again, we can run container images for different platforms as long as the proper configuration (e.g., QEMU) is in place.

3.4. Dockerfile and FROM

Furthermore, we can indicate a target platform via the –platform option in FROM when writing a Dockerfile:

$ cat Dockerfile
FROM --platform= debian:bookworm-slim
[...]

However, this method doesn’t specify the build target platform, only the platform of the fetched image. This is similar to the run subcommand parameter.

Still, we can access the current value of the buildx flag –platform inside the Dockerfile via the TARGETPLATFORM variable.

3.5. Platform Support

What platforms are supported by a given Docker setup is determined by the specific installation. For instance, Docker Desktop already has support for arm64 and arm out of the box.

To get the support list for a given environment, we can use the buildx subcommand with inspect:

$ docker buildx inspect --bootstrap
[...]
Platforms: linux/amd64, linux/arm64, linux/arm/v7, linux/386, linux/ppc64le, linux/s390x, linux/riscv64

Since we see a number of supported platforms on top of the native one, there seems to be a QEMU configuration in place.

In addition, we can check a specific image via the imagetools subcommand:

$ docker buildx imagetools inspect <IMAGE_NAME>

Similar to the –bootstrap inspection, the output contains a Platform: field with a list. The docker image inspect command should produce a similar result, albeit in a different format.

4. Platform Mismatches

Perhaps one of the main reasons for an exec format error when running a container is that the build platform doesn’t match the run platform. Let’s see how and when this happens.

4.1. Basic Case

Of course, we might just insert an unexpected platform as the value of –platform:

$ docker run --platform=linux/ppc64le ppc64le/debian sh
[...]
Status: Downloaded newer image for ppc64le/debian:latest
exec /usr/bin/sh: exec format error

In this case, ppc64le isn’t compatible with the native platform, and there is no emulation set up to handle that architecture.

Yet, how do we know what platform Docker expects? As already mentioned, the native build and run platform is that of the host, but we can check the support via docker buildx inspect –bootstrap to see what other platforms are configured.

4.2. Pull the Wrong Image

When using FROM within a Dockerfile, we might also set the wrong platform, resulting in the same problem:

$ cat Dockerfile
FROM --platform=linux/ppc64le debian:bookworm-slim
CMD ["sh"]
$ docker buildx build -t bad-platform .
[...]
$ docker run bad-platform
WARNING: The requested image's platform (linux/ppc64le) does not match the detected host platform (linux/amd64/v3) and no specific platform was requested
exec /usr/bin/sh: exec format error

So, a Dockerfile can also be the source of the mismatch.

4.3. Hidden Runners

To complicate matters, some cloud and other container runner environments like AWS and GitLab offer options to pick the execution platform.

In these cases, we should manually ensure that the configured and built containers run in the respective environment. While we can create a Dockerfile that adapts, doing so isn’t trivial.

5. Non-Platform Reasons for exec format error

Critically, format errors can be caused by more than architecture mismatches. Let’s briefly take a look at other triggers for the same issue.

5.1. Bad Script Header

When dealing with containers, we often create scripts that should run within them.

In Linux, the first line of scripts can be a special header called a shebang:

$ cat script.sh
#!/bin/bash
[...]

The shebang contains the path to the correct interpreter of the respective script. It has very strict formatting rules:

• starts at the first character of the file
• is allocated the whole first line
• uses absolute paths
• contains no unescaped space characters in the path
• the comparison with !# is only for the first two (2) bytes

Any deviation from the above, such as a missing shebang, space after !#, other characters on the first line, unexpected whitespace, or even an incorrect encoding, can cause an exec format error:

$ cat Dockerfile
FROM debian:bookworm-slim

RUN echo '#! /bin/sh\necho "bad"' > /bad.sh
RUN chmod +x /bad.sh

CMD ["/bad.sh"]
$ docker buildx build -t badbang
[...]
$ docker run badbang
exec /bad.sh: exec format error

Yet, confusingly, depending on the setup, terminal, and configuration, there might not be a problem when running the script directly on the host.

5.2. Bad File Permissions

As usual on Linux and UNIX in general, file permissions dictate whether a file can run in a given context. Specifically, if a script isn’t marked as executable, attempting to run it directly results in an error:

$ cat Dockerfile
FROM debian:bookworm-slim

RUN echo '#! /bin/sh\necho "norun"' > /norun.sh

CMD ["/norun.sh"]
$ docker buildx build -t badbang
[...]
$ docker run badbang
docker: Error response from daemon: failed to create task for container: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: error during container init: exec: "/norun.sh": permission denied: unknown

Run 'docker run --help' for more information

While this output doesn’t usually include the exec format error string, user reports state that it does so in some environments.

6. Summary

In this article, we talked about Docker build and run platforms, along with errors that they might lead to. Further, we discussed alternative reasons for exec format error issues.

In conclusion, mismatches between the build and run platforms for a Docker container often lead to format errors, but that isn’t always the only reason to get them.