Dockerizing the Python Poetry Project

1. Overview

Dependency management is one of the crucial aspects of software engineering. In Python, there have been several efforts to improve how we manage our projects and dependencies. Poetry is one such tool that aims to provide an easier way to manage our projects without relying on additional tools.

In this article, we’ll learn about the Poetry dependency manager. We’ll learn the basics of how to install and use it. Then, we’ll cover how to use Poetry inside a Docker container to coordinate our project.

2. The Poetry Dependency Manager

Poetry is a tool intended to make Python dependency management easier. It doesn’t rely on the traditional requirements.txt, setup.cfg, or Pipfile.

It only works with a single TOML file, pyproject.toml. We put all the requirements and project configuration inside this file. Then, we use the poetry command to build, test, and run the project.

In a way, it’s very similar to Rust’s Cargo.

2.1. Basic Usage

We can install Poetry from the official website using cURL:

$ curl -sSL https://install.python-poetry.org | python3 -

Then, we initialize a new project:

$ poetry new news-aggregator

Now, we can add dependencies for the project:

$ poetry add scrapy feedreader

Let’s view the pyproject.toml file now and see what it looks like:

$ cat pyproject.toml 
[tool.poetry]
name = "news-aggregator"
version = "0.1.0"
description = ""
authors = ["Your Name <[email protected]>"]
readme = "README.md"

[tool.poetry.dependencies]
python = "^3.12"
scrapy = "^2.11.2"
feedreader = "^0.3.1"

[build-system]
requires = ["poetry-core"]
build-backend = "poetry.core.masonry.api"

If we pull a Python repository that uses Poetry, we should install the dependencies and sync them to make sure they’re updated as per the lock file:

$ poetry install && poetry sync

This installs the packages and adds an entry for them in the pyproject.toml file automatically. Similarly, we can also remove packages:

$ poetry remove feedreader && poetry sync

We now have a consistent directory layout, where we can add our project files:

$ tree .
.
├── Dockerfile
├── main.py
├── news_aggregator
│   └── __init__.py
├── poetry.lock
├── pyproject.toml
├── README.md
└── tests
    └── __init__.py

Then, we activate the virtual environment that was created in the first step:

$ poetry shell

Finally, we build the project:

$ poetry build

Notably, it’s as easy as falling off a log compared to using Pip and Virtualenv. However, as simple as it is, it becomes a bit more complicated if we try to use Poetry in a Docker container, which we’ll cover next.

3. Using Poetry with Docker

In this section, we’ll dockerize our project and use Poetry as the dependency manager.

3.1. First Steps

We’ll create a Dockerfile that will bootstrap the Docker container. First, we’ll create the image from the latest stable version of Python:

FROM python:3.12.6-slim-bookworm

Then, we’ll specify the required project environment variables for configuration:

# Python
ENV PYTHONFAULTHANDLER=1 \
    PYTHONUNBUFFERED=1 \
    PYTHONHASHSEED=random \
    PIP_NO_CACHE_DIR=off \
    PIP_DISABLE_PIP_VERSION_CHECK=on \
    PIP_DEFAULT_TIMEOUT=100

# Poetry
ENV POETRY_NO_INTERACTION=1 \
    POETRY_VIRTUALENVS_CREATE=false \
    POETRY_CACHE_DIR='/var/cache/pypoetry' \
    POETRY_HOME='/usr/local' \
    POETRY_VERSION=1.8.3

# Additional Env Vars
ENV YOUR_ENV=${YOUR_ENV}

Mind that we disabled the virtual environment because we are using Docker, which already provides an isolated production environment. In the last statement, we can also include our custom environment variables.

3.2. Setting Up Poetry and Caching Dependencies

Now, we’ll install Poetry from the official source. However, we’ll need to install cURL before:

RUN apt-get update && apt-get install -y curl && rm -rf /var/lib/apt/lists/*

Then, we install Poetry:

RUN curl -sSL https://install.python-poetry.org | python3 -

Of course, we can combine the two commands to optimize the final image. However, for clarity, we’ll keep it this way.

Next, specify the directory for the project:

WORKDIR /app

Let’s also copy our pyproject.toml and poetry.lock:

COPY pyproject.toml poetry.lock /app/

We copy these files before adding any other files. Thus, it will optimize the re-installation of packages when we change the pyproject.toml. Therefore, Docker will reuse the cached layer if we change the code.

3.3. Installing Dependencies

We are now ready to install the dependencies. For convenience, we’ll use the same Dockerfile for development and production. We’ll make use of an environment variable to decide this for us:

$ poetry install --no-ansi $(test "$ENVIRONMENT" == production && echo "--only=main")

The –only-main option will only install the production dependencies. Specifically, it will install the dependencies given in the [tool.poetry.dependencies] section in pyproject.toml. Similarly, we add development dependencies to the [tool.poetry.dev-dependencies] section.

Finally, we also provided the –no-ansi switch to enable pretty logs.

3.4. Copying the Project

We copy the project files to /app:

COPY . /app

If we want to exclude files and directories, add them to .dockerignore.

3.5. Running the Project

In the final step, we add the command that runs our project:

CMD ["poetry", "run", "python", "main.py"]

We can add additional options and change the starting point as needed.

Here is our final Dockerfile:

FROM python:3.12.6-slim-bookworm

# Python
ENV PYTHONFAULTHANDLER=1 \
    PYTHONUNBUFFERED=1 \
    PYTHONHASHSEED=random \
    PIP_NO_CACHE_DIR=off \
    PIP_DISABLE_PIP_VERSION_CHECK=on \
    PIP_DEFAULT_TIMEOUT=100

# Poetry
ENV POETRY_NO_INTERACTION=1 \
    POETRY_VIRTUALENVS_CREATE=false \
    POETRY_CACHE_DIR='/var/cache/pypoetry' \
    POETRY_HOME='/usr/local' \
    POETRY_VERSION=1.8.3

ENV YOUR_ENV=${YOUR_ENV}

RUN apt-get update && apt-get install -y curl && rm -rf /var/lib/apt/lists/*
RUN curl -sSL https://install.python-poetry.org | python3 -
WORKDIR /app
COPY pyproject.toml poetry.lock /app/

COPY . /app
CMD ["poetry", "run", "python", "main.py"]

3.6. Running the Container

We are now ready to build the Docker image:

$ docker build -t news-aggregator .

Finally, let’s run it:

$ sudo docker run -d --name instance-1 -e ENVIRONMENT=production news-aggregator

4. Conclusion

In this article, we discussed what the Poetry tool is and how to use it. Then, covered a hands-on approach to using Poetry inside a Docker container. We wrote a simple yet effective Dockerfile to manage our Python projects reliably.