Making a Python Module Available System-Wide on Linux

1. Introduction

We can make a Python module available system-wide on a Linux machine in several ways. The main option would be to install the module while running pip or the system’s package manager with superuser privileges. But even that option can be done in different ways.

In this tutorial, we discuss how to make a Python module available system-wide on Linux.

2. Using pip

pip is the preferred package manager for Python modules. It follows a relatively straightforward syntax that allows the management of modules, including installation, uninstallation, and upgrades.

2.1. Installing a Python Module from PyPI for All Users

PyPI stands for Python Package Index. It is the public repository from which pip installs Python packages.

To install a module with pip so it’s available to all users, all we have to do is run pip install using superuser privileges:

$ sudo pip install python-dotenv

Alternatively, we can adjust the command by adding python -m or python3 -m as the case may be:

$ sudo python3 -m pip install python-dotenv

In the instances above, we’re able install the python-dotenv module directly from pip because it’s available on PyPI.

If a module isn’t on PyPI, we can still install it from the source code using pip. Of course, we’ll have to do things a bit differently.

2.2. Installing a Python Source Code System-Wide Using setup.py

Before we can install a module from source code, we need a pyproject.toml or a setup.py file. Either of these files would be in the project’s root directory — the same directory as the package or, in the case of a single file, the module.

setup.py is a module the setuptools package uses to build, install, and distribute a package. pyproject.toml plays a similar role but is more descriptive. In other words, while setup.py is used to build a package, pyproject.toml describes the package we want to build to our package installer.

pyproject.toml is newer than setup.py and is the preferred option for newer projects. But, we’ll demonstrate the installation with both options.

To show how to install a module with system-wide access using a setup.py file, we’ll create a simple Python script, hello_baeldung.py. Let’s cat into the script:

$ cat hello_baeldung.py
#! /usr/bin/env python3

print("Hello Baeldung")

This will be our module.

We’ll also create a minimal setup.py:

$ cat setup.py
from setuptools import setup

setup(
    name="hello",
    version="1.0.0",
    description="A module that prints Hello Baeldung",
    py_modules=["hello_baeldung"],
)

In this setup.py, the name, version, and description arguments represent the package’s metadata. Then, the py_modules argument specifies the module. If we were building a package and not just a module, we’d use package_dir instead of py_module.

Now that we have our setup.py, let’s have a quick look at our project directory tree:

$ tree .
.
├── hello_baeldung.py
└── setup.py

0 directories, 2 files

Finally, to install the module system-wide, we’ll run sudo pip install:

$ sudo pip install .
Processing /home/adeyomola/test
  Preparing metadata (setup.py) ... done
Building wheels for collected packages: hello
  Building wheel for hello (setup.py) ... done
...truncated...
Installing collected packages: hello
Successfully installed hello-1.0.0
...truncated...

Running sudo python3 setup.py install will do the same thing as the command above.

With the package installed, we can execute our module from anywhere:

$ python3 -m hello_baeldung
Hello Baeldung

2.3. Installing the Module With pyproject.toml

We can repeat the same installation with pyproject.toml:

$ cat pyproject.toml
[build-system]
requires = ["flit_core<4"]
build-backend = "flit_core.buildapi"

[project]
name = "hello_baeldung"
description = "A module that prints Hello Baeldung"
version = "1.0.0"

In the setup.py file, the name argument represented the name we wanted for our package. But in the pyproject.toml, name is the filename of the module or directory name of the package we are trying to build. It’s also the name the package will take upon installation.

Let’s check our project directory with tree:

$ tree .
.
├── hello_baeldung.py
└── pyproject.toml

0 directories, 2 files

Now, let’s install with pip:

$ sudo pip install .
Processing /home/adeyomola/test
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Building wheels for collected packages: hello_baeldung
  Building wheel for hello_baeldung (pyproject.toml) ... done
...truncated...
Successfully built hello_baeldung
Installing collected packages: hello_baeldung
Successfully installed hello_baeldung-1.0.0
...truncated...

Finally, we run the installed module:

$ python3 -m hello_baeldung
Hello Baeldung

2.4. Installing a Python Module From Its wheel File

Instead of having to copy and build the whole source code whenever we want to deploy the package on a different system, we could build a wheel file. With only the wheel file, we can seamlessly install the module on different systems and for different users.

So, how do we build a wheel file from our source code? First, we must have a pyproject.toml or setup.py file. Once any of them is available, we’ll install the build package:

$ pip install build

Next, we’ll create the wheel file by running the build command with the –wheel argument in the project directory:

$ python3 -m build --wheel
* Creating venv isolated environment...
* Installing packages in isolated environment... (flit_core<4)
* Getting build dependencies for wheel...
* Building wheel...
Successfully built hello_baeldung-1.0.0-py2.py3-none-any.whl

Once that’s done, we’ll have our wheel file, hello_baeldung-1.0.0-py2.py3-none-any.whl, in the dist/ directory. Running ls with dist/ as our argument shows us the file:

$ ls dist/
hello_baeldung-1.0.0-py2.py3-none-any.whl

Then, we can install the module system-wide from the wheel file using pip install:

$ sudo pip install dist/hello_baeldung-1.0.0-py2.py3-none-any.whl
Processing ./dist/hello_baeldung-1.0.0-py2.py3-none-any.whl
Installing collected packages: hello-baeldung
Successfully installed hello-baeldung-1.0.0
...truncated...

Let’s test the module:

$ python3 -m hello_baeldung
Hello Baeldung

3. Using a Linux Package Manager

We can also install Python modules using our Linux package manager, as long as the said module is in the package manager’s repositories. If we’re unsure of the package’s availability, we could search before trying to install it.

For instance, if we’re trying to install python-dotenv on an Ubuntu or Debian system, we’ll search with apt search:

$ apt search python-dotenv
...trunacted...
python3-dotenv/oldstable 0.15.0-1 all
  Get and set values in the .env file in local and production servers

Now that we know the package is available, we can install it for all users using sudo apt install:

$ sudo apt install python3-dotenv -y
...truncated...
Unpacking python3-dotenv (0.15.0-1) ...
Setting up python3-dotenv (0.15.0-1) ...
Processing triggers for man-db (2.9.4-2) ...

Of course, we can try similar steps on our Fedora system with dnf:

$ dnf search python3-dotenv
...truncated...
========================================= Name Exactly Matched: python3-dotenv =========================================
python3-dotenv.noarch : Add .env support to your Django/Flask apps in development and deployments
======================================== Name & Summary Matched: python3-dotenv ========================================
python3-dotenv+cli.noarch : Metapackage for python3-dotenv: cli extras

Now, let’s install the package:

$ sudo dnf install python3-dotenv -y
...truncated...
Installed:
  python3-click-7.1.2-5.fc34.noarch    python3-dotenv-0.15.0-2.fc34.noarch    python3-dotenv+cli-0.15.0-2.fc34.noarch

Complete!

4. Using a Virtual Environment

Whenever we run pip with sudo, we get a warning:

$ sudo pip install .
...truncated...
Successfully installed hello_baeldung-1.0.0
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv

The warning highlights the potential for unintended behavior when running pip with sudo. It also advises that we use a virtual environment instead of running pip with superuser privileges.

With a virtual environment, our module will be available system-wide as long as its preconfigured virtual environment is active.

Let’s create a virtual environment in our project directory:

$ python3 -m venv venv

When the command is done, we’ll have a subdirectory named venv in our project directory:

$ ls
hello_baeldung.py  pyproject.toml  venv

Now, let’s activate the virtual environment:

$ source venv/bin/activate
(venv) $

With the virtual environment active, we can install the module using any of the methods previously described:

(venv) $ pip install .
...truncated...
Successfully installed hello-baeldung-1.0.0

Now, wherever we activate this virtual environment, we can use our module:

(venv) $ python -m hello_baeldung
Hello Baeldung

5. Conclusion

In this article, we discussed the system-wide installation of Python modules on Linux. We talked about installing packages with pip from PyPI and files like setup.py, pyproject.toml, and wheel. Then, we went over Python package installation with Linux package managers.

System-wide installation with pip or a Linux package manager requires superuser privileges, but it is better to configure a virtual environment for the module to avoid unexpected behavior.