How to Properly Manage Multiple Python Versions

1. Overview

In this article, we’ll see how we should manage different versions of Python and Python packages. We’ll discuss virtual environments: They are the best way to safely handle multiple versions of Python in a Linux device.

2. Overall Default Python Version in Linux

In general, manually changing the default version of Python in Linux is a procedure that may end up in broken dependencies. This is a bad idea because we can break packages within the distribution. There are only a few cases where we truly need to change the default Python version.

Our Linux distribution may or may not come with a default Python interpreter. If it does, when we change the default version of Python in Linux, we’re changing the interpreter. This can make other packages stop working, especially the ones written for the default Python version that might not have support in the replaced version. Linux systems rely on Python for applications that go from the storm built-in in Ubuntu to the RPM package manager yum in Debian/Red Hat. If we change the default Python version of Linux, we might break these applications.

In case we’ve already changed the default Python version of our Linux environment, we might need to revert our changes. To fix the problem, we just need to reinstall Python, and the package manager should take care of redoing all the symbolic links that we broke.

Nonetheless, we can install multiple versions of Python without replacing the default one (usually located in /usr/bin/python). Before installing a new version, it’s worth checking the default one:

$ python --version 
Python 3.8.2

We can install multiple Python versions to have /usr/bin/python3.7 and /usr/bin/python3.8. We should avoid modifying /usr/bin/python and just allow Linux to handle the default version of the interpreter.

3. Tools for Managing Different Versions Using Virtual Environments

We shouldn’t change the default Python version of Linux, but we might still need a different Python version than the default one. The right way to install another Python version is by using virtual environments. This way, we avoid overwriting or causing trouble with Linux packages.

Virtual environments are tools that developers use to isolate Python versions and Python packages for specific projects, without messing around with other Python versions, such as that of the Linux system. However, there are many different tools to manage these virtual environments. We can’t discuss all of them, but we should be aware of the variety that the Python ecosystem offers. Let’s go through the most relevant ones before discussing which one is more suited for a given project.

Before diving more deeply into how to manage different Python or package versions, we need to introduce pip, which is considered the package manager for Python. With pip, we can choose to install packages globally or for a given user. pip also handles dependencies and uninstalls packages. It’s likely that the package manager of our Linux distribution can also install Python packages. However, especially if we also use virtual environments, we should get used to pip as it makes installing packages rather easy:

$ pip install <package>

Python programs may depend on third-party packages whose versions might differ between programs. Thus, we need to keep track not only of Python versions but also of the versions of Python packages themselves. Virtual environments help us manage different Python versions and different package versions.

3.1. Virtual Environments With Tools From Outside the Standard Library

We’ll briefly present pyenv, virtualenv and pipenv. Although there are others such as pyvenv, pyenv-virtualenv, and poetry that we can use for the same goals, they are less frequent, are losing official support, and there is less documentation about them. In fact, from Python 3.6, the use of pyvenv is deprecated.

pyenv is the reference tool used to manage multiple isolated Python versions, as pyenv supports both Python 2 and Python 3. We can have several virtual environments with different Python versions. Based on the virtual environment we’ve activated, pyenv will intercept all our calls to commands such as python, pip, and more, and redirect them to the appropriate tool (found in PATH) and from there to the right binary file. pyenv also simplifies downloading, installing, and managing several Python versions, making their installation as simple as:

$ pyenv install 3.9

virtualenv creates Python environments for Python packages. Instead of using the default directory provided by Linux, virtualenv installs these packages in a custom /bin directory. In that folder, virtualenv places a binary of Python (which we can also customize and doesn’t need to be the default one in Linux). We can easily create an environment named env_name:

$ virtualenv env_name

Then, we need to activate it before using commands such as pip to install packages with specific versions:

$ source env_name/bin/activate
$ pip install --force-reinstall <package>==1.2.3

pipenv combines pip and virtualenv with one extra feature: the pipfile. This file contains all the information about a virtual environment. When we install new packages, the pipfile is updated automatically. When sharing the pipfile, another person can quickly replicate the exact same environment:

$ pipenv install

We can also install packages at a given version:

$ pipenv install <package>~=1.2.3

However, the performance of pipenv is worse than that of virtualenv.

3.2. Virtual Environments With Tools From the Standard Library

Since Python 3.3, venv is the default tool that Python ships for handling multiple Python versions. Even if it should be the preferred approach, other tools (such as pyenv) are still popular since they also support Python 2. Nevertheless, since the end-of-life for Python 2, venv should gain more traction.

To use venv, we need to first install the Python binary with the package manager of our distribution. Then, we can create our new virtual environment based on this binary:

$ python3.9 -m venv ~/.venvs/my-venv-name

Thus, if we want to have different Python versions with venv, we need different binaries installed in our system with our package manager.

3.3. Which Tools Should We Use?

Even if part of the spirit of Python is to keep things simple, it looks like there are already too many tools for handling virtual environments. However, if we use Python 3, the best solution is to use venv. After Python 3.3, we should definitely use venv over virtualenv, since it’s shipped with Python directly. These installed packages in venv won’t be visible from the outside of the environment. This way, we won’t mess up the rest of the system. We also don’t need sudo rights to perform these operations.

If we really need Python 2 in our project, the best solution is pyenv, since it is similar to venv. One drawback of pyenv is that we cannot easily get installed packages back to their previous state. Furthermore, we need sudo rights to perform certain actions.

4. Conclusion

In this article, we’ve covered Python version management in our Linux system. We discussed why we shouldn’t change the default Python version of Linux because doing so can lead to problems. Moreover, we presented virtual environments as a solution to manage multiple versions of Python and different versions of Python packages living in the same system. There are many tools we can use to manage Python virtual environments, but if we only use the newer Python 3, we should stick to venv.