How to Properly Manage Multiple Python Versions

1. Overview

In this article, we’ll see how we should manage different versions of Python and Python packages. We’ll discuss virtual environments and how we can use them to safely have multiple versions of Python in a Linux device.

2. Overall Default Python Version in Linux

In general, manually changing the default version of Python in Linux is a critical procedure that usually ends up in broken systems. This is a bad idea because we can break packages or dependencies within the distribution. There are only a few cases where we truly need to change the default Python version.

When we change the default version of Python in Linux, we’re changing the interpreter, which is a component of the Operating System. This can make other components stop working, especially the ones written for the replaced Python version. Linux systems have Python components that go from the storm built-in on Ubuntu to the critical package manager yum in Debian/Red Hat. When we change the default Python version of Linux, we can break these components.

In case we’ve already changed the default Python version of Linux, we might need to revert our changes. To fix the problem, we just need to reinstall Python, and the package manager should take care of undoing all the symbolic links that we broke.

Nonetheless, we can always install multiple versions of Python without replacing the default one. Before installing a new version, it’s worth checking the default one:

$ python --version 
Python 3.8.2

We can install multiple Python versions to have /usr/bin/python2.7 and /usr/bin/python3.8. We should avoid modifying /usr/bin/python and just allow Linux to handle its default version.

3. Tools for Managing Different Versions Using Virtual Environments

Even if we shouldn’t change the default Python version of Linux, we might still need a different Python version than the default one. The right way to install another Python version is by using virtual environments. This way, we avoid overwriting or causing trouble with Linux components.

Virtual environments are tools that developers use to isolate Python versions (and Python packages) for specific projects, without messing around with the Python version of the Linux system. However, there are different tools that exist to manage these virtual environments, depending on what we want to isolate. Let’s go through them before discussing which is more suited for a given project.

3.1. Managing Different Python Versions

There are many tools that we can use to manage these Python virtual environments. We cannot go through all of them, but we should be aware of the variety that the whole Python ecosystem offers. The two most well-known tools for managing different Python versions in our system are pyenv and venv.

On the one hand, pyenv was the reference tool used to manage multiple isolated Python versions, as pyenv supports both Python 2 and Python 3. We can have several virtual environments linked with potentially different Python versions. pyenv will intercept all our calls to commands such as python, pip, and more, and redirect them to the appropriate tool (found in PATH) and from there to the right binary, based on the virtual environment we’ve activated. pyenv also simplifies downloading, installing, and managing several Python versions, making their installation as simple as:

$ pyenv install 3.9

On the other hand, since Python 3.3, venv is the default tool that Python ships for handling multiple Python versions. Even if it should be the preferred approach, other tools are more popular since they support Python 2. Nevertheless, since the end-of-life for Python 2, this tool should gain more traction. In fact, from Python 3.6, the use of pyenv is deprecated.

To use venv, we need to first install the Python binary with the package manager of our distribution. Then, we can create our new virtual environment based on this binary:

$ python3.9 -m venv ~/.venvs/my-venv-name

Thus, if we want to have different Python versions with venv, we need different binaries installed in our system with our package manager.

3.2. Managing Different Versions of Python Packages

Before deepening more into how to manage different versions, we need to introduce pip, which is considered the package manager for Python. We can choose to install packages globally or for a given user. pip also handles dependencies and uninstalling packages. It’s likely that the package manager of our Linux distribution can also install Python packages. However, we should get used to pip – especially if we also use virtual environments – as it makes installing packages rather easy:

$ pip install <package>

Python programs may depend on third-party packages whose versions might differ between programs. Thus, we need to keep track not only of Python versions but also of the versions of Python packages. To do so, there are again different tools. We’ll briefly present virtualenv and pipenv, but there are others as pyenv-virtualenv, pyvenv, and poetry that we can use for the same goals. However, they are less commonly used and are losing official support, and there is less documentation about them.

virtualenv creates Python environments for Python packages. Instead of using the default directory provided by Linux, virtualenv installs these packages in a custom /bin directory. In that folder, virtualenv places a binary of Python (which we can also customize and doesn’t need to be the default one in Linux). We can easily create an environment named env_name:

$ virtualenv env_name

Then, we need to activate it before using commands such as pip to install packages with specific versions:

$ source env_name/bin/activate
$ pip install --force-reinstall <package>==1.2.3

pipenv combines pip and virtualenv with one extra feature: the pipfile. This file contains all the information of a virtual environment, and if we install new packages, it’s updated automatically. Moreover, when sharing this pipfile, another person can quickly replicate the exact same environment:

$ pipenv install

We can also install packages at a given version:

$ pipenv install <package>~=1.2.3

However, the performance of pipenv is worse than that of virtualenv.

3.3. Which Tools Should We Use?

Even if part of the spirit of Python is to keep things simple, it looks like there are already too many tools for handling virtual environments. However, if we use Python 3, the best solution is to use venv.

After Python 3.3, we should use venv over virtualenv, since it’s shipped with Python directly. Moreover, after creating a virtual environment with venv, we can install any package that we want. These installed packages are not visible from the outside of the environment. This way, we won’t mess up the rest of the system. We also don’t need sudo rights to perform these operations.

pyenv is similar to venv, but we should only use it if we really need Python 2. One drawback of pyenv is that we cannot easily get installed packages back to a previous state. Furthermore, we need sudo rights to perform certain actions.

4. Conclusion

In this article, we’ve covered Python version management in our Linux system. We discussed how we shouldn’t mess with the default Python version of Linux because doing so can lead to problems. Moreover, we presented virtual environments as a solution to manage multiple versions of Python and different versions of Python packages living in the same system. There are many tools we can use to manage Python virtual environments, but if we only use the newer Python 3, we should stick to venv.