Baeldung Pro – Linux – NPI EA (cat = Baeldung on Linux)
announcement - icon

Learn through the super-clean Baeldung Pro experience:

>> Membership and Baeldung Pro.

No ads, dark-mode and 6 months free of IntelliJ Idea Ultimate to start with.

Partner – Orkes – NPI EA (tag=Kubernetes)
announcement - icon

Modern software architecture is often broken. Slow delivery leads to missed opportunities, innovation is stalled due to architectural complexities, and engineering resources are exceedingly expensive.

Orkes is the leading workflow orchestration platform built to enable teams to transform the way they develop, connect, and deploy applications, microservices, AI agents, and more.

With Orkes Conductor managed through Orkes Cloud, developers can focus on building mission critical applications without worrying about infrastructure maintenance to meet goals and, simply put, taking new products live faster and reducing total cost of ownership.

Try a 14-Day Free Trial of Orkes Conductor today.

1. Overview

Systemd is the backbone of most modern Linux distributions like Debian and Ubuntu. It’s the primary init system and service manager, launching essential services, managing dependencies, and maintaining process control from the moment the kernel hands over control. Accidentally deleting Systemd can result in a non-bootable system, rendering the operating system unusable. Fortunately, we can often restore this functionality.

In this tutorial, we’ll walk through the process of recovering a system after the unintentional removal of Systemd. Specifically, we’ll demonstrate the steps using a virtual machine (VM), but the process works the same on an actual system. In general, we discuss how to simulate the issue, recover with a live ISO, access the system using chroot, and finally reinstall Systemd.

2. Impact of Deleting or Losing Systemd

Systemd is fundamental to how modern Linux systems boot and operate. It serves as the init system – the first process that runs when the kernel hands over control during startup. Further, Systemd is responsible for launching all other services.

Deleting or corrupting Systemd disrupts this entire chain of events. Without Systemd, the system cannot locate the init binary (e.g., /sbin/init), which leads to boot failures, emergency shells, or kernel panic errors.

This issue doesn’t stop at booting; essential background services like NetworkManager, journald (used for logging), user sessions (logind), and even timed tasks (timers) depend on Systemd. Their failure means the system may lack Internet connectivity and service management. Additionally, it interferes with logging, hence the system lacks meaningful logs to trace what went wrong.

Package management becomes nearly impossible since many modern packages assume the presence of Systemd as a dependency. Moreover, graphical desktop environments such as GNOME or KDE also rely on Systemd to manage user sessions and startup processes.

As a result, an accidental deletion of Systemd cripples the operating system, leaving us with a machine that cannot start properly or run most of its core services.

3. Accidental Deletion of Systemd

In a live environment, Systemd may be deleted due to a broken package installation, a forced package removal, or an accidental user action. For this tutorial, let’s simulate a system without Systemd by intentionally deleting Systemd.

Notably, none of the following commands should run in a live or production system.

To begin with, we can either switch to the root user to get full control over system operations, including critical package management, or just use sudo before any of the commands:

$ sudo -i

Following, let’s use the native package manager to remove Systemd and all associated configuration files:

$ sudo apt remove --purge systemd

However, due to dependencies, this may not succeed on all systems. So, we can skip dependency checks and forcibly remove the package on a lower level:

$ sudo dpkg --remove --force-depends systemd

If the above commands fail, we should be able to simulate the failure by deleting the init binary itself:

$ sudo rm /sbin/init

Since the init binary is a symbolic link to Systemd on Debian-based systems. Removing it ensures the kernel has no entry point to start the user space, effectively breaking the boot process.

Once Systemd is removed, the system should fail to boot. It may drop into a shell or fail silently.

So, let’s reboot the system and see its behaviour:

$ sudo reboot

Upon booting, the system may start but freeze shortly afterwards:

4. Recovery via Live ISO and chroot

To recover the broken system, we can use a live ISO environment either directly for a VM or through a USB drive for a physical machine.

4.1. Create and Access Live Environment

Let’s see the steps for the latter case:

  1. Create a live USB by burning an ISO image to a USB stick using tools like Rufus or BalenaEtcher.
  2. Boot from the newly created USB medium.

This gives us access to a live environment where we can inspect and repair the broken system.

By booting into a live system, either from an ISO in VirtualBox or a live USB on physical hardware, we won’t be installing a new operating system. In this case, we temporarily run Linux from external media, so we can access and repair the installed system without modifying or overwriting it.

Both the live ISO and the bootable USB should behave the same. When we start the machine from the USB medium, we should boot into the live environment:

Booting into a live system

In this case, we use an Ubuntu ISO.

4.2. Locate Main Partition

Once inside the live environment, let’s open a terminal to check the main storage information and find the partition where the broken system is installed:

$ sudo fdisk -l | grep 'sda*'

This should give us all the available partitions for /dev/sda:

Storage information filtered through grep

Alternatively, we can skip grep, run fdisk -l, and look for lines with /dev/sda* or similar with a Linux filesystem:

Storage information

Either way, we should make sure we find the correct partition. In this case, we assume /dev/sda1 is the main Linux partition of the problematic installation.

4.3. Mount Broken Environment Filesystem

Next, let’s link the broken system with the live system, so we can access and operate on the damaged filesystem as if it were booted normally:

$ mkdir /mnt/sysdres
$ sudo mount /dev/sda1 /mnt/sysdres
$ sudo mount --bind /dev /mnt/sysdres/dev
$ sudo mount --bind /proc /mnt/sysdres/proc
$ sudo mount --bind /sys /mnt/sysdres/sys

First, we create a subdirectory in the live environment /mnt directory. Next, we mount the root partition (/dev/sda1) inside it, giving us access to the system files.

Following, we bind-mount essential virtual filesystems:

  • /dev provides us with access to device files, such as disks and terminals
  • /proc exposes the running process information
  • /sys reveals hardware and kernel-related data

These mounts provide the bare essentials for a non-booted system to interact with hardware and processes just like a normal running system.

Now, let’s use chroot to enter the broken system and operate it with full administrative privileges:

$ sudo chroot /mnt/sysdres

At this point, the prompt should change to #.

5. Reinstalling and Configuring Systemd

Now that we can access the broken system, let’s restore the critical components.

First, we refresh the package list:

# apt update

Now, let’s install Systemd:

# apt install systemd

Next, we fix missing or broken dependencies:

# apt --fix-broken install

Let’s ensure the system uses Systemd as its default init:

# ln -sf /lib/systemd/systemd /sbin/init

The command above creates a symlink that points to /lib/systemd/systemd to ensure Systemd is the first process after boot. Here, /sbin/init is the default location for the init process in most Linux distributions.

Additionally, let’s update the initramfs to ensure the system has the correct boot images:

# update-initramfs -u

Finally, let’s exit the live environment, reboot, and remove the USB drive:

# exit
$ sudo umount -R /mnt
$ sudo reboot

At this point, the system should boot into the fixed stable environment.

6. Conclusion

In this article, we looked at how to restore an accidentally deleted Systemd. Specifically, we mount the broken system on a live system, then utilize chroot to gain direct access with admin privileges to the broken system. In this situation, we perform a full reinstall of the init system, resolve broken dependencies, and rebuild essential components.

In conclusion, restoring Systemd after being accidentally deleted might seem intimidating, but we can recover full functionality using a live ISO from a USB drive.