Authors Top

If you have a few years of experience in the Linux ecosystem, and you’re interested in sharing that experience with the community, have a look at our Contribution Guidelines.

1. Introduction

There isn’t always a way to clean up zombie processes. Yet, almost any process can become a zombie.

In this tutorial, we explore what happens when it’s a child process of the one that can’t – the initialization process (init). First, we look at how init can end up as the parent of a zombie. Next, we delve into the mechanics of initialized zombies. Finally, we discuss ways to manually resolve zombies that are direct children of the initialization process.

We tested the code in this tutorial on Debian 11 (Bullseye) with GNU Bash 5.1.4. It should work in most POSIX-compliant environments.

2. Initialized Zombies

Whether it’s systemd, System V (SysV), or another option, Linux needs a mechanism to start itself. As the more modern alternative, let’s look at how systemd can spawn a process that goes on to transition to a zombie (Z) state.

For brevity, despite their differences, we use the classic SysV name init alongside systemd interchangeably to describe the initialization process.

2.1. init as Parent Process

Regardless of the run level, we can have a basic set of kernel modules and packages which run at the start of the operating system (OS). Such basic programs are usually leveraged by processes, which spawn as a fork of PID 1:

$ pstree
systemd─┬─blkmapd
        ├─cron
        ├─dbus-daemon
        ├─login───bash
        ├─nmbd
        ├─rpcbind
        ├─rsyslogd───3*[{rsyslogd}]
        ├─smbd─┬─cleanupd
        │      ├─lpqd
        │      └─smbd-notifyd
        ├─sshd───sshd───sshd───bash───su───bash───su───bash───+
        ├─2*[systemd───(sd-pam)]
        ├─systemd-journal
        ├─systemd-logind
        ├─systemd-timesyn───{systemd-timesyn}
        └─systemd-udevd

Here, we use pstree to see many processes directly stemming from the systemd initialization process:

Of course, any of the above is a candidate for becoming an initialized zombie, i.e., a zombie that has the initialization process as its parent.

2.2. Process to Zombie

Processes turn into zombies by being left by their parents even after their work as a child is done.

In particular, to release completed children and avoid zombies, the parent executes the wait() system call, usually when handling SIGCHLD.

When the parent doesn’t acknowledge their completion, children remain in the zombie (Z, <defunct>) state.

3. Handling Initialization Zombies

Usually, we have two main options to handle zombie processes:

  • send SIGCHLD to the parent
  • kill the parent

In fact, we might be able to work around cases when even kill -9 doesn’t appear to work for the second case.

However, the situation is complicated when the parent is the initialization process, which:

  • ignores unexpected SIGCHLD signals
  • can’t be killed without rebooting

Still, one of the functions of systemd and init is to poll the execution status of its children and wait() for them. Since the initialization process usually becomes the new parent of orphan processes, they often get resolved instead of becoming zombies.

Yet, sometimes the mechanism fails and results in a <defunct> zombie process as a direct child of, e.g., systemd, init, or another such root parent.

4. Manually Resolving Initialization Zombies

When init is the parent of a zombie, we don’t have much choice. There are several ways to attempt and resolve the situation:

  • ignore the process
  • restart init
  • reboot

Ignoring zombies might be acceptable when there is a limited number of such processes, and they don’t take up resources or spawn constantly. However, in some cases, despite their definition, zombies can continue holding a resource, preventing a new process from using it.

Also, depending on the initialization process, we might be able to restart it with different tools:

  • telinit for SysV provides telinit u for init restarts
  • systemctl for systemd provides systemctl daemon-reexec for systemd restarts

Despite not requiring a reboot, such commands may still have unwanted consequences, and whether their benefit outweighs any negative side effects depends on the situation.

Finally, as a last resort, a clean reboot or shutdown plus a new start should solve the problem.

5. Summary

In this article, we talked about our options when an initialization process is the parent of a zombie.

In conclusion, we have ways of handling initialized zombies, but they are limited, and the root cause of the issue should be of primary concern.

Authors Bottom

If you have a few years of experience in the Linux ecosystem, and you’re interested in sharing that experience with the community, have a look at our Contribution Guidelines.

Comments are closed on this article!