List Running Programs That Were Started With nohup

1. Overview

In this tutorial, we’ll look at ways to get a list of programs that were started using the nohup command in Linux. We’ll take a closer look at some of the Bash utilities that can be used for this purpose.

2. The Problem Description

There are times when we might want to execute a program that is immune to session hangups or the SIGHUP signal. Executing a program using the nohup command ensures that the hangup signals are ignored and the process continues to run.

Let’s execute a process using the nohup command:

$ nohup sleep 50000
nohup: ignoring input and appending output to 'nohup.out'

Now, let’s start another login session and check the process tree using the pstree command:

$ pstree -p
init(1)─┬─init(7)───init(8)───bash(9)───pstree(1140)
        ├─init(1121)───init(1122)───bash(1123)───sleep(1138)
        ├─{init}(5)
        └─{init}(6)

As we can see, the sleep process with PID 1138 is running from the Bash shell with PID 1123. Let’s now terminate the shell with PID 1123 to hang up the session and verify the processes again:

$ kill -9 1123
$ pstree -p
init(1)─┬─init(7)───init(8)───bash(9)───pstree(1150)
        ├─init(1122)───sleep(1138)
        ├─{init}(5)
        └─{init}(6)

As expected, the sleep process continues its run even after the session terminates.

In the next sections, we’ll look at ways to filter the processes that were started using the nohup command.

3. Using the lsof Command

If the STDOUT of the process is executed using the nohup command is terminal, then the output is logged in the nohup.out file. We can use this information to filter the processes using the lsof command:

$ lsof nohup.out
COMMAND  PID  USER   FD   TYPE DEVICE SIZE/OFF NODE NAME
sleep   1138 shubh    1w   REG   8,16        0 3096 nohup.out
sleep   1138 shubh    2w   REG   8,16        0 3096 nohup.out

Here, the lsof command lists the PID and name of the processes that are writing to the nohup.out file. As expected, our sleep process with a PID of 1138 appears in the output.

It’s important to note that this technique works only for processes whose STDOUT is redirected to the terminal. If a command was started with nohup but its STDOUT is written to some other file, then the above solution will not capture that command. Let’s check this with an example:

$ nohup sleep 500 > output.log &
[1] 43
$ nohup: ignoring input and redirecting stderr to stdout

$ lsof nohup.out
lsof: status error on nohup.out: No such file or directory

As we can observe, the lsof command gave an error since the output of the nohup command is being redirected to output.log, and not to nohup.out.

4. Using the jobs Command

We can also check whether any process is running with nohup from the same ssh session using the jobs command:

$ sleep 7000 &
[1] 104
$ nohup sleep 500000 &
[2] 105
$ nohup: ignoring input and appending output to 'nohup.out'

$ jobs -l | grep nohup
[2]+ 105 Running nohup sleep 500000 &

Note that this method works only on the same ssh session and will not show the nohup process if we check the same from a different session or a Bash shell:

$ jobs -l | grep nohup
[2]+   105 Running                 nohup sleep 500000 &
$ bash
$ jobs -l
$ exit
exit
$ jobs -l | grep nohup
[2]+   105 Running                 nohup sleep 500000 &

5. Conclusion

In this article, we learned how to get a list of programs that were started using the nohup command. We learned that there isn’t any fail-safe way to determine the list. However, there are certain scenario-specific Bash utilities that can help us.