1. Overview

When we work with the Linux command line, we usually run Linux commands in the foreground. However, in some situations, we need to run multiple commands in the background.

In this tutorial, we’ll see how to run multiple commands as background jobs using two approaches:

  • Running multiple commands as a single background job
  • Running multiple commands in multiple background jobs

We’ll focus on the Bash shell in this tutorial.

2. Running Multiple Commands as a Single Job

Usually, we want to run multiple commands as a single job when the commands are related to each other.

We can start multiple commands as a single job through three steps:

  1. Combining the commands – We can use ;, &&, or || to concatenate our commands, depending on the requirement of conditional logic, for example: cmd1; cmd2 && cmd3 || cmd4
  2. Grouping the commands – We can group the combined commands by “{ }” or “( ), for example: ( cmd1; cmd2 && cmd3 || cmd4 )
  3. Sending the command group to the background as a job – If we add the & operator after our command group, the shell will run the command group as a background job, for example: ( cmd1; cmd2 && cmd3 || cmd4 ) &

Next, let’s see an example of how to execute multiple commands as a single background job.

Let’s say we want to do an odd-even check on a number from an expensive calculation, and output the result. Since the expensive calculation may take some time, we want to run the commands as a background job. So we can apply the three steps above to build the command:

$ ( echo "Calculating the number..."; sleep 8; \
    NUM=$RANDOM; [ $((NUM%2)) -eq 0 ] && echo "$NUM: even" || echo "$NUM: odd" ) &

In the command above, we use the sleep command to simulate the number calculation process. Now, let’s see how it is running as a job through a little demo:

As the demo shows, we started the command three times. Every time we launched the command, the command group was running as a background job. We can monitor the status of the jobs using the jobs command.

3. Controlling the Output

We’ve learned how to launch multiple commands as a single background job. If we review the demo, we’ll find all outputs from the jobs are printed to our terminal. This is because the background process started by the & operator will inherit the stdout and stderr from the shell.

In practice, we often want to redirect the output of the job to a file so that the job outputs do not mess up the current terminal. Moreover, we can check the result of the job or the log of the job execution efficiently. To achieve that, we can redirect the output of the command group to a file:

$ ( echo "Calculating the number..."; sleep 8; \
    NUM=$RANDOM; [ $((NUM%2)) -eq 0 ] && echo "$NUM: even" || echo "$NUM: odd" ) > result.txt &
[1] 40030
$ 
[1]+  Done                    ( echo "Calculating the number..."; sleep 8; ... ) > result.txt
$ cat result.txt 
Calculating the number...
30027: odd

Thus, the output of the job goes to the file result.txt. We only see the PID of the job and the done notification in the current terminal.

The PID of the job is essential information. Usually, we won’t want to suppress it. However, we can hide it if we don’t want to see any output after launching the command:

$ { ( echo "Calculating the number..."; sleep 8; NUM=$RANDOM; \
    [ $((NUM%2)) -eq 0 ] && echo "$NUM: even" || echo "$NUM: odd" ) > result.txt & } 2>/dev/null
$
$ jobs
[1]+  Running                 ( echo "Calculating the number..."; sleep 8; ... ) > result.txt &
$
[1]+  Done                    ( echo "Calculating the number..."; sleep 8; ... ) > result.txt
$ cat result.txt 
Calculating the number...
7667: odd

As the output above shows, we redirected the stderr to /dev/null to suppress the output of the “&” operator because it writes the PID information to the stderr.

If we take a closer look at the output above, we may see the done notification of the job is still printed. The done notification of a job is a feature of the shell. It cannot be controlled by IO redirection.

If we want to hide the done notification too, we have to start the job in a sub-shell by changing the “{ }” group into “( ). However, the side effect is that we lose control of the job. Therefore, this is not recommended:

$ ( ( echo "Calculating the number..."; sleep 8; NUM=$RANDOM; \
     [ $((NUM%2)) -eq 0 ] && echo "$NUM: even" || echo "$NUM: odd" ) > result.txt & )
$ jobs

As the command above shows, we don’t need to redirect the stderr if we wrap the job in a “( )” group. This is because the job is not started in the current shell, and it doesn’t use the stderr of the current shell either.

Further, we can see the jobs command outputs nothing. That is to say, the current shell doesn’t know about this job. Therefore, we can’t monitor or control the job. We don’t know when is the proper time to check our result.txt either since we don’t know if the job has been finished.

4. Running Multiple Commands as Multiple Jobs

We can also run multiple commands as different jobs to let them run in parallel. 

To achieve that, we add the “&” operator to the command or command group we want to send to the background, for example:

cmd1 & cmd2 & (cmd3; cmd4) &

In the example above, we’ll start three jobs and they run in parallel:

  • Job1: cmd1
  • Job2: cmd2
  • Job3: (cmd3; cmd4)

A concrete example can help us to understand it quickly:

$ date & (sleep 5; echo "cmd2 done") & (sleep 3; echo "cmd3 done") & 
[1] 41012
[2] 41013
[3] 41014
Sun Sep 13 01:31:57 PM CEST 2020
$ jobs
[1]   Done                    date
[2]-  Running                 ( sleep 5; echo "cmd2 done" ) &
[3]+  Running                 ( sleep 3; echo "cmd3 done" ) &
$ cmd3 done
cmd2 done
[2]-  Done                    ( sleep 5; echo "cmd2 done" )
[3]+  Done                    ( sleep 3; echo "cmd3 done" )

As the output above shows, we start three background jobs and they run in parallel.

5. Conclusion

In this article, we’ve demonstrated how to execute multiple commands as a single background job. Further, we’ve discussed how to control the output of the jobs.

Finally, we’ve addressed the technique to launch multiple commands as multiple background jobs and let them run in parallel.

guest
0 Comments
Inline Feedbacks
View all comments