What’s the Meaning of $! in Bash Scripting

1. Overview

When we read shell scripts, sometimes we see $! in the code.

In this tutorial, we’ll take a closer look at it, understand what $! is and how to use it when we write scripts.

2. Short Introduction to the $! Variable

The $! variable is a special shell variable that stores the PID of the most recently executed background process. A background process, also known as a background job, allows us to continue using the command line interface for other tasks.

There are different ways to start a background job in Linux. For simplicity, we’ll use the & operator to run commands in the background.

Next, we’ll see the $! variable in action through some examples.

3. The $! Variable in Action

First, let’s see a simple script:

$ cat flight-booking-api.sh
#!/bin/bash
echo "Sending flight-booking data to the remote service...."
#simulate the remote service call...
sleep 20
echo "Sending flight-booking data done at $(date) !"

The flight-booking-api.sh script sends the flight-booking data to a remote service. Of course, we won’t really talk to some flight services. Instead, we use the sleep command to make the process sleep for 20 seconds to simulate the data-sending procedure.

Next, let’s run the script as a background job using the & operator:

$ ./flight-booking-api.sh > /tmp/test/flight.log &
[1] 4148642

As we can see from the output above, the shell reports the job is started with the JobID [1] and PID 4148642.

Now, if we run the jobs command, we’ll see our script is running:

$ jobs
[1]  + running    ./flight-booking-api.sh > /tmp/test/flight.log

Also, our job’s PID is stored in the $! variable:

$ echo $!
4148642

4. The $! Variable Only Stores the Latest Job’s PID

We’ve seen the background job’s PID is stored in the $! variable. If we start multiple jobs in the same shell, the $! variable only holds the most recently started job’s PID.

Next, let’s understand it quickly with an example. Let’s say we add a new script, hotel-booking-api.sh:

$ cat hotel-booking-api.sh
#!/bin/bash
echo "Sending hotel-booking data to the remote service...."
#simulate the remote service call...
sleep 10
echo "Sending hotel-booking data done at $(date) !"

This time, the script sleeps for 10 seconds to simulate notification of the remote hotel-booking service.

Now, let’s start flight-booking-api.sh and hotel-booking-api.sh in turn, and check the $! variable’s change:

$ ./flight-booking-api.sh > /tmp/test/flight.log &
[1] 4187529

$ echo $!
4187529

$ ./hotel-booking-api.sh >/tmp/test/hotel.log &
[2] 4187668

$ echo $!
4187668

As the output above shows, the $! only stores the latest job’s PID, whether previous jobs are finished or not.

It’s worth mentioning that $! stores the latest background job’s PID instead of the last command’s PID. Some articles on the internet recommend reading $! to get the PID of the last command. This is incorrect. An example can show it quickly:

$ ./flight-booking-api.sh > /tmp/test/flight.log &
[1] 6131

$ echo $!
6131

$ echo "a new command executed"
a new command executed

$ echo $!
6131

As we can see in the example above, executing a command and making it run in the foreground won’t change $!‘s value.

5. The $! Variable’s Value Doesn’t Depend on the Status of the Job Process

We’ve learned that $! holds the PID of the most recently started job. Now, let’s check what the $! variable’s value will change if the job is finished.

Again, we’ll use our flight-booking-api.sh script as an example. First, let’s start the script as a background job:

$ ./flight-booking-api.sh > /tmp/test/flight.log &
[1] 4189529

$ echo $!
4189529

Then, let’s wait for 20 seconds. When the job is done, the shell will notify us:

[1]  + 4189529 done       ./flight-booking-api.sh > /tmp/test/flight.log

Now, if we take a look at the flight.log file, we should see the “Job Done” log entry too:

$ cat /tmp/test/flight.log 
Sending flight-booking data to the remote service....
Sending flight-booking data done! at Thu Apr  6 08:55:33 PM CEST 2023

As the job finished, process 4189529 ended. In other words, process 4189529 doesn’t exist anymore. We can check the /proc/<PID> directory to verify if the process still lives:

$ ls /proc/4189529
ls: cannot access '/proc/4189529': No such file or directory

Now, if we check $!‘s value, it still holds 4189529. Therefore, even if the corresponding job is finished, the $! variable still holds the job’s PID:

$ echo $!
4189529

Next, let’s kill the job after starting it and see if $!‘s value will change:

$ ./flight-booking-api.sh > /tmp/test/flight.log &
[1] 3538

$ echo $!
3538

$ kill -9 3538
[1]  + 3538 killed     ./flight-booking-api.sh > /tmp/test/flight.log

$ echo $!
3538

As we can see in the test above, after we killed the job, the shell notified us that the job was killed. However, the $! variable still kept the killed process’s PID.

Therefore, the $! variable merely stores the PID of the most recently started background job. Further, its value won’t change depending on the process’s status.

6. The Usage of the $! Variable

Now that we understand what the $! variable is, let’s see the typical usage of this special variable through an example.

Let’s say we have a script to book holidays:

$ cat ./holiday-booking.sh
#!/bin/bash

# Booking Flight and Hotel through APIs in Parallel

echo "[$(date)] Calling flight booking API..."
./flight-booking-api.sh > /tmp/test/flight.log &
JOB_FLIGHT=$!

echo "[$(date)] Calling hotel booking API..."
./hotel-booking-api.sh > /tmp/test/hotel.log &
JOB_HOTEL=$!

echo "[$(date)] Await flight and hotel booking jobs' completion... "
wait $JOB_HOTEL $JOB_FLIGHT

echo "[$(date)] Holiday has been booked successfully."

In the holiday-booking.sh script, we need to book a hotel and a flight. However, these two API calls could take longer time. Therefore, we started two API scripts as background jobs to make them run in parallel.

However, we want to inform users that the holiday booking is successful only if both API call jobs are complete. So, after starting a background process, we read $!‘s value immediately and record the job’s PID in a variable, for example, JOB_FLIGHT and JOB_HOTEL.

Then we can use the wait command to wait for both jobs to finish before we proceed.

So next, let’s test our script:

$ ./holiday-booking.sh 
[Thu Apr  6 10:12:25 PM CEST 2023] Calling flight booking API...
[Thu Apr  6 10:12:25 PM CEST 2023] Calling hotel booking API...
[Thu Apr  6 10:12:25 PM CEST 2023] Await flight and hotel booking jobs' completion... 
#< ... blocking about 20 seconds ...>
[Thu Apr  6 10:12:45 PM CEST 2023] Holiday has been booked successfully

Finally, let’s verify the log files the two jobs produced:

$ head *.log
==> flight.log <==
Sending flight-booking data to the remote service....
Sending flight-booking data done! at Thu Apr  6 10:12:45 PM CEST 2023

==> hotel.log <==
Sending hotel-booking data to the remote service....
Sending hotel-booking data done! at Thu Apr  6 10:12:30 PM CEST 2023

7. Conclusion

In this article, we first discussed what the special shell variable $! means through examples. Further, we’ve seen its typical usage in shell scripting.