How to Monitor the Progress of dd

1. Overview

The dd command is well-known for converting and copying data. Still, the execution of the command is time-consuming when it’s used to copy large files. Therefore, it’s often helpful to have a means of seeing how the command is progressing.

In this tutorial, we’ll see ways to monitor the progress of the dd command.

2. Default Execution

The default settings of the dd command don’t provide any indication of the progress during execution. Nevertheless, dd provides a synopsis after the process is finished or stopped.

To demonstrate this, we’ll run the command so that it infinitely copies /dev/zero to a destination file in our working directory:

$ dd if=/dev/zero of=dest bs=1
^C1293197+0 records in
1293196+0 records out
1293196 bytes (1.3 MB, 1.2 MiB) copied, 2.72658 s, 474 kB/s

As we can see, we stopped the command’s execution with the Ctrl+c key combination. While the command was running, there was no indication of the progress, like how many blocks have been copied.

Upon stopping the execution, the command printed a summary that included several pieces of information:

1. blocks read
2. blocks written
3. number of bytes
4. execution time
5. data transfer rate

Having some or all of these statistics while copying can be handy. Let’s see how we can achieve that.

3. The status Option

The status option of the dd command sets the level of information printed to stderr. It accepts three values:

1. none displays no information besides errors, not even an execution summary
2. noxfer prints only an execution summary
3. progress provides information about the progress of the transfer while the command is running as well as an execution summary

Consequently, we can set the progress value to the status option of the dd command to monitor the progress while dd is running:

As we anticipated, we were able to monitor the progress of the data transfer. The command displays how many bytes of data have been transferred, the execution time, and the transfer rate. All of these are updated in real-time.

If we stop the execution, a summary of the data transfer is still displayed:

$ dd if=/dev/zero of=dest bs=1 status=progress
8522722 bytes (8.5 MB, 8.1 MiB) copied, 18 s, 473 kB/s^C
8770947+0 records in
8770947+0 records out
8770947 bytes (8.8 MB, 8.4 MiB) copied, 18.5756 s, 472 kB/s

Indeed, we can see the final statistics of the data transfer.

4. Using the watch Command

An alternative way to monitor the progress of the dd command is to use watch. The watch command executes a designated command periodically and displays its output in full screen.

4.1. Monitoring the Output File Size

The concept here is that we can run the dd command as a background job and then run ls periodically through watch to monitor the output file’s size.

First, let’s execute dd as a background job:

$ dd if=/dev/zero of=dest bs=1 &
[1] 70269

As we can see, we used & to run dd in the background.

Next, let’s run ls through watch to monitor the changes in the size of the dest file:

$ watch -d ls -alh dest

Here, the -d option highlights changes in the output. In this case, we see the size of the dest file being highlighted:

As we expected, we can monitor the progress of the data transfer.

4.2. Sending a USR1 Signal to dd

Interestingly, the dd command prints a summary of the data transfer, upon receiving a USR1 signal. Combining this functionality with the watch command, enables us to monitor the progress of our data transfer.

As previously, let’s start by running dd as a background job:

$ dd if=/dev/zero of=dest bs=1 &
[1] 70681

Following, we send a USR1 signal to dd via the kill command. The watch command can periodically perform this step:

$ watch kill -USR1 $(ps | grep dd | awk '{print $1}')

Here, we identify the PID of the process that executes dd using the ps command in conjunction with the grep and awk commands. The grep command filters the result of ps using the dd keyword, and awk prints only the column that contains the process ID. Finally, kill sends the USR1 signal to the process ID that we find.

So, watch prints the dd summary every two seconds:

Indeed, watch repeatedly printed the summary of the dd command. As a result, we’re able to monitor the progress of the data transfer.

4.3. The progress Command

Another way to display the progress of dd is to use the progress command. On Ubuntu distributions, we can install progress with apt:

$ sudo apt install progress

Notably, progress can monitor only some commands of the coreutils package, like cp, dd, tar, and others.

The progress command doesn’t repeatedly execute itself, so we should use it together with the watch command.

First, let’s start a new dd process running in the background. Following that, we invoke progress through watch:

$ watch progress -w -c dd

Here, we utilize two options for progress:

1. w estimates I/O throughput
2. c monitors only the specified command name

In this case, we set dd to the c option so that we monitor the progress of the dd command that we run in the background:

As we anticipated, we can see the bytes transferred, as well as the data transfer rate.

5. Conclusion

In this article, we explored methods to monitor the progress of the dd command. Initially, we saw the default behavior of the command when it comes to displaying information. Subsequently, we learned about the status option of dd that can adjust the command’s output level. Finally, we used watch to periodically run three commands:

1. ls to monitor the output file’s size
2. kill to send a USR1 signal to dd, so that it displays an execution summary
3. progress, to display the bytes transferred and the data rate

In conclusion, we have many choices to monitor the dd command.