Authors Top

If you have a few years of experience in the Linux ecosystem, and you’re interested in sharing that experience with the community, have a look at our Contribution Guidelines.

1. Overview

The mv command is the most widely used tool for moving files on Linux. However, by default, it doesn’t have the ability to show progress to the user. Sometimes, when we move gigantic files, we need some kind of an indicator that we can use to track the progress of the move operation.

In this tutorial, we’ll learn how we can move files and see their progress in a terminal by using several alternatives to the Linux mv command.

2. progress

progress is a UNIX tool that enables us to monitor the progress of a copy or move operation. Normally, we’ll use the mv command to move a file or a directory and then use the progress command in another terminal to monitor the progress.

2.1. Installation

The progress tool doesn’t ship with the distributions. So, we’d need to install the progress package from our distro’s official repository using a package manager such as yum or apt.

On Debian and Ubuntu, do:

$ apt install progress

On Fedora, CentOS, and RHEL:

$ yum install progress

On Arch and its derivatives:

$ pacman -S progress

Once the installation is complete, let’s verify it:

$ progress -v
progress version 0.16

2.2. Usage

Now, as an example, we’ll move the ~/.cache directory to /tmp. Then, we’ll use the progress command with a few useful options. So, let’s start moving the directory:

$ mv ~/.cache /tmp

As we know, the mv command, by default, doesn’t print anything unless there is an error. We’ll keep it intact and open another terminal instance, where we monitor the command’s progress. So, let’s open another terminal instance and just type in the progress command:

$ progress
[ 5187] mv /home/hey/.cache/paru/clone/android-studio/android-studio-2020.3.1.26-linux.tar.gz
	0.0% (0 / 935.2 MiB)

The command will print out the progress at the current state and exit instantly. This is useful if we need to use the progress command in a shell script. On the other hand, if we want to monitor the progress, we’ll need to use the –monitor or -m option:

$ progress -mw

The -w or –wait option enables ETA. After running the command, we’ll see live progress of the mv command:

$ [ 5251] mv /home/hey/.cache/paru/clone/amdgpu-pro-installer/amdgpu-pro-21.30-1290604-ubuntu-20.04.tar.xz
	35.31% (241.3 / 683.2 MiB) 0/s

If we’ve got multiple mv operations, we can also provide the –pid option:

$ progress -mw --pid $(pgrep mv)

3. rsync

rsync is a highly configurable copying tool that supports a lot of options as compared to the cp and mv commands. We can use rsync to not only copy/move files locally but remotely as well. By default, rsync doesn’t show us any progress. Fortunately, there’s the -P option that displays a progress indicator for each file it copies.

3.1. Installation

Most complete distributions come with rsync pre-installed. If we’re on a minimal distribution, we can install it from the official repository for our distribution.

On Ubuntu derivatives:

$ apt install rsync

On Fedora, CentOS, and RHEL:

$ yum install rsync

On Arch and its derivatives:

$ pacman -S rsync

Once installed, let’s verify it:

$ rsync -V
rsync  version v3.2.3  protocol version 31
Copyright (C) 1996-2020 by Andrew Tridgell, Wayne Davison, and others.

3.2. Basic Usage

The basic syntax for copying files using rsync is:

$ rsync [OPTIONS] [SOURCE...] [DESTINATION]

We can specify as many sources as we need. Let’s copy our directory with rsync with the -r or —recursive option:

$ rsync -r ~/.cache/* /tmp/cache

You’ll notice that, similar to cp or mv command, it doesn’t print anything, with the exception of errors. Let’s enable the progress bar with the -P option:

$ rsync -r -P ~/cache/* /tmp/cache
chromium/Default/Cache/Cache_Data/2d0111cfb1b11d58_0
          6,177 100%   60.32kB/s    0:00:00 (xfr#811, ir-chk=1565/2529)
chromium/Default/Cache/Cache_Data/2d152dfa21362cbf_0
          7,724 100%   75.43kB/s    0:00:00 (xfr#812, ir-chk=1564/2529)
chromium/Default/Cache/Cache_Data/2d1e7c47b097a512_0
          9,468 100%   92.46kB/s    0:00:00 (xfr#813, ir-chk=1563/2529)
chromium/Default/Cache/Cache_Data/2d490754d7b0a757_0
.
.
.

Once we run the command, you’ll notice that rsync prints progress for each file. Now, it’s all good, but it still doesn’t remove the source files. By default, rsync doesn’t move files from one location to another – rather, it copies them. Luckily, we’ve also got the –remove-source-files option that will remove the source files once they are completely copied:

$ rsync -aP --remove-source-files ~/.cache /tmp/cache

3.3. A Better Way

Although the –remove-source-files option does remove the files, it doesn’t remove the directories from the sources. So, to make rsync emulate the mv command, we’re going to write a very simple bash script that will handle multiple sources and remove the source directories and files after being copied.

The script usage syntax will be similar to how we use rsync. First, we’ll provide one or more input files and the destination directory as the last argument. Then, we’ll remove the source files and directories entirely once the copying is complete:

#!/bin/bash

# Safety check
[ "$#" -eq 0 ] && "No input files" && exit 1

rsync -aP --remove-source-files "[email protected]" && rm -rf "${@:1:$#-1}"

The “[email protected]” operator will expand to the positional parameters, which in our case will be the sources and a destination. The second command removes the sources recursively. The “${@:1:$#-1}” expression will basically expand to the provided arguments minus the last one, which is our destination directory.

The command after “&&” will run only when the rsync commands exit successfully. Otherwise, the files will be kept and not deleted. Once we’re done, let’s take the script for a test after we’ve made it executable:

$ rsmv ~/.cache/ctrlp /tmp/cache
mru/
mru/cache.txt
            749 100%    0.00kB/s    0:00:00 (xfr#1, to-chk=0/2)

4. Pipe Viewer

Pipe Viewer or pv is a tool that we use to monitor the progress of data transferring through a pipeline. It’s useful in cases where we only have a huge file that we want to move while monitoring its progress. Although it doesn’t work for multiple files, we could write a script to handle multiple sources.

4.1. Installation

On most major distributions, pv is already installed. We can also install it from our official package repository.

On Debian and Ubuntu-derivatives, do:

# apt install pv

On Fedora, CentOS, and RHEL, do:

# yum install pv

On Arch-derivatives, do:

# pacman -S pv

Once it’s installed, we’ll verify it:

$ pv -V
pv 1.6.20 - Copyright 2015 Andrew Wood <[email protected]>

4.2. Usage

Using pv is pretty straightforward. Either we place the pv command before a pipe operation, or we can place it between two commands:

$ pv INPUT > OUTPUT
$ CMD | pv | CMD
$ CMD | pv [OPTIONS] > OUTPUT

As an example, let’s pipe the contents of one file into another file:

$ pv amdgpu-pro-21.30.tar.xz > /tmp/amdgpu-pro.tar.xz
683MiB 0:00:01 [ 494MiB/s] [==================================================================================>] 100%

Now, this is great, but we still have the original amdgpu-pro-21.30.tar.xz file in the source directory. For that reason, we’ll use the mv command and have pv measure the progress for us. The mv command has a -v or –verbose option that prints feedback for each file that is moved.

We’ll use the -v option for mv as a clue for the pv command to figure out the progress:

$ mv source_dir dest_dir | pv -l -s $(find source_dir | wc -l) > /tmp/log

Let’s break it down:

  • The -l option to the pv command will count lines instead of bytes
  • The -s option specifies the size of the data being transferred so that pv can estimate progress; by default, this is specified in bytes, but since we’ve used the -l option, it’s the number of files

We find the estimated total number of files through the find command, which we specified to the -s option. This way, pv will now update the progress as more files are moved.

Though this solution is not as elegant as the previous two, it should be a good starting point to writing a customizable shell script.

5. Conclusion

In this tutorial, we discussed several ways to monitor the progress of the move operation. First, we covered the UNIX-y progress tool and, then, went on to use the rsync tool. Finally, we used the Pipe Viewer along with the mv command.

Authors Bottom

If you have a few years of experience in the Linux ecosystem, and you’re interested in sharing that experience with the community, have a look at our Contribution Guidelines.

Comments are closed on this article!