Using rsync to Transfer Files

1. Overview

Remote Sync (rsync) is a utility for efficiently transferring and synchronizing files and directories across networked computers.

In this tutorial, we’ll learn how to use it to copy files between two networked computers that both have rsync installed.

2. Advantages of Using rsync

rsync provides many advantages as a file-copying tool, as it:

• supports copying links, devices, owners, groups, and permissions
• has –exclude and –exclude-from options that allow us to exclude files that match specific patterns
• can run securely over SSH
• can minimize the amount of data that is being transferred by only sending the files that are new or modified
• uses compression and decompression while sending and receiving data. So, we can use it to send bigger files

Now that we know about rsync and its advantages, let’s learn how to install it.

3. Installing rsync

rsync comes pre-installed on most Linux distros. But, if it got deleted, we can re-install it:

# On Debian-based distros

$ sudo apt update
$ sudo apt install rsync

# On Fedora

$ sudo dnf install rsync

# On CentOS and other Red Hat-based distros

$ sudo yum install rsync

# On openSUSE 

$ sudo zypper install rsync

# On Arch Linux

$ pacman -S rsync

After that, we can check if it’s installed:

$ rsync --version
rsync  version 3.1.3  protocol version 31
Copyright (C) 1996-2018 by Andrew Tridgell, Wayne Davison, and others.

We’ve successfully installed rsync.

4. Copying Files and Directories

Generally, when calling rsync, we need to use this pattern:

$ rsync [options] [source files and directories] [destination]

Now let’s see more specific cases.

4.1. Copying Files

To copy files from the local system to the remote system, we can call rsync in this manner:

$ rsync -a /home/user/Downloads/file.txt [email protected]:/home/Files
[email protected]'s password: 

In the above command:

• -a stands for archive mode; it is a quick way of saying that you want recursion (for syncing directories) and to preserve almost everything (symbolic links, permissions, modification times, group, ownership, device files)
• file.txt is the file that we want to copy
• /home/Files is the remote directory that we want to copy the file into

After running the command, it’ll ask us to enter the SSH password. After that, rsync will copy the file.

4.2. Copying a Directory

To copy a directory from the local system to the remote system, we can:

$ rsync -a /home/user/Downloads/Temp [email protected]:/home/Files
[email protected]'s password: 

The command will copy the Temp directory inside the /home/Files directory on the remote system. It’ll ask for the SSH password again, and then it’ll copy the directory.

4.3. Copying a Directory’s Content

To copy all files and directories inside a directory, we can run rsync like this:

$ rsync -a /home/user/Downloads/Temp/ [email protected]:/home/Files

After that, rsync will copy every file and folder inside the Temp directory to /home/Files.

Moreover, we can see that this time, the source directory has a trailing slash ‘/’ which means the content of Temp will be copied and not itself.

4.4. Excluding Files

We can also use the –exclude and –exclude-from options to exclude file names that match specific patterns. –exclude receives a single pattern, while —exclude-from receives multiple patterns located inside a file.

For example:

$ ls
file1.txt  file2.txt  file3.txt  new_file.pdf
$ rsync -a --exclude="*.txt" /home/user/Downloads/Temp/ [email protected]:/home/Files
[email protected]'s password: 

This will copy exclude all the text files inside the Temp directory. Therefore, only new_file.pdf will be copied.

Let’s see another example:

$ ls
exclude_patterns.txt  file1.txt  file2.txt  file3.txt  new_file.pdf  picture.jpg
$ cat exclude_patterns.txt 
*.txt
*.pdf
$ rsync -a --exclude-from="exclude_patterns.txt" /home/user/Downloads/Temp/ [email protected]:/home/Files/Temp
[email protected]'s password: 

This time, we had multiple patterns that we wanted to give rsync to exclude from. So, we wrote them inside a file named exclude_patterns.txt and gave it to rsync using the –exclude-from option.

After running the above command, only picture.png gets copied to the remote directory.

4.5. Sending Big Files

To transfer large amounts of data, we can add the -P option, which tells rsync to show progress during transfer and also to keep partially transferred files:

$ ls
file1.txt  file2.txt  file3.txt  new_file.pdf  picture.jpg  video.mkv
$ rsync -a -P /home/user/Downloads/Temp/ [email protected]:/home/Files/Temp
[email protected]'s password: 
sending incremental file list
./
file1.txt
              0 100%    0.00kB/s    0:00:00 (xfr#2, to-chk=5/8)
file2.txt
              0 100%    0.00kB/s    0:00:00 (xfr#3, to-chk=4/8)
file3.txt
              0 100%    0.00kB/s    0:00:00 (xfr#4, to-chk=3/8)
new_file.pdf
              0 100%    0.00kB/s    0:00:00 (xfr#5, to-chk=2/8)
picture.jpg
              0 100%    0.00kB/s    0:00:00 (xfr#6, to-chk=1/8)
video.mkv
    245,379,066 100%  570.99kB/s    0:06:59 (xfr#7, to-chk=0/8)

After running the above command, rsync copies every file inside the Temp directory to the remote directory.

5. Conclusion

In this tutorial, we looked at several examples of using rsync in different scenarios. rsync provides many advantages when synchronizing files and directories across networked computers.