Copy Directory Recursively, Excluding Some Subdirectories

1. Overview

Sometimes, we want to avoid copying a subdirectory and its contents while replicating a directory structure.

In this tutorial, we’ll explore tools that enable us to exclude one or more subdirectories while copying a directory in Linux.

2. Selective Copy Using mkdir and cp

First, let’s create a directory structure for our experiment and preview the contents with tree:

$ mkdir -p /tmp/baeldung/{dir1,dir2,dir3}/{sub1,sub2,sub3}
$ touch > /tmp/baeldung/{dir1,dir3}/{sub1,sub2}/{file1,file2}
$ tree /tmp/baeldung
/tmp/baeldung
├── dir1
│   ├── sub1
│   │   ├── file1
│   │   └── file2
│   ├── sub2
│   │   ├── file1
│   │   └── file2
│   └── sub3
├── dir2
│   ├── sub1
│   ├── sub2
│   └── sub3
└── dir3
├── sub1
│   ├── file1
│   └── file2
├── sub2
│   ├── file1
│   └── file2
└── sub3

12 directories, 8 files

For our example, let’s say we want to copy /tmp/baeldung to /tmp/baeldung-new, but we don’t want to retain /tmp/baeldung/dir1/sub1/ and /tmp/baeldung/dir3/sub2/, and their contents. Notably, there are several empty directories that should also be replicated.

In a naive approach, we can create the desired directory structure using mkdir in the target location first, and subsequently copy relevant files using cp:

$ mkdir /tmp/baeldung-new
$ cd /tmp/baeldung
$ find -path './dir1/sub1' -prune -o -path './dir3/sub2' -prune -o -type d -printf "%P\0" | xargs -0I{} mkdir '/tmp/baeldung-new/{}'
$ find -path './dir1/sub1' -prune -o -path './dir3/sub2' -prune -o -type f -printf "%P\0" | xargs -0I{} cp {} '/tmp/baeldung-new/{}'
$ tree /tmp/baeldung-new
/tmp/baeldung-new
├── dir1
│   ├── sub2
│   │   ├── file1
│   │   └── file2
│   └── sub3
├── dir2
│   ├── sub1
│   ├── sub2
│   └── sub3
└── dir3
├── sub1
│   ├── file1
│   └── file2
└── sub3

10 directories, 4 files

The first command creates the target directory. Then, we change our current working directory to /tmp/baeldung and this location becomes our current working directory throughout the rest of our experiment.

In the third line, we use the find command to exclude the undesired directories using the path parameters and prune action. The output contains a null-delimited list of our desired directories. The xargs command separates each item in the list and creates the directories in the storage using mkdir. A null delimited list enables us to avoid any problem due to whitespace.

Similarly, in the fourth line, we pipe the output of find to xargs and cp to copy the files to our target location. However, as we can see, this is an extensive process, and we can employ better tools to accomplish the same in a more succinct way.

3. Selective Copy Using find and cpio

We can combine the creation of a directory and copying files in a single command using find and cpio.

If the cpio command isn’t available, we need to install it first:

$ sudo apt install cpio

Let’s review the flags that we can use for our purpose:

-p, --pass-through
        Pass-through. Read a list of file names from the standard input and copy them to the specified directory.
-0, --null
        Filenames in the list are delimited by null characters instead of newlines
-d, --make-directories
        Create leading directories where needed.

So, we can filter out undesired directories using the find command and pipe the output to cpio to copy:

$ rm -r /tmp/baeldung-new
$ find -path './dir1/sub1' -prune -o -path './dir3/sub2' -prune -o -print0 | cpio -pd0 /tmp/baeldung-new
1 block
$ tree /tmp/baeldung-new
/tmp/baeldung-new
├── dir1
│   ├── sub2
│   │   ├── file1
│   │   └── file2
│   └── sub3
├── dir2
│   ├── sub1
│   ├── sub2
│   └── sub3
└── dir3
├── sub1
│   ├── file1
│   └── file2
└── sub3

10 directories, 4 files

In this method, we have filtered and copied the directory structure in a single command using find and cpio. This way, we avoid the use of a more complex pipeline and multiple commands.

4. Selective Copy Using tar

We can further reduce dependency on the find command if we use tar. The tar command can create and extract an archive. Furthermore, if we pipe the output of the creation command to the extraction command, there is very little overhead.

First, let’s review the interesting flags:

-c, --create
        Create a new archive.
--exclude=PATTERN
        Exclude files matching PATTERN
-f, --file=ARCHIVE
        Use archive file or device ARCHIVE. If ARCHIVE is "-", the output is redirected to stdout
-C, --directory=DIR
        Change to DIR before performing any operations.
-x, --extract, --get
        Extract files from an archive

Now, we can create a TAR archive excluding undesired directories and extract them in the new location using the tar command:

$ rm -r /tmp/baeldung-new
$ mkdir /tmp/baeldung-new
$ tar -cf -  --exclude './dir1/sub1' --exclude './dir3/sub2' . | tar -xC /tmp/baeldung-new
$ tree /tmp/baeldung-new
/tmp/baeldung-new
├── dir1
│   ├── sub2
│   │   ├── file1
│   │   └── file2
│   └── sub3
├── dir2
│   ├── sub1
│   ├── sub2
│   └── sub3
└── dir3
├── sub1
│   ├── file1
│   └── file2
└── sub3

10 directories, 4 files

As we can see, the tar method looks more efficient. In the second line, we ensure the target location exists before performing the TAR operation. This step isn’t necessary for other methods such as when using cpio.

5. Selective Copy Using rsync

The rsync command provides fast incremental file transfer.

In case it’s not available locally, we can install the utility using our package manager:

$ sudo apt install rsync

The exclude flag in the rsync command enables us to specify directories we do not want to copy:

$ rm -r /tmp/baeldung-new
$ rsync -a --exclude={'dir3/sub2','dir1/sub1'} . /tmp/baeldung-new
$ tree /tmp/baeldung-new
/tmp/baeldung-new
├── dir1
│   ├── sub2
│   │   ├── file1
│   │   └── file2
│   └── sub3
├── dir2
│   ├── sub1
│   ├── sub2
│   └── sub3
└── dir3
├── sub1
│   ├── file1
│   └── file2
└── sub3

10 directories, 4 files

Here, the brace expansion and exclude flag syntax performs our task well. Importantly, a leading slash isn’t required in the parameter to the exclude flag.

6. Selective Copy Using cp and Globbing

As with most scenarios that involve including or excluding more than a single object, we can turn to globbing as a potential solution.

In fact, by using the basic cp with advanced globbing, we can achieve our aims:

$ rm -r /tmp/baeldung-new
$ shopt -s extglob
$ cp --verbose --archive ./!(dir3) /tmp/baeldung-new
'./dir1' -> '/tmp/baeldung-new/dir1'
'./dir1/sub1' -> '/tmp/baeldung-new/dir1/sub1'
'./dir1/sub1/file1' -> '/tmp/baeldung-new/dir1/sub1/file1'
'./dir1/sub1/file2' -> '/tmp/baeldung-new/dir1/sub1/file2'
'./dir1/sub2' -> '/tmp/baeldung-new/dir1/sub2'
'./dir1/sub2/file1' -> '/tmp/baeldung-new/dir1/sub2/file1'
'./dir1/sub2/file2' -> '/tmp/baeldung-new/dir1/sub2/file2'
'./dir1/sub3' -> '/tmp/baeldung-new/dir1/sub3'
'./dir2' -> '/tmp/baeldung-new/dir2'
'./dir2/sub1' -> '/tmp/baeldung-new/dir2/sub1'
'./dir2/sub2' -> '/tmp/baeldung-new/dir2/sub2'
'./dir2/sub3' -> '/tmp/baeldung-new/dir2/sub3'

Of course, after clearing the target directory, we use shopt to enable the extglob setting.

Next, we perform a full archive copy, meaning we preserve all file attributes and duplicate everything recursively. To ensure the correct data goes through, we also add more verbosity.

Importantly, the !(dir3) globbing exclusion ensures we don’t perform any operations on the dir3 subdirectory. Similarly, we can use other globbing expression forms:

• ./!(dir1|dir3): exclude both dir1 or dir3
• ./**/!(sub1): exclude any paths with sub1
• ./**/!(sub1|sub2): exclude any paths with sub1 or sub2

Let’s check the resulting structure for ./!(dir3):

$ tree /tmp/baeldung-new
/tmp/baeldung-new
├── dir1
│   ├── sub1
│   │   ├── file1
│   │   └── file2
│   ├── sub2
│   │   ├── file1
│   │   └── file2
│   └── sub3
└── dir2
    ├── sub1
    ├── sub2
    └── sub3

9 directories, 4 files

As expected, we don’t see the dir3 top-level directory.

7. Conclusion

In this article, we have explored several methods to copy a subset of a directory structure to the destination.

For minor exclusions, using cp with globbing is usually sufficient. In a minimal Linux installation, the tar method is usually the most suitable. Additionally, tar can also be employed remotely. However, rsync is the as it enables the specification of directories via brace expansion.