How to Recursively grep All Directories and Subdirectories

1. Introduction

The grep command in Linux is a powerful tool for finding and extracting patterns from structured or unstructured data. Further, we can use grep to recursively search across all files in a directory and its subdirectories. Recursive grep offers a versatile and efficient solution for comprehensive searches in complex directory structures.

In this tutorial, we’ll learn how to recursively grep all directories and their subdirectories in Linux.

The code in this tutorial underwent testing on a Debian 12 system.

2. Basic Recursive Search With grep

To begin with, let’s understand the -r (–recursive) option for performing a recursive search using grep:

$ grep -r "Miller" .
./sql-tutorials/sql-queries/identify-duplicate-values/sample-dataset.sql:    (3, 'Linda', 'Miller', '[email protected]', 'Chicago'),
./sql-tutorials/sql-queries/identify-duplicate-values/sample-dataset.sql:    (3, 'Linda', 'Miller', '[email protected]', 'Chicago'),
./sql-tutorials/sql-queries/identify-duplicate-values/sample-dataset.sql:    (3, 'Linda', 'Miller', '[email protected]', 'Chicago'),

This command performs a recursive search (-r) for the term Miller starting from the current directory (.). The output displays all lines matching the provided pattern, i.e., Miller, from the files within the specified directory and its subdirectories.

Additionally, we can get the line numbers of the matches by using the -n (–line-number) option:

$ grep -rn "Miller" .
./sql-tutorials/sql-queries/identify-duplicate-values/sample-dataset.sql:15:    (3, 'Linda', 'Miller', '[email protected]', 'Chicago'),
./sql-tutorials/sql-queries/identify-duplicate-values/sample-dataset.sql:31:    (3, 'Linda', 'Miller', '[email protected]', 'Chicago'),
./sql-tutorials/sql-queries/identify-duplicate-values/sample-dataset.sql:33:    (3, 'Linda', 'Miller', '[email protected]', 'Chicago'),

This command outputs the filenames, followed by the line numbers and the matching line content from the files. Thus, the -n option can be essential for pinpointing the exact location of a match.

Lastly, the -l (–files-with-matches) option comes in handy when we want to focus only on filenames containing the text:

$ grep -rl "Miller" .
./sql-tutorials/sql-queries/identify-duplicate-values/sample-dataset.sql

This command results in a list of filenames relative to the current directory (.) that contain the term Miller. Each filename is displayed only once, even if grep finds matches on multiple lines within the file.

Alternatively, we can provide the absolute path instead of the relative path to get the full file paths in the output.

3. Excluding Specific Files From Recursive Searches

We can use grep to find all the matches except those in one or more files:

$ grep -rl "baeldung" ./sql-tutorials/ --exclude "sample-dataset.sql"
./sql-tutorials/README.md
./sql-tutorials/sql-queries/README.md

In this case, we utilize –exclude to recursively search for the word baeldung in the sql-tutorials/ directory while excluding the sample-dataset.sql file. As a result, no matches from the excluded files are shown in the output.

Similarly, we can also exclude directories from the recursive search:

$ grep -rl "baeldung" ./sql-tutorials/ --exclude-dir "sql-queries"
./sql-tutorials/README.md

The –exclude-dir option prevents grep from searching within the specified directory and its subdirectories.

Moreover, we can use the –exclude and –exclude-dir options multiple times to specify and exclude several files and directories from the recursive search.

4. Using grep and find Recursively

The combination of grep and find provides more advanced filters for file search and text matching. The find command locates and filters files with provided criteria, while grep searches for target text patterns within the filtered files.

To demonstrate, let’s search for the text only within files that have been modified in the last seven days:

$ find sql-tutorials/ -type f -mtime -7 -exec grep -H "baeldung" {} + 
sql-tutorials/README.md:This project contains SQL files that support the tutorials on https://www.baeldung.com/sql/ recently updated

Here, the find command leverages grep to search for the term baeldung in all files (-type f) that were modified in the last seven days (-mtime -7) and are present inside the sql-tutorials/ directory.

As another example, we can filter files based on file size before performing grep:

$ find sql-tutorials/ -type f -size +1k -exec grep -nH "baeldung" {} + 
sql-tutorials/sql-queries/identify-duplicate-values/sample-dataset.sql:13:    (1, 'John', 'Doe', '[email protected]', 'New York'),
...

The find command outputs only files larger than one kilobyte in size (-size +1k), while grep searches for the baeldung in each of the filtered file list. The -H option ensures the output of file names along with the matching lines.

5. Conclusion

In this article, we learned to recursively grep directories and their subdirectories.

We used several options to recursively search with grep and pinpoint matches in different files. Finally, we employed the combination of grep and find to filter files based on more advanced criteria and then execute grep on each of those filtered files.