1. Introduction

The du command is a way to get file size information. Although it has many options, filtering by extension isn’t part of them. While extensions aren’t that critical in Linux, they still usually distinguish file types. Because of this, adding extensions, as well as separating and handling data based on them, is often beneficial.

In this tutorial, we’ll look at the du command and methods to check the total file size of files with a particular extension. First, we go over file sizes and space usage. Next, we thoroughly explore the du command. Finally, we check ways to use du for our needs.

We tested the code in this tutorial on Debian 11 (Bullseye) with GNU Bash 5.1.4. It should work in most POSIX-compliant environments unless otherwise specified.

2. File Sizes and Space Usage

To begin with, files usually occupy at least a given minimal chunk of storage due to the blocks at different levels in the hierarchy:

  • physical device controller builds, where 512-byte sectors or blocks are common
  • filesystem configuration and limitations may necessitate different block or allocation unit sizes like 1024b, 2048b, 4096b, 8192b, and more

So, instead of getting the size of a file, we might end up getting the space used by it. Also, there are other specific conditions to consider:

Thus, a 1b file would usually occupy at least the physical block size of 512 bytes on most physical storage mediums. Meanwhile, the actual logical size taken up by its data on the partition filesystem is usually larger. On the other hand, a 0b file often resides only as a logical mark instead of actually using even a single block of space on storage:

+------------------------------------------------------------------------------------------+
|            Real Size       |   Apparent Size  |            Size on Storage               |
+------------------------------------------------------------------------------------------+
| [ minBlockSize, inf]       | realSize         | ceil(realSize/minBlockSize)*minBlockSize |
|       ^                    | + fragmentedData | + fragmentedData                         |
|   min(physicalBlockSize,   | + indirectBlocks | + indirectBlocks                         |
|       filesystemBlockSize) | + sparseData     | + sparseData                             |
|------------------------------------------------------------------------------------------|
| [1b, minBlockSize]         | realSize         | minBlockSize                             | 
|------------------------------------------------------------------------------------------|
| 0b                         | 0b               | 0b                                       |
+------------------------------------------------------------------------------------------+

Because of these file size discrepancies, some tools have a way of getting the apparent and real sizes of a file.

3. The du Command

The du (Disk Usage) command is a way to check how much space given files use and what their sizes are:

$ du
12      ./subdir
40      .

In our sample dataset, we see the current working directory . (period) uses up a total of 40K on the medium, while the only subdirectory subdir accounts for 12K.

Alternatively, we can pass a file or directory as an argument:

$ du ./dir
12      ./dir/subdir
40      ./dir

To do so for multiple files or directories, we use –files0-from with a file name or for stdin as an argument:

$ printf './dir/file1\0./dir/file2\0./dir\0' | du --files0-from=-
4       ./dir/file1
4       ./dir/file2
12      ./dir/subdir
32      ./dir

Either source should pass all desired paths with NUL terminators, as we did here with printf and \0.

3.1. Human-Readable Sizes

Since 1K blocks aren’t always a helpful measure of sizes, du offers flags to augment this unit:

So, let’s try to get a better picture of sizes:

$ du --human-readable
12K     ./subdir
40K     .
$ du --bytes
6199    ./subdir
24224   .

Still, from the above, only the non-standard –bytes or -b ensures we get the apparent file size. Let’s see a switch for that specific purpose.

3.2. Apparent Size

Due to the varying file sizes depending on the viewpoint, du has the –apparent-size flag for getting the as-close-as-possible real file size:

$ du --human-readable file1
4.0K    file1
$ du --human-readable --apparent-size file1
666     file1

Notably, the second output displays the actual byte size instead of an approximation. This way, we can avoid hardware and filesystem blocks to get the actual size instead of space usage. For most purposes, –bytes (-b) and –apparent-size (which -b implies) provide the most accurate file size.

3.3. List Files

Instead of providing files as arguments, we can use –all or -a to list the whole hierarchy below the current or supplied directory with sizes:

$ du --bytes --all
666     ./file1
79      ./file2
1120    ./subdir/file4
983     ./subdir/file5
6199    ./subdir
13184   ./file3
24224   .

Now, we have a wider view of the size distributions.

In addition to the usual –dereference (-L) and –no-dereference (-P) flags to follow and not follow links, we can dereference only arguments with –dereference-args or -D (-H).

Importantly, we can also perform filtering:

In fact, we can filter and get a total for a given subset of the data.

3.4. Totals

Conveniently, we can get a –total (-c) of the sizes for a given directory:

$ du --human-readable --apparent-size --all --total ./dir
666     ./dir/file1
79      ./dir/file2
1.1K    ./dir/subdir/file4
983     ./dir/subdir/file5
6.1K    ./dir/subdir
13K     ./dir/file3
24K     ./dir
24K     total

Thus, we see the total directory size along with its whole structure and a breakdown. The flag is often more useful when combining files and directories from different paths:

$ printf '/etc/ssh/sshd_config\0./dir\0' | du --human-readable --apparent-size --all --total --files0-from=-
3.3K    /etc/ssh/sshd_config
666     ./dir/file1
79      ./dir/file2
1.1K    ./dir/subdir/file4
983     ./dir/subdir/file5
6.1K    ./dir/subdir
13K     ./dir/file3
24K     ./dir
27K     total

Moreover, we can extract only the last row value for each argument with –summarize or -s:

$ du --human-readable --apparent-size --summarize ./dir
24K     ./dir

Further, we can combine both options and use tools like tail to get only the last row.

Now, let’s see how we can get the total for a given extension.

4. File Size for Extension

There are several ways to get the total size for files of a specific type.

4.1. Using Globbing

Many shells support globbing to select multiple paths at once.

First, we can apply globbing by file name within a single directory without recursion to get the total size of .c files:

$ du --human-readable --apparent-size --total --summarize ./dir/*.c

To descend in any single subdirectory, we can add the ** double wildcard supported by Bash:

$ du --human-readable --apparent-size --total --summarize ./dir/**/*.c

In this case, we exclude the .c files in the root of ./dir. Yet, we do have a way to get the total size of all files with a given extension under a provided path.

4.2. Using find

As usual, the output of find provides a flexible way to process file paths:

$ LC_ALL=C find ./dir -iname '*.c' -type f -print0 |
  du --human-readable --apparent-size --total --summarize --files0-from=- |
  tail --lines=1

In essence, we use find with a NUL-terminated -print0 to output all paths of [f]ile -type objects with an -iname match of *.c, i.e., .C and .c. Notably, we ensure the locale settings are standard with LC_ALL=C.

All paths we get are piped to du for processing with –files0-from=-. Finally, we only extract the total summary with tail.

5. Summary

In this article, we looked at the du command and ways to get the total size of files with a given extension.

In conclusion, while du is a flexible command, combining it with find produces more detailed results when it comes to directory traversal.

Comments are closed on this article!