1. Introduction

Limits are a vital part of any computer system. Having them ensures consistent and defined behavior even when processes don’t enforce their own constraints. One category of limits involves the filesystem. In particular, there are restrictions on the number of filesystem objects, as well as their size.

In this tutorial, we explore size limits for files, volumes, and directories for different filesystems. First, we talk about the subdivisions we can encounter when working with storage components. Next, we go over native Linux filesystems, their characteristics, directory size, and other limitations. After that, we turn to more universal filesystem types. Finally, we cover the native storage organization of another major operating system (OS).

We tested the code in this tutorial on Debian 12 (Bookworm) with GNU Bash 5.2.15. It should work in most POSIX-compliant environments unless otherwise specified.

2. Size Containers

In theory, storage can be arbitrarily big. In practice, physical constraints make storage mediums of a limited size. Moreover, these limits are applied to different levels of the storage component.

Further, Linux rarely uses components like a hard disk drive (HDD) and solid-state disk (SSD) without preparing them with some structure.

2.1. Partitioning

To begin with, we separate the empty storage medium into partitions, providing different areas depending on the operating system, data types, files, and others:

$ fdisk --list /dev/sda
Disk /dev/sda: 32 GiB, 34359738368 bytes, 67108864 sectors
Disk model: xdisk
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disklabel type: dos
Disk identifier: 0x16bfb916

Device     Boot    Start      End  Sectors  Size Id Type
/dev/sda1  *        2048 40677375 40675328 19.4G 83 Linux
/dev/sda2       40677376 67108863 26431488 12.6G  5 Extended
/dev/sda5       40679424 61364223 20684800  9.9G 83 Linux
/dev/sda6       61366272 63365119  1998848  976M 82 Linux swap / Solaris
/dev/sda7       63367168 63913983   546816  267M 83 Linux
/dev/sda8       63916032 67108863  3192832  1.5G 83 Linux

Here, we employ the fdisk command to –list all partitions on drive /dev/sda.

The partition table is very much like the interior walls within a house that separate the rooms. Exactly like walls, this separation takes up space, albeit not very much.

2.2. Apply Filesystems

Next, we usually format each partition with a filesystem. The formatting process provides even more structure to an empty partition:

Again, this process reduces the available storage, since the formatting structure itself takes away from the free space.

Importantly, the size limits set on sub-containers like directories are dictated by the filesystem type. So, let’s look at specific constraints different filesystems impose.

3. ext* Filesystems

The ubiquitous ext* filesystems have had several versions over the years:

Actually, one of the main differences between them comes down to the object counts and sizes they support.

3.1. ext2

To begin with, some constraints of the older ext2 depend on the chosen Block Size, i.e., the minimum allocation unit:

-----------------------------------------------------------------------------------------+
| Block Size [KiB] | Total Files | Files Dir     | Subdirs Dir  | File Size | Volume Size |
|------------------+-------------+---------------+--------------+-----------+-------------|
| 1                |             |               |              | 16GiB     | 4TiB        |
|------------------|             |               |              |-----------+-------------|
| 2                |             |               |              | 256GiB    | 8TiB        |
|------------------| 10^18       | ~ 1.3 * 10^20 | 31998        |-----------+-------------|
| 4                |             |               |              | 2TiB      | 16TiB       |
|------------------|             |               |              |-----------+-------------|
| 8                |             |               |              | 2TiB      | 32TiB       |
+-----------------------------------------------------------------------------------------+

Here, we have the block size in the first column, while the next columns provide different hard maximum limits.

So, from the total number of files in a directory Files Dir and the maximum File Size, we can extrapolate the maximum theoretical directory size to be between around 1.3 * 10^20 * 16 and 1.3 * 10^20 * 2^11 GiB. Yet, since these limits are way over the Volume Size, the practical constraints are much less and near the Volume Size bottleneck.

In fact, having too many subdirectories or files within a directory can severely degrade the performance to the point of the system becoming unusable in certain cases.

3.2. ext3

Next, the ext3 filesystem improves on some limitations of ext2:

+-------------------------------------------------------------------------------------------+
| Block Size [KiB] | Total Files | Files Dir     | Subdirs Dir  | File Size | (Vol)ume Size |
|------------------+-------------+---------------+--------------+-----------+---------------|
| 1                |             |               |              | 16GiB     | 4TiB          |
|------------------|             |               |              |-----------+---------------|
| 2                | min(        |               |              | 256GiB    | 8TiB          |
|------------------| Vol / 2^13, | ~ 1.3 * 10^20 | 31998        |-----------+---------------|
| 4                | block count |               |              | 2TiB      | 16TiB         |
|------------------| )           |               |              |-----------+---------------|
| 8                |             |               |              | 2TiB      | 32TiB         |
+-------------------------------------------------------------------------------------------+

Still, despite the increased limits, we’re in the same situation as with ext2: the (Vol)ume Size decides the upper bound for a directory size.

3.3. ext4

When it comes to ext4, the block size doesn’t have an effect to the overall limits:

+-----------------------------------------------------------------+
| Total Files | Files Dir | Subdirs Dir | File Size | Volume Size |
|-------------+-----------+-------------+-----------+-------------|
| 2^32 - 1    | unlimited | unlimited   | 2^44b - 1 | 2^48b - 1   |
|             |           |             | 16TiB - 1 | 256TiB - 1  |
+-----------------------------------------------------------------+

Naturally, again, with an unlimited number of files in a directory, the final size of the latter only depends on the Volume Size regardless of the File Size.

4. XFS Filesystem

The kernel-supported XFS is also a classic choice when it comes to native Linux filesystems.

Due to its focus and internal implementation, XFS has noticeably different limits:

+-----------------------------------------------------------------+
| Total Files | Files Dir | Subdirs Dir | File Size | Volume Size |
|-------------+-----------+-------------+-----------+-------------|
| 2^64        | 2^64      |  2^64       | 2^63b - 1 | 2^63b - 1   |
|             |           |             | 8EiB - 1  | 8EiB - 1    |
+-----------------------------------------------------------------+

Similar to the case with ext*, we end up with the upper limit of a directory being the Volume Size, which in this case is around 8EiB.

5. File Allocation Table (FAT32) Filesystem

Although not tailored or native to Linux, the ubiquitous File Allocation Table (FAT32) filesystem plays a key role as it’s fairly simple, universally supported, and easier to implement.

However, these characteristics come with a cost:

+-----------------------------------------------------------------+
| Total Files | Files Dir | Subdirs Dir | File Size | Volume Size |
|-------------+-----------+-------------+-----------+-------------|
| ~ 2^28      | 2^16 - 1  | 2^16 - 1    | 4GB       | 2-16TB      |
| 268173300   | 65535     | 65535       |           |             |
+-----------------------------------------------------------------+

With one of the lowest Total Files counts and the absolute lowest File Size, the maximum directory size of a FAT32 structure is again limited by the Volume Size, which varies according to sectors and clusters but is very low overall.

6. exFAT

The Extensible File Allocation Table (exFAT) aimed to upgrade and replace the now outdated FAT32 filesystem, particularly for USB devices.

In many ways, exFAT managed to do that. However, it wasn’t as widely adopted despite its relatively more complex implementation as well as the already-established FAT32 components in many systems, especially embedded ones.

Still, the benefits of exFAT in terms of its characteristics are obvious:

+-----------------------------------------------------------------+
| Total Files | Files Dir | Subdirs Dir | File Size | Volume Size |
|-------------+-----------+-------------+-----------+-------------|
| unlimited   | > 2^21    | > 2^21      | 128PB     | 128PB       |
|             | 2796202   | 2796202     |           |             |
+-----------------------------------------------------------------+

In this case, the number of objects in a directory is again limited, but much higher than that for FAT32. Yet, the total file count is unlimited. So, an exFAT directory size is again only really constrained by the Volume Size.

7. New Technology File System (NTFS)

As the filesystem native to Microsoft Windows, the New Technology File System (NTFS) imposes certain restrictions but manages to compete with Linux filesystems:

+----------------------------------------------------------------------------+
| Total Files | Files Dir | Subdirs Dir | File Size     | Volume Size        |
|-------------+-----------+-------------+---------------+--------------------|
| ~ 2^32 - 1  | 2^16 - 1  | 2^16 - 1    | 2^44 - 26     | 2^32 - 1 clusters  |
| 4294967295  | 65535     | 65535       | 16TiB - 64KiB | 256TiB - 64KiB     |
+----------------------------------------------------------------------------+

In this context, a cluster or block is 64KiB. Critically, there are different versions but we’re only looking at the most recent version of NTFS, since, similar to Linux, older ones are now rarely used in special cases.

Further, NTFS has two theoretical limits:

  • file size limit of 2^64 – 26 bytes, i.e., 16EiB – 64KiB
  • volume size limit of 2^64 – 1 clusters, i.e., 16EiB – 64KiB

Again, we see the same pattern: Volume Size deciding the maximum directory size.

8. Summary

In this article, we talked about different major filesystems, their constraints, and the maximum directory size of each.

In conclusion, we established that the main limitations for a directory are the permitted maximum volume size and the maximum number of files per directory for the filesystem.

Comments are closed on this article!