Huge Page Settings and Disabling Huge Pages in Linux

Last updated: March 18, 2024

Reviewed by: Michal Aibin

1. Introduction

The minimal chunk of random access memory (RAM) is called a page. Usually around 4K in size, pages are also the smallest unit for swapping. Since a small page-to-RAM ratio means many pages and potentially slow addressing, Linux supports huge pages (hugepages, HP) with much bigger sizes. In fact, there are also transparent huge pages (THP) that the kernel strives to integrate seamlessly. Despite this, the feature introduces its own drawbacks.

In this tutorial, we’ll discuss ways to disable huge pages in Linux. First, we probe multiple sources for checking the current state and configuring standard HP and THP. Next, we go over the basic settings for both. Finally, we explore how to disable standard HP and THP.

We tested the code in this tutorial on Debian 11 (Bullseye) with GNU Bash 5.1.4. It should work in most POSIX-compliant environments unless otherwise specified.

2. Huge Page Settings

We have many ways to browse through the settings and statistics for huge pages. To find them faster, we can filter the output of commands with grep.

2.1. Using /proc/vmstat

As usual, the /proc pseudo-filesystem provides useful information about the current state of the kernel and virtual memory statistics. Many virtual memory parameters can be found in the /proc/vmstat file, available since Linux kernel version 2.6:

$ cat /proc/vmstat | grep -e 'thp' -e 'huge*page'
nr_shmem_hugepages 0
nr_file_hugepages 0
nr_anon_transparent_hugepages 12
thp_migration_success 0
thp_migration_fail 0
thp_migration_split 0
thp_fault_alloc 79
thp_fault_fallback 0
thp_fault_fallback_charge 0
thp_collapse_alloc 6
thp_collapse_alloc_failed 0
thp_file_alloc 0
thp_file_fallback 0
thp_file_fallback_charge 0
thp_file_mapped 0
thp_split_page 10
thp_split_page_failed 0
thp_deferred_split_page 13
thp_split_pmd 28
thp_split_pud 0
thp_zero_page_alloc 1
thp_zero_page_alloc_failed 0
thp_swpout 0
thp_swpout_fallback 0

In this case, we mainly see settings for transparent huge pages.

2.2. Using /proc/meminfo

In particular, the /proc/meminfo file shows huge page data, each with a separate meaning:

$ cat /proc/meminfo | grep --ignore-case -e 'huge' -e 'filepmd' | grep --invert-match 'Shmem'
AnonHugePages:     10666 kB
FileHugePages:         0 kB
FilePmdMapped:         0 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
Hugetlb:               0 kB

Moreover, two of these parameters are particularly important when it comes to the status of HP and THP.

2.3. Using /proc/cpuinfo

The /proc/cpuinfo file contains the flags currently on for our central processing unit (CPU):

$ cat /proc/cpuinfo
processor       : 0
vendor_id       : GenuineIntel
cpu family      : 066
model           : 142
model name      : Intel(R) Core(TM) i6-6660U CPU @ 6.60GHz
stepping        : 10
microcode       : 0xffffffff
cpu MHz         : 6660.667
cache size      : 8192 KB
physical id     : 0
siblings        : 2
core id         : 0
cpu cores       : 1
apicid          : 0
initial apicid  : 0
fpu             : yes
fpu_exception   : yes
cpuid level     : 21
wp              : yes
flags           : [...]

Among these, we might find two flags relating to huge pages:

  • pse – 2M page support
  • pdpe1gb – 1G page support

Either only indicates support, but doesn’t mean that a feature is enabled in the operating system (OS).

2.4. Using sysctl

In addition, we can use the sysctl tool to get all available settings and filter that output:

$ sysctl -a | grep 'huge*page'
vm.nr_hugepages = 0
vm.nr_hugepages_mempolicy = 0
vm.nr_overcommit_hugepages = 0

The main parameter, vm.nr_hugepages at /proc/sys/vm/nr_hugepages, indicates the persistent explicit huge page count, which we can change as a superuser.

2.5. Exploring /sys

Of course, we can traverse the /sys pseudo-filesystem for data and settings related to huge pages using the find command and its wildcard-enhanced -name switch:

$ find /sys -name '*huge*page*'
/sys/kernel/mm/hugepages
/sys/kernel/mm/hugepages/hugepages-2048kB
/sys/kernel/mm/hugepages/hugepages-2048kB/free_hugepages
/sys/kernel/mm/hugepages/hugepages-2048kB/resv_hugepages
/sys/kernel/mm/hugepages/hugepages-2048kB/surplus_hugepages
/sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages_mempolicy
/sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages
/sys/kernel/mm/hugepages/hugepages-2048kB/nr_overcommit_hugepages
/sys/kernel/mm/hugepages/hugepages-1048576kB
[...]
/sys/kernel/mm/transparent_hugepage
/sys/kernel/mm/transparent_hugepage/khugepaged
/sys/kernel/tracing/events/huge_memory/mm_khugepaged_scan_pmd
/sys/kernel/tracing/events/huge_memory/mm_collapse_huge_page
/sys/kernel/tracing/events/huge_memory/mm_collapse_huge_page_isolate
/sys/kernel/tracing/events/huge_memory/mm_collapse_huge_page_swapin
/sys/kernel/slab/khugepaged_mm_slot
/sys/kernel/debug/split_huge_pages
/sys/kernel/debug/tracing/events/huge_memory/mm_khugepaged_scan_pmd
/sys/kernel/debug/tracing/events/huge_memory/mm_collapse_huge_page
/sys/kernel/debug/tracing/events/huge_memory/mm_collapse_huge_page_isolate
/sys/kernel/debug/tracing/events/huge_memory/mm_collapse_huge_page_swapin
/sys/devices/system/node/node0/hugepages
/sys/devices/system/node/node0/hugepages/hugepages-2048kB
/sys/devices/system/node/node0/hugepages/hugepages-2048kB/free_hugepages
[...]
/sys/fs/cgroup/dev-hugepages.mount

In this case, we have both statistics and configuration options. Notably, a directory exists for each huge page size.

2.6. Main Settings

Let’s use tree to dig deeper into the main huge pages settings directories in /sys:

$ tree /sys/kernel/mm/hugepages /sys/kernel/mm/transparent_hugepage

/sys/kernel/mm/hugepages
├── hugepages-1048576kB
│   ├── free_hugepages
│   ├── nr_hugepages
│   ├── nr_hugepages_mempolicy
│   ├── nr_overcommit_hugepages
│   ├── resv_hugepages
│   └── surplus_hugepages
└── hugepages-2048kB
    ├── free_hugepages
    ├── nr_hugepages
    ├── nr_hugepages_mempolicy
    ├── nr_overcommit_hugepages
    ├── resv_hugepages
    └── surplus_hugepages
    
/sys/kernel/mm/transparent_hugepage
├── defrag
├── enabled
├── hpage_pmd_size
├── khugepaged
│   ├── alloc_sleep_millisecs
│   ├── defrag
│   ├── full_scans
│   ├── max_ptes_none
│   ├── max_ptes_shared
│   ├── max_ptes_swap
│   ├── pages_collapsed
│   ├── pages_to_scan
│   └── scan_sleep_millisecs
├── shmem_enabled
└── use_zero_page

3 directories, 26 files

Some Linux distributions may include a prefix such as redhat to the *hugepage directories.

At this point, we should have the majority of options available for checking and managing the huge page configuration and statistics of our system. Let’s now distinguish and toggle the main ones.

3. Standard Huge Pages (HP) and Transparent Huge Pages (THP)

There are several specifics of the huge page technologies:

  • the kernel needs to be built with the CONFIG_HUGETLBFS and CONFIG_HUGETLB_PAGE options
  • standard huge pages (HP) pre-allocate memory at startup
  • transparent huge pages (THP) need dynamic memory allocation at runtime via the khugepaged kernel thread
  • THP currently only works for anonymous memory mappings and tmpfs or shmem
  • best practices in many production environments, such as database servers, suggest disabling THP, but leaving standard HP

Considering this, let’s see how we can check the current status of both.

3.1. Check Standard Huge Pages (HP) Support and Status

The usual way to verify support for standard HP is by checking for the relevant flags in the /proc/cpuinfo file we saw earlier.

$ cat /proc/cpuinfo
[...]
flags           : [...] pdpe1gb pse [...]

In this case, the CPU supports 2M and 1G huge page sizes. Another way to know we can use HP is the hugetlbfs entry in the /proc/filesystems file:

$ cat /proc/filesystems
[...]
nodev   hugetlbfs
[...]

Furthermore, we can verify standard huge pages exist in the system by ensuring the value of HugePages_Total is a positive integer:

$ cat /proc/meminfo | grep 'HugePages_Total:'
HugePages_Total:       10

After that, we check for hugepagesz= and hugepages= in the kernel boot parameters. In fact, hugepages= is equivalent to the vm.nr_hugepages (/proc/sys/vm/nr_hugepages) setting, but it’s more reliable since memory is not fragmented on boot.

3.2. Check Transparent Huge Pages (THP) Support and Status

Naturally, THP relies upon standard HP, so checking for that first is critical.

One simple way to verify THP support, in particular, is by checking for a positive integer after the AnonHugePages value in the /proc/meminfo file:

$ cat /proc/meminfo | grep 'AnonHugePages:'
AnonHugePages:      6660 kB

In addition, the transparent_hugepage= kernel boot parameter can indicate whether we have THP enabled.

4. Disable Huge Pages

Due to their potentially profound effect on performance, whether we use standard HP or THP or we don’t use huge pages at all can make a significant difference.

In all cases where kernel boot parameters are involved, we can use the standard GRUB files:

  • /etc/default/grub, in GRUB_CMDLINE_LINUX
  • /etc/grub.conf, on the kernel line

Naturally, these vary according to the bootloader.

4.1. Standard Huge Pages

There are several steps to ensure we don’t use standard huge pages and, by extension, transparent huge pages.

First, we set vm.nr_hugepages (/proc/sys/vm/nr_hugepages) to 0:

$ sysctl vm.nr_hugepages=0
[OR]
$ echo 0 > /proc/sys/vm/nr_hugepages

Of course, this only works at runtime until a reboot. To persist the setting, we can add vm.nr_hugepages=0 to /etc/sysctl.conf.

Next, to turn off HP at boot time, we remove the relevant kernel boot parameters hugepagesz= and hugepages=.

4.2. Transparent Huge Pages

We can use the feature’s respective settings to disable transparent huge pages alone.

First, we stop THP functions at runtime:

$ echo never > /sys/kernel/mm/transparent_hugepage/enabled
$ echo never > /sys/kernel/mm/transparent_hugepage/defrag

To persist the settings, we can run the lines above on start-up or in a start-up script.

Although the process of disabling huge pages is relatively straightforward, some Linux distributions have introduced a libhugetlbfs-bin package with administration tools to ease the configuration.

In particular, we can disable transparent pages with hugeadm:

$ hugeadm --thp-never

Of course, to ensure THP is off on boot, we append transparent_hugepage=never as a kernel parameter.

5. Summary

In this article, we went over the settings of standard and transparent huge pages, as well as how to check their status and disable them completely.

In conclusion, although there are many statistics and parameters related to the functionality of huge pages, toggling them can be done in a couple of steps.

Comments are open for 30 days after publishing a post. For any issues past this date, use the Contact form on the site.