Linux Free, Available and Cached Memory and How to Fill Them

1. Overview

Linux memory usage policy is built around the ‘unused memory is wasted memory’ paradigm. Therefore, we should always expect that the memory is occupied to some extent. Two main areas of use of RAM are file caching and application memory. The kernel balances these uses to keep the system responsive.

In this tutorial, we’ll learn how to fill the memory in a controllable way.

2. Reading Memory Figures

We can find the current information about memory in the /proc/meminfo file. First, let’s read the MemTotal entry to find the memory size of a 4 GB RAM virtual machine:

$ grep MemTotal /proc/meminfo
MemTotal:        3995876 kB

Next, let’s take a look at three figures which we’ll use throughout this article. They are free, available, and cached memory:

$ grep 'MemFree\|MemAvailable\|^Cached' /proc/meminfo
MemFree:          120848 kB
MemAvailable:    2135812 kB
Cached:          2155388 kB

Note that modern Linux systems distinguish between available and free memory. The available memory may be in use at the very moment, but the system can reclaim it when needed. Free memory means exactly what it is – the not-used one. Finally, cached is the size of file pages mapped into memory.

3. Filling Memory With File

The simplest way to surge memory consumption is to perform an operation on a file. For example, it’s enough to print a file in the terminal with cat.

First, let’s check the memory amounts:

$ grep 'MemFree\|MemAvailable\|^Cached' /proc/meminfo
MemFree:         2621572 kB
MemAvailable:    2860908 kB
Cached:           341752 kB

Then, we cat into the void a big, around 2 GB file test.txt:

$ cat test.txt > /dev/null

Subsequently, let’s take a look at the memory usage:

$ grep 'MemFree\|MemAvailable\|^Cached' /proc/meminfo
MemFree:          617852 kB
MemAvailable:    2849036 kB
Cached:          2333600 kB

So, we discovered that the amount of cached memory increased at the cost of the free memory. However, we still have almost the same amount of available memory, as the kernel can evict cached pages easily.

4. Locking Memory Pages With vmtouch

We can use vmtouch to gain more control over the caching process. The command allows us to cache files, read the number of cached pages per file, or remove file pages from memory.

Here, we’re interested in locking memory pages with the -l option. As long as the process that holds the lock is alive, the kernel can not evict such pages. However, we need a sudoer privilege to do that:

$ sudo vmtouch -l test.txt
LOCKED 497406 pages (1G)

Afterward, let’s check the memory figures:

$ grep 'MemFree\|MemAvailable\|^Cached' /proc/meminfo
MemFree:          730664 kB
MemAvailable:    1041264 kB
Cached:          2490584 kB

We see that locking the cache pages decreases not only the free but also the available memory of the system.

5. Using stress-ng

The stress-ng utility provides a vast suite of stress tests to check various aspects of the computer and system architecture. With stress-ng, we can put a controllable pressure on memory, disk, CPU, or I/O, to name a few.

First, we need to install the program. On Ubuntu, let’s use apt:

$ sudo apt install stress-ng

stress-ng demands the specification of at least one test case and a number of processes to be spawned. So let’s run two CPU stressors with the –cpu option:

$ stress-ng --cpu 2 --timeout 60s
stress-ng: info:  [7496] setting to a 1 min, 0 secs run per stressor
stress-ng: info:  [7496] dispatching hogs: 2 cpu
stress-ng: info:  [7496] passed: 2: cpu (2)
stress-ng: info:  [7496] failed: 0
stress-ng: info:  [7496] skipped: 0
stress-ng: info:  [7496] metrics untrustworthy: 0
stress-ng: info:  [7496] successful run completed in 1 min, 0.03 secs

Then, stress-ng reports two stressors, known as hogs. The test run lasted for one minute, as indicated by the –timeout option.

5.1. Stressing the Virtual Memory

With the –vm-bytes option, we put memory pressure. It takes an absolute amount of memory, in bytes, KBytes, MBytes, and GBytes, or percent of available memory. Then, let’s ask stress-ng to consume 2 GB:

$ stress-ng --vm 1 --vm-bytes 2G --timeout 120s

Next, let’s target a given percent of available memory, e.g., 45 %:

$ stress-ng --vm 1 --vm-bytes 45% --timeout 120s

Since stress-ng alternately takes and releases memory, we can observe (e.g., on top) that the memory consumption of related processes varies. In this case, the –vm-bytes value is an upper limit of memory occupation. We should add the –vm-keep option to fill the memory permanently:

$ stress-ng --vm 1 --vm-bytes 2G --vm-keep --timeout 120s

As a result, the available memory is reduced:

$ grep 'MemFree\|MemAvailable\|^Cached' /proc/meminfo
MemFree:          666604 kB
MemAvailable:     950588 kB
Cached:           474544 kB

5.2. More Hogs

So far, we’ve used one process to consume the memory, but we can also start more hogs. In such a case, we should keep in mind that stress-ng regards –vm-bytes as the overall memory usage of all processes. For example, let’s send 10 hogs to eat 3 GB of memory:

$ stress-ng --vm 10 --vm-bytes 3G --vm-keep --timeout 120s

6. Using Default Tools

Instead of installing utilities such as vmtouch or stress-ng, let’s use tools we can find in every distribution. The well-known tail command stores the input in memory until it encounters the newline character. It’s necessary if the current line proves to be the last one.

All we need to do is provide tail with one line long enough to fill a given amount of memory. A promising producer of such a line is a special file /dev/zero. It can provide as many null characters as we want to read. Therefore, let’s exhaust all available memory with:

$ tail /dev/zero

If we want to have more control over memory filling, we can limit the tail input with head -c, which outputs a given number of bytes. So let’s fill 2 GB of memory:

$ head -c 2G /dev/zero | tail

6.1. Keeping the Memory Busy

We can easily notice the drawback of this method – the memory is released immediately after tail is done. To approximately set the time of memory occupation, we need to use cat:

$ cat <(head -c 2G /dev/zero) <(sleep 120) | tail

We use twice the process substitution <() construct here. It allows the output of one command, enclosed in parenthesis, to be an input of another one. Basically, we applied the equivalent of the cat file1 file2 syntax. This way, cat redirects the head output to tail first. Afterward, cat waits for the sleep command to complete, keeping the tail’s process active.

During the occupation period, we can learn that the available memory is affected:

$ grep 'MemFree\|MemAvailable\|^Cached' /proc/meminfo
MemFree:          645076 kB
MemAvailable:     939544 kB
Cached:           484816 kB

6.2. Another Endless Source

In the case when the /dev/zero file is not available in our system, we can use the yes command as the infinite source of data. The command returns an endless sequence of the ‘y’ character followed by the new line mark. All we need to do is strip the newlines with tr:

$ yes | tr -d '\n' | head -c 2G | tail > /dev/null

6.3. Throttling Down the Data Transfer

We can use the pv command to monitor and control the data transfer. We should insert pv as a part of the pipeline to do this.

As the command isn’t a part of the default installation, we need to install it by hand:

$ sudo apt install pv

Then, let’s show the data transfer and the progress bar :

$ head -c 1G /dev/zero | pv | tail
45MiB 0:00:04 [ 224MiB/s] [   <=>                                            ]

Next, we’re going to limit the data transfer to 100 mb/s with the -L option:

$ head -c 1G /dev/zero | pv -L 100m | tail
1.00GiB 0:00:10 [ 100MiB/s] [          <=>                                     ]

7. Conclusion

In this article, we learned how to fill the memory in the Linux system. First, we looked at the figures describing the memory consumption. Then, we populate the cached memory by performing operations on files. We found this way particularly effective if we could lock the pages with vmtouch.

Next, we engaged stress-ng, a command intended to load the system. Finally, we applied tail, a command available in every Linux box. Thanks to its design, we could turn it into an excellent tool to fill the memory.