1. Introduction
In this tutorial, we’ll investigate cold and warm caches. Our objective is to understand the main differences and potential consequences of a system employing a cold or warm cache memory, defining the pros and cons.
At first, we’ll have a brief review on caching, remembering its purposes and most common benefits. So, we’ll specifically study the categories of cold and warm caches, highlighting their most relevant characteristics. Then, we’ll see some techniques for warming up caches. Finally, we’ll systematically compare both the studied cache categories.
2. Computer Memory and Caching
Due to the characteristics of how current computers process data, accessing memory is a necessary but typically costly operation.
Of course, some memories are faster than others. For example, accessing the main memory (Random Access Memory) is faster than accessing secondary memories, such as magnetic disks.
However, we also have heterogeneous performances among primary memory components, as well as among the secondary ones. As a general rule, the better the memory accessing performance, the higher the price, and the most complicated is to integrate them into the computer systems.
So, to tackle this scenario, we have a memory hierarchy in computer systems. From the costly and fastest memory to the cheaper and slowest one, we have registers, cache, other primary memories (Random Access Memory, Read-Only Memory), solid-state memory, and several other secondary memories (magnetic disk, magnetic tape).
In particular, caches are fast memories allocated very close to the computer system’s processors. Due to architectural limitations and price, caches are commonly smaller than other primary memories.
Caches, however, are tailored to keep data that is recurrently used by the processor or probably will be requested soon. So, the caching process consists of identifying such data to load it into the cache memory.
There are strategies to select data to load or replace in the cache memory during the execution of a computer system. However, it is a fact that when a computer system startups, the cache is clean or with irrelevant data. Thus, as we use the system, it gets populated with useful data and speeds up memory access.
This process of loading data differentiates a cold from a warm cache, as we’ll see next.
3. Cold and Warm Caches
As we previously discussed, caches are fast memories allocated near the processors of a computer system. But, caches are typically much smaller than other primary memories. So, we must carefully decide which data to allocate inside it.
Initializing a cache is a great challenge in this context: should we only accept that it is empty in the initial state, or should we try to populate the cache with potentially requestable data?
There is no definite answer to the previous question. Keeping the cache empty will result in multiple cache misses (querying a cache that does not have the required data). However, trying to infer data requests and populate the cache a priori has a cost. This cost only compensates if we have correct inferences.
The decision of how initializing (or not) cache memories, in turn, is what defines their categories of cold and warm:
- Cold Caches: a cache in a cold state is an empty cache or populated with stale data. It means that the cache memory will not speed up the memory accessing until it gets, progressively and naturally, charged with frequently demanded data
- Warm Caches: a warmed-up cache contains valuable data considering the current state of a computer system. It means that we expect lots of cache hits from it (finding the required data in the cache memory)
The image next depicts, at a high level, some usual events related to cold and warm caches:
It is relevant to highlight that cold and warm caches apply to other cache categories besides the computer memory ones, such as content caches in content delivery networks.
Although warm caches are desirable, it is not easy to predict the data requested by the computer systems. We’ll study some popular warming-up processes for cache memories in the following section.
4. Cache Warming
The cache warming process fills a cold cache with high request probability data. There exist multiple cache warming techniques. So, we have warming techniques particularly tailored for computer system caches and others for different categories of caches (such as the content ones).
Since heterogeneous scenarios have different requirements, it is hard to determine core techniques of cache warming. But, concentrating on the computer system’s cache memories, we can cite some traditional warming techniques:
- Prefetching metadata: the computer system fills a cold cache with metadata usually used in the early-stage execution of the system
- Prefetching metadata and data: if some specific data has known high-probability usage, the computer system may load it to the cache together with metadata
- Warmed-up cache copying: for multi-cache schemes, the computer system can copy data from an already warmed-up cache to fill a cold one
Of course, there exist multiple advantages to warming up a cache. Among them, we can cite the fast response to processor requests, which impacts the overall computer system performance.
Moreover, if we consider content caches of content delivery networks, warming up a content cache will make the clients get the desired resources faster, increasing their satisfaction and consequently improving User Experience (UX) indicators.
However, we should employ cache warming with caution. Well-done warming processes bring lots of benefits to computer systems, but they do have a cost anyway.
So, warming up a cache with data that the computer system will not need represents the execution of useless operations and the creation of a more complex scenario for data replacement in the cache memory. In such cases, warming up a cache is not beneficial for the systems.
5. Systematic Summary
Memory accessing represents a bottleneck for computer systems in some scenarios. Fast memories locate as near as possible to the processors, use dedicated buses, and typically are expensive.
One of the fastest memory types of a computer system is caches. Caches are small, and we employ them to keep data recurrently used by the processor. We can also infer which data will be required soon by the processor and load it into the cache.
A possible categorization of cache memories regards its state considering the requested data (or initial state): cold and warm caches.
In such a scenario, cold caches represent empty memory or memory filled with deprecated data (no longer requested by the processor). Cold caches generate lots of caches misses until getting updated.
On the other hand, warm caches employ some strategies to initialize and always keep data updated. So, we expect to get lots of cache hits with warmed caches, thus improving the overall performance of a computer system.
The following table summarizes some characteristics of cold and warm caches:
6. Conclusion
In this tutorial, we studied cold and warm caches. First, we had a brief review of computer system memories, specifically discussing cache memories. Thus, we examined the categories of cold and warm caches. We then explored some popular cache warming algorithms. At last, we compared cold and warm cache memories in a systematic summary.
We can conclude that caches are crucial pieces of memory in a computer system. But, cold caches do not represent many performance improvements in memory access. So, warming up processes are, in the vast majority of scenarios, necessary to make better use of this memory.