A CPU cache is a hardware cache which helps reduce the cost from main memory access. It is a smaller and faster memory part which locates closer to a CPU core than the main memory. Most modern CPUs have mulitple level of caches like L1, L2, and L3. When trying to read and write the main memory, the processor will first check if the data already exists in its local cache.
The modern CPUs have at least three independent caches:
- Instruction cache - Used to speed executable instruction fetch
- Data cache - Used to speed data fetch and store
- Translation lookaside buffer (TLB) - Used to speed virtual-to-physical address translation for both executable instructions and data. The TLB cache is part of the memory management unit (MMU) but not directly related to the CPU cache
Let’s take an example to see how the CPU and cache looks like in a physical server.
In the following server, there are 48 CPU cores and each core processor has three levels of caches(L1, L2 and L3). L1 cache has both instruction cache and data cache.
[root@init531-e43 ~]# lscpu
The getconf command outputs the detail cache size for each level.
[root@init531-e43 ~]# getconf -a | grep CACHE
From the above output, we can see cacheline size for each cache level. It’s all 64 bytes in this example.
When the processor accesses a memory portion which is not already in the cache, it will load a chunk of the memory around the accessed address into the cache such that the cached data could be reused. The chunks of memory loaded by the cache are called cache lines. The size of chunk is called the cache line size. The common cache line sizes are 32, 64 and 128 bytes.
The limited number of cache lines can be held by the cache, which is determined by the cache size. In this example, the size of L1 data cache is 48k and it can hold 768 cache lines(49152/64).