A bypass first policy for energy-efficient last level caches

Abstract

The last level cache (LLC) is critical for mobile computer systems in terms of both energy consumption and performance because it takes a large portion of the chip area and misses often cause expensive stalls. Prior works have studied the importance of bypassing the LLC, and focused on improving LLC performance. However, they did not fully exploit the opportunity for reducing energy consumption because they all employ a Cache First Policy (CFP). In CFPs, blocks are initially cached to monitor their re-reference behavior to make bypass decisions. As a result, CFPs tend to populate the LLC with useless blocks, and consume extra energy for unnecessary writes. In this paper, we take the opposite approach and propose a Bypass First Policy (BFP), where cache blocks are bypassed by default and only inserted if they are expected to be reused. A BFP can save significant energy by reducing the number of never-rereferenced cache blocks written to the LLC. Evaluations show that BFP reduces energy consumption by 57.1% across SPEC CPU2006 and 21.7% across MediaBench benchmark suites on average. Furthermore, BFP achieves a geometric mean speedup of 18.3% for LLC-intensive benchmarks with less than 8kB of extra storage, which is better or comparable to state-of-the-art CFPs while consuming similar or less storage overhead.