Selective DRAM cache bypassing for improving bandwidth on DRAM/NVM hybrid main memory systems

Abstract

Satisfying a demand for higher memory capacity is a major problem for computing systems. Conventional solutions are reaching those limits; instead, DRAM/NVM hybrid main memory systems which consist of emerging Non-Volatile Memory for large capacity and DRAM last-level cache for high access speed were proposed for further improvement. However, in these systems, the two device types share limited memory channels/ranks and NVM channels/ranks are often less utilized than DRAM ones. This paper proposes an OBYST (On hit BYpass to STeal bandwidth) technique to improve memory bandwidth by selectively sending read requests that hit on DRAM cache to NVM instead of busy DRAM. We also propose an inter-device request scheduling policy optimized to OBYST. With negligible area overhead, OBYST improves bandwidth, IPC, and EDP by up to 22%, 21%, and 26% over the baseline without bandwidth optimizations, respectively.