CAMPuS: Concurrent Acceleration of Memory Access and Parallel Processing in Near-Memory SpMV Architecture

Abstract

Sparse matrix-vector multiplication (SpMV) is one of the key computations in many deep-learning networks. However, the performance to compute SpMV is often limited by the DRAM bandwidth. To resolve such DRAM bandwidth issues, several studies have proposed DIMM-based near-memory processing (NMP) architectures to accelerate SpMV computation. However, these studies face limitations in performance improvements owing to computational overheads in NMP cores, memory access overheads, or both. In this paper, we propose a novel method called CAMPuS to accelerate SpMV operations on a DIMM-based NMP platform. CAMPuS is designed to enhance computational parallelism, thereby boosting computational performance while maintaining high memory access efficiency. One of the key ideas is dual storage management: one for the input vector and the other for the hot-spot vector, which stores a frequently-accessed part of the input vector. The paper also presents a hot-spot-aware remapping technique for the sparse matrix to optimally use the resources of the hot-spot vector. The results of simulations conducted indicate that CAMPuS significantly outperforms one of the state-of-the-art works called Fafnir by up to 3.23× . We design a Verilog-based NMP Execution system for an end-to-end evaluation, and the end-to-end evaluation of CAMPuS with the NMP Execution system confirms as 4.49× improvement over a software SpMV implementation using the OpenMP libraries, demonstrating the effectiveness of CAMPuS in the real-world computer system.