Hardware-Efficient Activation Approximation based on Error-Sensitivity Analysis for Deep Neural Networks

Abstract

Implementing hardware units for non-linear activation functions is challenging due to their distinct characteristics. While approximation methods offer a promising solution, conventional approaches such as CORDIC and Chebyshev polynomials suffer from high latency, and piecewise linear (PWL) methods require large lookup tables (LUTs) to achieve acceptable accuracy. Furthermore, existing methods often overlook the varying impact of approximation errors across input regions. This paper proposes a hardware-efficient approximation method based on symmetric PWL approximation with error compensation guided by error-sensitivity analysis. A symmetry-aware base function is first constructed using PWL approximation. Then, only the difference between this base function and the target function is selectively compensated in high error-sensitivity regions using a lightweight error compensation module. This selective compensation enables accurate approximation across various non-linear functions using significantly fewer LUTs. Synthesized with Synopsys Design Compiler and UMC 28nm libraries, the proposed design achieved over 90% LUT area reduction per function compared to uniform PWL, with only 1.22%p average accuracy loss. Experimental results confirm that the method delivers scalable and flexible activation function computation for resource-constrained hardware.