High-Precision Softmax Division without Multipliers or Look-Up Tables

Abstract

Softmax plays a critical role in deep learning accelerators, but its hardware implementations often suffer from precision loss due to inaccurate division operations. This paper focuses on the design of a high-precision division unit for Softmax, introducing a three-segment piecewise approximation of 1{{1 + s}} that effectively reduces approximation errors. The proposed approach improves numerical accuracy over previous designs, while preserving hardware efficiency and maintaining a multiplication-free and look-up table-free architecture.