ARNorm: Hardware-Efficient Normalization for Lightweight Edge Models

Abstract

We propose ARNorm, a hardware-friendly normalization algorithm designed for efficient inference in transformer-based models. ARNorm combines the structural simplicity of RMSNorm [1] with the hardware-optimized techniques of AILayerNorm, introduced in SOLE [2] achieving accurate normalization using only 8-bit integer precision (INT8) arithmetic. By employing dynamic compression and a priority encoder-based Look-Up Table (LUT) for root approximation, ARNorm eliminates costly floating-point operations such as mean, variance, and square root calculations. Experiments on six pre-trained Vision Transformer models demonstrate that ARNorm reduces quantization error by up to 10% compared to AILayerNorm and maintains accuracy comparable to 32-bit floating point precision(FP32)-based RMSNorm, making it highly suitable for edge and embedded AI applications.