Projected variable three-term conjugate gradient algorithm for enhancing generalization performance in deep neural network training

Abstract

Deep learning optimization faces a fundamental trade-off between convergence efficiency and generalization. First-order methods such as stochastic gradient descent (SGD) and adaptive moment estimation (Adam) tend to find flatter minima but converge slowly, while higher-order methods converge rapidly but are often drawn to sharp minima that generalize poorly. To address this, we introduce the projected variable three-term conjugate gradient (PVTTCG) algorithm. Motivated by the geometric instabilities in modern networks that use techniques such as batch normalization (BN), PVTTCG integrates an orthogonal projection into the higher-order optimization framework. This mechanism eliminates radial components from the search direction, inherently guiding the optimization toward flatter regions without requiring additional regularization terms or hyperparameters. The effectiveness of PVTTCG is validated across diverse tasks, including language modeling, large-scale image classification, and a real-world engineering application. In complex scenarios, PVTTCG consistently improves upon its higher-order baseline, achieving up to a 3.92 percentage point gain on CIFAR-100 while remaining competitive with leading first-order methods. A systematic analysis reveals that PVTTCG demonstrates superior robustness to batch size variations, particularly excelling at larger batch sizes. This robustness enables the algorithm to process batch sizes up to 2,048 in engineering applications, achieving a 35.9% test loss reduction compared to Adam. These findings establish PVTTCG as an effective solution for bridging the convergence-generalization trade-off.