Less is More: An Empirical Study of Undersampling Techniques for Technical Debt Prediction

Abstract

Technical Debt (TD) prediction is crucial to preventing software quality degradation and maintenance cost increase. Recent Machine Learning (ML) approaches have shown promising results in TD prediction, but the imbalanced TD datasets can have a negative impact on ML model performance. Although previous TD studies have investigated various oversampling techniques that generates minority class instances to mitigate the imbalance, potentials of undersampling techniques have not yet been thoroughly explored due to the concerns about information loss. To address this gap, we investigate the impact of undersampling on TD model performance by utilizing 17,797 classes from 25 Java open-source projects. We compare the performance of the models with different undersampling techniques and evaluate the impact of combining them with widely-used oversampling techniques. Our findings reveal that (i) undersampling can significantly improve TD model performance compared to oversampling and no resampling; (ii) the combined application of undersampling and oversampling techniques leads to a synergy of further performance improvement compared to applying each technique exclusively. Based on these results, we recommend practitioners to explore various undersampling techniques and their combinations with oversampling techniques for more effective TD prediction. © The Author(s), under exclusive license to Springer Nature Switzerland AG 2025.