Transformer-based foundation models for assessing earthquake- and vehicle-induced damage in bridges

Abstract

This study investigates the transformer-based foundation model, tabular prior-data fitted network (TabPFN), as a viable alternative to traditional boosting methods (extreme gradient boosting, light gradient boosting, and categorical boosting) in structural engineering. Boosting methods have demonstrated strong predictive performance for assessing the structural response of structures subjected to external loadings; however, they rely heavily on dataset-specific hyperparameter tuning and often lack transferability. In this work, TabPFN is applied to two representative tasks: (1) regression problem for the maximum curvature ductility estimation of bridge columns under earthquake loading, and (2) classification problem for the damage state identification of bridge columns under vehicle impact. Model performance is compared with boosting baselines through hold-out testing, 10-fold cross-validation, and statistical analysis on the statistical tests for algorithms comparison platform. The results demonstrate that TabPFN consistently matches or outperforms tuned boosting models across multiple error measures, while requiring no dataset-specific tuning and significantly reducing computational efforts. These findings highlight the potential of TabPFN as a robust and efficient predictive tool for structural engineering applications, particularly under conditions of limited data availability.