Synergistic in-domain and out-of-domain learning to strengthen visual scene understanding in data-scarce, imbalanced construction settings

Abstract

A scarcity of in-domain training images from target workplaces has hindered the broader adoption of visual scene understanding in construction. While out-of-domain images from non-target workplaces hold promising potential as supplementary training data, traditional approaches—put-it-all-together training and sequential finetuning—struggle with either over-generalization or catastrophic forgetting when both datasets are mixed due to distributional inconsistencies. To overcome this limitation, this article introduces a synergistic learning strategy that acquires domain-invariant visual knowledge from out-of-domain datasets while reinforcing in-domain predictive capabilities. Results show that the synergistic strategy consistently outperforms traditional approaches across various evaluation criteria, regardless of dataset size, imbalance, and distribution. Remarkably, the strategy achieves comparable or superior performance with merely half or one-twentieth of the in-domain dataset, showing robustness to diverse and challenging site conditions. Further studies further validate its effectiveness when tested on different object category and model architecture, as well as with an irrelevant out-of-domain dataset. These results strongly indicate that synergistic learning can effectively balance in-domain and out-of-domain visual knowledge, enhancing model’s performance and generalizability in diverse field conditions. These findings will be a solid foundation for advancing visual scene understanding in data-scarce, imbalanced industry settings including construction