A Deep Learning-Based Image Captioning for Automated Description of Structural Components Condition

Abstract

Along with visual data, textual information on civil engineering projects can provide a rich source of expert experiences and technical knowledge for diagnosing structural damage causes and their countermeasures. By implementing a cutting-edge deep learning approach in SHM (Structural Health Monitoring), the visual assessment, along with the interrelation of structural components and their working condition, can be retrieved to provide efficient damage evaluation and generate professional description documentation. The study proposes a methodology of an image captioning architecture of convolutional neural networks (CNNs) and recurrent neural networks (RNNs) for generating condition assessments of structural components. The purpose of this study is to investigate the pragmatic implementation of image captioning technology in structural health monitoring scenarios, improving the quality of inspection, and addressing the labor shortage of conventional maintenance. The results indicate that the proposed method can provide automated coherent text descriptions of structural components and their working conditions, simplify the inspection process, and deliver efficient maintenance management. © 2024, The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.