Construction regulatory document digitalization with layout knowledge-informed object detection and semantic text recognition

Abstract

Construction documents, containing extensive project information, are often stored and shared in unstructured paper formats, leading to inefficiencies in retrieval and transfer among stakeholders. There has been a pressing need for digitalizing construction documents by converting Portable Document Format documents into machine-readable, structured texts. However, current optical character recognition technologies struggle with complex layouts commonly found in construction project documents. To address this issue, we propose a construction document digitalization approach integrated with layout knowledge-informed object detection and semantic text recognition, improving recognition accuracy across various layouts and preserving the structural integrity of texts. Results show that our approach can reduce the average word error rate by 5.6 %p with the assistance of layout knowledge and achieve a structural similarity of 78.8 %, while achieving 87.4 % mAP@50 for layout analysis. These findings highlight the positive impacts of layout knowledge on digitalizing construction documents and underscore the practical viability of our approach.