Enhancing Retrieval-Augmented Generation Performance through Network Analysis of Question Types in Public Procurement Law

Abstract

Public procurement and construction contracts underpin national industries; however, their complex legal frameworks and the engagement of multiple stakeholders frequently produce interpretive difficulties and disputes. Small- and medium-sized specialty contractors are particularly disadvantaged by their limited access to legal information, sparking interest in generative AI systems for statutory interpretation and contract support. However, these systems pose risks of factual errors and legal misinterpretations due to hallucinations. To mitigate these limitations, retrieval-augmented generation (RAG)–based question answering has been investigated to evaluate the reliability and efficiency of AI outputs, yet systematic analyses of legal taxonomies and the effectiveness of different question types remain scarce. This study developed a classification of the interrelationships between statutes and contracts in public procurement and conducted literature and network analyses on the Republic of Korea’s Act on Contracts to Which the State is a Party and the Standard Construction Contract. The study adopts a six-part framework comprising (1) core values and definitions; (2) documents and forms; (3) stakeholders; (4) procurement and performance; (5) special clauses; and (6) dispute-related provisions. Using this framework, a GPT-4o–driven RAG system was implemented for question-answering experiments. The accuracy by question type was evaluated using the cosine similarity between the generated answers and reference responses. The results indicate higher accuracy for conceptual queries (e.g., definitions and principles), whereas items requiring procedural or composite interpretation perform comparatively worse, empirically demonstrating that AI answer quality is directly shaped by query structure and context design. By systematizing the structural linkages between statutes and contracts through a framework and empirically validating RAG-enabled legal Q&A performance with an emphasis on question types, this study outlines a pathway to knowledge-structure standardization that supports digital transformation in public procurement and mitigates legal risk in research and practice.