Bridging the Language Gap: Domain-Specific Dataset Construction for Medical LLMs

Abstract

The advent of large language models (LLMs) has transformed the field of natural language processing (NLP), demonstrating impressive capabilities across a variety of tasks such as text generation, translation, and question answering. However, their effectiveness in specialized domains is constrained by the lack of domain-specific data. This paper presents an effective methodology for constructing domain-specific datasets using domain-specific corpora, thus overcoming the challenges posed by linguistic and cultural differences in non-English speaking regions. By leveraging mining techniques, this methodology facilitates the construction of datasets tailored to local languages and cultures. A Korean medical corpus served as the foundation for dataset construction, leading to the development of a medical language model that demonstrated high performance and versatility across various NLP tasks. A bidirectional encoder representation from transformer-based comparative analysis revealed comparable performance. The objective is to streamline LLM applications across diverse domains, thereby enhancing language model efficiency. In the future, our efforts will be directed towards implementing the proposed methodology across diverse domains and investigating strategies for extracting domain-specific tasks and vocabulary to enhance the quality of domain datasets.