Development of machine learning based natural language processing system

Abstract

For disease diagnostic knowledge base system (including Q&A, consistency checker, and informativity checker) requiring higher degree of strictness on the results of a query to the system, at the development stage, great efforts are necessary to improve machine learning based statistical performance tests, measured by precision and recall rates, on the training error and prediction. Performance of the test runs is generally dependent on the two basic factors as the following: first of all, the underlying technique of in-depth context analysis on corpora with inference capability, secondly, that of user's query sentence analysis with inference capability. More importantly, a disease diagnostic knowledge base system should be able to update effectively the latest research achievements in timely manner. To meet the requirements, we propose an automatic system for construction of knowledge base from the academic archive of medical literatures. For the purpose of presentation in this paper, a prototype of knowledge base construction using natural language processing system for early diagnosis of Alzheimer disease has been designed and implemented. Since there are plenty of knowledge base systems available for Alzheimer diagnosis in English language, to differentiate our works with the existing data, we performed our research with the literatures written in Korean. The natural language processing system proposed in this paper consists of 8 modules most of which are machine learning trainer/prediction model based on maximum entropy algorithm. Tests showed that, for all the modules, iterative training have been succeeded with precision over 90%.