자동화된 기계학습(AutoML)을 활용한 특허 특화 번역엔진의 영한번역 성능 평가

Abstract

This paper compares the quality of English-Korean patent translations by a patent-specific NMT engine trained using AutoML with the general Google Translate. The evaluation was based on both automatic and human evaluations of the Korean translations of 200 English patent sentences excerpted from a number of semiconductor patent gazettes. In automatic evaluation, BLEU scores showed that the patent-specific NMT engine significantly outperformed Google Translate. Human evaluation, carried out by sampling as well as error detection and correction analysis, confirmed the results of automatic evaluation, revealing that patent-specific NMT results were better than Google Translate results. In the error detection and correction analysis, Google Translate had more major errors than patent-specific NMT. Moreover, most errors in Google Translate were addressed in the patent-specific NMT, while errors in the patent-specific NMT still remained in Google Translate. In the sampling analysis, shorter sentences and longer sentences were sampled and analyzed. According to the results, both patent-specific NMT and Google Translate showed better performance in translating shorter sentences. In translating longer sentences, both translation engines exhibited accuracy-related errors and syntactic errors, though patent-specific NMT slightly outperformed Google Translate. Overall, translation results by patent-specific NMT showed better quality than those by Google Translate.