Bi-directional Maximal Matching Algorithm to Segment Khmer Words in Sentence

Abstract

In the Khmer writing system, the Khmer script is the official letter of Cambodia, written from left to rightwithout a space separator; it is complicated and requires more analysis studies. Without clear standardguidelines, a space separator in the Khmer language is used inconsistently and informally to separate words insentences. Therefore, a segmented method should be discussed with the combination of the future Khmernatural language processing (NLP) to define the appropriate rule for Khmer sentences. The critical process inNLP with the capability of extensive data language analysis necessitates applying in this scenario. One of theessential components in Khmer language processing is how to split the word into a series of sentences andcount the words used in the sentences. Currently, Microsoft Word cannot count Khmer words correctly. So,this study presents a systematic library to segment Khmer phrases using the bi-directional maximal matching(BiMM) method to address these problematic constraints. In the BiMM algorithm, the paper focuses on the Bidirectionalimplementation of forward maximal matching (FMM) and backward maximal matching (BMM) toimprove word segmentation accuracy. A digital or prefix tree of data structure algorithm, also known as a trie,enhances the segmentation accuracy procedure by finding the children of each word parent node. The accuracyof BiMM is higher than using FMM or BMM independently; moreover, the proposed approach improvesdictionary structures and reduces the number of errors. The result of this study can reduce the error by 8.57%compared to FMM and BFF algorithms with 94,807 Khmer words.