A semantic similarity measure in document databases: An Earth mover's distance-based approach
- Authors
- Jang, Min-Hee; Eom, Tae-Hwan; Kim, Sang-Wook; Hwang, Young-Sup
- Issue Date
- Oct-2013
- Publisher
- Association for Computing Machinary, Inc.
- Keywords
- document search; Earth Mover' s distance; latent dirichlet allocation
- Citation
- Proceedings of the 2013 Research in Adaptive and Convergent Systems, RACS 2013, pp.94 - 99
- Indexed
- SCOPUS
- Journal Title
- Proceedings of the 2013 Research in Adaptive and Convergent Systems, RACS 2013
- Start Page
- 94
- End Page
- 99
- URI
- https://scholarworks.bwise.kr/hanyang/handle/2021.sw.hanyang/161776
- DOI
- 10.1145/2513228.2513245
- ISSN
- 0000-0000
- Abstract
- Measuring document similarity is important in order to find documents which are similar to a given query document from a user. Text-based document similarity is measured by comparing the words in two documents. The representative text-based document similarity is the cosine similarity. Since the cosine similarity computes document similarity by estimating the frequency of common words, it cannot reflect word similarity. To solve this problem, we propose a new document similarity measure based on the earth mover's distance (EMD). The EMD is one of the most popular distance functions used to search similar multimedia contents and is known to provide good search results. To apply the EMD to compute document similarity, we have to solve two problems: (1) The EMD is too time consuming to be used in a document database, (2) the distance between words should be defined. Our proposed approach first extracts topics as new features of a document by applying the latent Dirichlet allocation, which is a generative model of a document. It can decrease the computational cost of the EMD because the number of topics is much smaller than the number of words in a document. After extracting the topics, the proposed approach calculates the distance between topics based on the relation between the topics and the words in a document database, thereby making computing document similarity based on the EMD possible. Our approach searches documents more accurately since we can consider the semantic similarity by using the EMD. Experimental results on a real-world document database indicate that the proposed approach outperforms the cosine similarity in terms of the accuracy and the performance. ? 2013 ACM.
- Files in This Item
-
Go to Link
- Appears in
Collections - 서울 공과대학 > 서울 컴퓨터소프트웨어학부 > 1. Journal Articles
![qrcode](https://api.qrserver.com/v1/create-qr-code/?size=55x55&data=https://scholarworks.bwise.kr/hanyang/handle/2021.sw.hanyang/161776)
Items in ScholarWorks are protected by copyright, with all rights reserved, unless otherwise indicated.