Hypert: hypernymy-aware BERT with Hearst pattern exploitation for hypernym discoveryopen access
- Authors
- Yun, Geonil; Lee, Yongjae; Moon, A-Seong; Lee, Jaesung
- Issue Date
- Sep-2023
- Publisher
- Springer Science and Business Media Deutschland GmbH
- Keywords
- Hearst pattern; Hypernym discovery; Hypernym relationship; Language model; Masked language modeling; Natural language processing
- Citation
- Journal of Big Data, v.10, no.1
- Journal Title
- Journal of Big Data
- Volume
- 10
- Number
- 1
- URI
- https://scholarworks.bwise.kr/cau/handle/2019.sw.cau/68250
- DOI
- 10.1186/s40537-023-00818-0
- ISSN
- 2196-1115
2196-1115
- Abstract
- Hypernym discovery is challenging because it aims to find suitable instances for a given hyponym from a predefined hypernym vocabulary. Existing hypernym discovery methods used supervised learning with word embedding from word2vec. However, word2vec embedding suffers from low embedding quality regarding unseen or rare noun phrases because entire noun phrases are embedded into a single vector. Recently, prompting methods have attempted to find hypernyms using pretrained language models with masked prompts. Although language models alleviate the problem of w embeddings, general-purpose language models are ineffective for capturing hypernym relationships. Considering the hypernym relationship to be a linguistic domain, we introduce Hypert, which is further pretrained using masked language modeling with Hearst pattern sentences. To the best of our knowledge, this is the first attempt in the hypernym relationship discovery field. We also present a fine-tuning strategy for training Hypert with special input prompts for the hypernym discovery task. The proposed method outperformed the comparison methods and achieved statistically significant results in three subtasks of hypernym discovery. Additionally, we demonstrate the effectiveness of the several proposed components through an in-depth analysis. The code is available at: https://github.com/Gun1Yun/Hypert . © 2023, Springer Nature Switzerland AG.
- Files in This Item
-
- Appears in
Collections - College of Software > Department of Artificial Intelligence > 1. Journal Articles
![qrcode](https://api.qrserver.com/v1/create-qr-code/?size=55x55&data=https://scholarworks.bwise.kr/cau/handle/2019.sw.cau/68250)
Items in ScholarWorks are protected by copyright, with all rights reserved, unless otherwise indicated.