SwahBERT: Language Model of Swahili
- Authors
- Martin, Gati L.; Mswahili, Medard E.; Jeong, Young-Seob; Woo, Jiyoung
- Issue Date
- Nov-2022
- Publisher
- ASSOC COMPUTATIONAL LINGUISTICS-ACL
- Citation
- NAACL 2022: THE 2022 CONFERENCE OF THE NORTH AMERICAN CHAPTER OF THE ASSOCIATION FOR COMPUTATIONAL LINGUISTICS: HUMAN LANGUAGE TECHNOLOGIES, pp.314 - +
- Journal Title
- NAACL 2022: THE 2022 CONFERENCE OF THE NORTH AMERICAN CHAPTER OF THE ASSOCIATION FOR COMPUTATIONAL LINGUISTICS: HUMAN LANGUAGE TECHNOLOGIES
- Start Page
- 314
- End Page
- +
- URI
- https://scholarworks.bwise.kr/sch/handle/2021.sw.sch/21858
- Abstract
- The rapid development of social networks, electronic commerce, mobile Internet, and other technologies has influenced the growth of Web data. Social media and Internet forums are valuable sources of citizens' opinions, which can be analyzed for community development and user behavior analysis. Unfortunately, the scarcity of resources (i.e., datasets or language models) has become a barrier to the development of natural language processing applications in low-resource languages. Thanks to the recent growth of online forums and news platforms of Swahili, we introduce two datasets of Swahili in this paper: a pre-training dataset of approximately 105MB with 16M words and an annotated dataset of 13K instances for the emotion classification task. The emotion classification dataset is manually annotated by two native Swahili speakers. We pre-trained a new monolingual language model for Swahili, namely SwahBERT, using our collected pre-training data, and tested it with four downstream tasks including emotion classification. We found that SwahBERT outperforms multilingual BERT, a well-known existing language model, in almost all downstream tasks.
- Files in This Item
- There are no files associated with this item.
- Appears in
Collections - SCH Media Labs > Department of Big Data Engineering > 1. Journal Articles
![qrcode](https://api.qrserver.com/v1/create-qr-code/?size=55x55&data=https://scholarworks.bwise.kr/sch/handle/2021.sw.sch/21858)
Items in ScholarWorks are protected by copyright, with all rights reserved, unless otherwise indicated.