A Deep-Learned Embedding Technique for Categorical Features Encodingopen access
- Authors
- Dahouda, Mwamba Kasongo; Joe, Inwhee
- Issue Date
- Aug-2021
- Publisher
- IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
- Keywords
- Encoding; Numerical models; Machine learning; Data models; Training; Biological neural networks; Computational modeling; Data preprocessing; categorical variables; natural language processing; machine learning
- Citation
- IEEE ACCESS, v.9, pp.114381 - 114391
- Indexed
- SCIE
SCOPUS
- Journal Title
- IEEE ACCESS
- Volume
- 9
- Start Page
- 114381
- End Page
- 114391
- URI
- https://scholarworks.bwise.kr/hanyang/handle/2021.sw.hanyang/141386
- DOI
- 10.1109/ACCESS.2021.3104357
- ISSN
- 2169-3536
- Abstract
- Many machine learning algorithms and almost all deep learning architectures are incapable of processing plain texts in their raw form. This means that their input to the algorithms must be numerical in order to solve classification or regression problems. Hence, it is necessary to encode these categorical variables into numerical values using encoding techniques. Categorical features are common and often of high cardinality. One-hot encoding in such circumstances leads to very high dimensional vector representations, raising memory and computability concerns for machine learning models. This paper proposes a deep-learned embedding technique for categorical features encoding on categorical datasets. Our technique is a distributed representation for categorical features where each category is mapped to a distinct vector, and the properties of the vector are learned while training a neural network. First, we create a data vocabulary that includes only categorical data, and then we use word tokenization to make each categorical data a single word. After that, feature learning is introduced to map all of the categorical data from the vocabulary to word vectors. Three different datasets provided by the University of California Irvine (UCI) are used for training. The experimental results show that the proposed deep-learned embedding technique for categorical data provides a higher F1 score of 89% than 71% of one-hot encoding, in the case of the Long short-term memory (LSTM) model. Moreover, the deep-learned embedding technique uses less memory and generates fewer features than one-hot encoding.
- Files in This Item
-
- Appears in
Collections - 서울 공과대학 > 서울 컴퓨터소프트웨어학부 > 1. Journal Articles
Items in ScholarWorks are protected by copyright, with all rights reserved, unless otherwise indicated.