The Geometry of Feature Space in Deep Learning Models: A Holistic Perspective and Comprehensive Reviewopen access
- Authors
- Lee, Minhyeok
- Issue Date
- May-2023
- Publisher
- MDPI
- Keywords
- feature space geometry; deep learning models; manifold structures; disentangled representations
- Citation
- MATHEMATICS, v.11, no.10
- Journal Title
- MATHEMATICS
- Volume
- 11
- Number
- 10
- URI
- https://scholarworks.bwise.kr/cau/handle/2019.sw.cau/69887
- DOI
- 10.3390/math11102375
- ISSN
- 2227-7390
2227-7390
- Abstract
- As the field of deep learning experiences a meteoric rise, the urgency to decipher the complex geometric properties of feature spaces, which underlie the effectiveness of diverse learning algorithms and optimization techniques, has become paramount. In this scholarly review, a comprehensive, holistic outlook on the geometry of feature spaces in deep learning models is provided in order to thoroughly probe the interconnections between feature spaces and a multitude of influential factors such as activation functions, normalization methods, and model architectures. The exploration commences with an all-encompassing examination of deep learning models, followed by a rigorous dissection of feature space geometry, delving into manifold structures, curvature, wide neural networks and Gaussian processes, critical points and loss landscapes, singular value spectra, and adversarial robustness, among other notable topics. Moreover, transfer learning and disentangled representations in feature space are illuminated, accentuating the progress and challenges in these areas. In conclusion, the challenges and future research directions in the domain of feature space geometry are outlined, emphasizing the significance of comprehending overparameterized models, unsupervised and semi-supervised learning, interpretable feature space geometry, topological analysis, and multimodal and multi-task learning. Embracing a holistic perspective, this review aspires to serve as an exhaustive guide for researchers and practitioners alike, clarifying the intricacies of the geometry of feature spaces in deep learning models and mapping the trajectory for future advancements in this enigmatic and enthralling domain.
- Files in This Item
-
- Appears in
Collections - College of ICT Engineering > School of Electrical and Electronics Engineering > 1. Journal Articles
![qrcode](https://api.qrserver.com/v1/create-qr-code/?size=55x55&data=https://scholarworks.bwise.kr/cau/handle/2019.sw.cau/69887)
Items in ScholarWorks are protected by copyright, with all rights reserved, unless otherwise indicated.