Multimodal sensor fusion models for real-time exercise repetition counting with IMU sensors and respiration data
- Authors
- Lee, Sujee; Lim, Yooseok; Lim, Kyuhee
- Issue Date
- Apr-2024
- Publisher
- ELSEVIER
- Keywords
- Sensor fusion; Repetition counting; Human activity recognition; Multimodal deep learning; Mel spectrogram; IMU sensors
- Citation
- INFORMATION FUSION, v.104
- Journal Title
- INFORMATION FUSION
- Volume
- 104
- URI
- https://scholarworks.bwise.kr/ssu/handle/2018.sw.ssu/49101
- DOI
- 10.1016/j.inffus.2023.102153
- ISSN
- 1566-2535
1872-6305
- Abstract
- This study introduces an automated exercise repetition counting model for a variety of exercises. Although external devices such as fitness trackers and smart watches are capable of tracking exercise progress, they are still incapable of recording all pertinent information essential for comprehensive exercise tracking, such as the number of repetitions.To address this limitation, we propose a deep learning model that utilizes multimodal sensor and respiration audio data obtained from wearable devices, specifically smart earbuds that record IMU sensors and collect surrounding sounds. A diverse dataset of sensor data and respiratory audio data are collected by leveraging these smart earbuds during 30 different types of exercises, which enables the study to develop a comprehensive and accurate multimodal deep learning model that can be applied to a wide range of physical activities. Advanced techniques such as data augmentation and time-series interpolation are also employed to address the challenges of working with large and complex datasets.The proposed deep learning model, which incorporates multimodal sensor data and advanced data processing techniques, can be utilized to promote more effective and personalized exercise plans, ultimately leading to improved health outcomes for individuals who seek to maintain an active and healthy lifestyle.
- Files in This Item
-
Go to Link
- Appears in
Collections - ETC > 1. Journal Articles
Items in ScholarWorks are protected by copyright, with all rights reserved, unless otherwise indicated.