Robust Human Activity Recognition by Integrating Image and Accelerometer Sensor Data Using Deep Fusion Network
- Authors
- Kang, Junhyuk; Shin, Jieun; Shin, Jaewon; Lee, Daeho; Choi, Ahyoung
- Issue Date
- Jan-2022
- Publisher
- MDPI
- Keywords
- Accelerometer sensors; Deep learning; Fusion network; Human activity recognition; Skeleton detection
- Citation
- Sensors, v.22, no.1
- Journal Title
- Sensors
- Volume
- 22
- Number
- 1
- URI
- https://scholarworks.bwise.kr/gachon/handle/2020.sw.gachon/83473
- DOI
- 10.3390/s22010174
- ISSN
- 1424-8220
- Abstract
- Studies on deep-learning-based behavioral pattern recognition have recently received considerable attention. However, if there are insufficient data and the activity to be identified is changed, a robust deep learning model cannot be created. This work contributes a generalized deep learning model that is robust to noise not dependent on input signals by extracting features through a deep learning model for each heterogeneous input signal that can maintain performance while minimizing preprocessing of the input signal. We propose a hybrid deep learning model that takes heterogeneous sensor data, an acceleration sensor, and an image as inputs. For accelerometer data, we use a convolutional neural network (CNN) and convolutional block attention module models (CBAM), and apply bidirectional long short-term memory and a residual neural network. The overall accuracy was 94.8% with a skeleton image and accelerometer data, and 93.1% with a skeleton image, coordinates, and accelerometer data after evaluating nine behaviors using the Berkeley Multimodal Human Action Database (MHAD). Furthermore, the accuracy of the investigation was revealed to be 93.4% with inverted images and 93.2% with white noise added to the accelerometer data. Testing with data that included inversion and noise data indicated that the suggested model was robust, with a performance deterioration of approximately 1%. © 2021 by the authors. Licensee MDPI, Basel, Switzerland.
- Files in This Item
- There are no files associated with this item.
- Appears in
Collections - IT융합대학 > 소프트웨어학과 > 1. Journal Articles
- 공과대학 > 기계공학과 > 1. Journal Articles
![qrcode](https://api.qrserver.com/v1/create-qr-code/?size=55x55&data=https://scholarworks.bwise.kr/gachon/handle/2020.sw.gachon/83473)
Items in ScholarWorks are protected by copyright, with all rights reserved, unless otherwise indicated.