Transferable Convolutional Neural Networks for IMU-based Motion Gesture Recognition in Human-Machine Interaction

Abstract

Hand gestures, a fundamental aspect of human non-verbal communication, are often leveraged in the domain of Human-Machine Interaction (HMI) to implement more user-friendly interfaces. In this study, we propose a Convolutional Neural Network (CNN) model designed for efficient motion gesture recognition, designed to be deployed on a smartwatch, using only one Inertial Measurement Unit (IMU) sensor worn on the wrist. By directly processing low-dimensional motion data on linear acceleration and angular velocity, our model demonstrates high performance using a simplified model structure. Furthermore, we explore the potential of applying a transfer learning approach to our CNN model for novel gesture classification problems. This method demonstrates that a well-trained CNN model's backbone network effectively extracts motion features necessary for the recognition of new gestures. Validation processes in scenarios with limited data-employing specific training-to-test ratios of 1:3, 1:7, and 1:19-allowed for a comparison of our model's performance against baseline models trained from scratch. Our approach initially achieves an accuracy rate of 99.48±0.25% in recognizing ten distinct motion gestures through the directly processing raw data on linear acceleration and angular velocity directly. Moreover, the transfer learning model outperformed the baseline model trained from scratch with 95.62±0.99%, 93.23±1.41%, 92.81±1.62% accuracy in learning four new gestures under data limitations, respectively. This study shows that the proposed model maintains high performance with lightweight structure, while also highlighting how transfer learning approach can address the challenges of data collection and set the stage for creating more intuitive and user-centric interaction systems. © 2024 ICROS.