Physical Activity Recognition With Statistical-Deep Fusion Model Using Multiple Sensory Data for Smart Health

Abstract

Nowadays, enhancing the living standard with smart healthcare via the Internet of Things is one of the most critical goals of smart cities, in which artificial intelligence plays as the core technology. Many smart services, deployed according to wearable sensor-based physical activity recognition, have been able to early detect unhealthy daily behaviors and further medical risks. Numerous approaches have studied shallow handcrafted features coupled with traditional machine learning (ML) techniques, which find it difficult to model real-world activities. In this work, by revealing deep features from deep convolutional neural networks (DCNNs) in fusion with conventional handcrafted features, we learn an intermediate fusion framework of human activity recognition (HAR). According to transforming the raw signal value to pixel intensity value, segmentation data acquired from a multisensor system are encoded to an activity image for deep model learning. Formulated by several novel residual triple convolutional blocks, the proposed DCNN allows extracting multiscale spatiotemporal signal-level and sensor-level correlations simultaneously from the activity image. In the fusion model, the hybrid feature merged from the handcrafted and deep features is learned by a multiclass support vector machine (SVM) classifier. Based on several experiments of performance evaluation, our fusion approach for activity recognition has achieved the accuracy over 96.0% on three public benchmark data sets, including Daily and Sport Activities, Daily Life Activities, and RealWorld. Furthermore, the method outperforms several state-of-the-art HAR approaches and demonstrates the superiority of the proposed intermediate fusion model in multisensor systems.