Real-Time Human Action Prediction via Pose Kinematics

Abstract

Recognizing human actions in real time poses a fundamental challenge especially when those actions coordinate with humans or objects in a shared space. Such systems must be capable of recognizing and assessing real-world human actions from different angles and viewpoints. Therefore, a significant volume of multi-dimensional human action training data is needed to enable data-driven algorithms to operate effectively in real-world scenarios. This paper introduces the Action Clip dataset, which offers a 360o view of human action with rich features from multiple angles, and describes the design and implementation of Human Action Prediction via Pose Kinematics (HAPtics), a comprehensive pipeline for real-time human pose estimation and action recognition, achievable with standard monocular camera sensors. HAPtics transforms initially noisy human pose kinematic structures into skeletal features using convolutional layers, such as body velocity, joint velocity, joint angles, and limb length from joint positions, fed into a classification layer for action recognition. Results on our proposed Action Clip dataset, NW-UCLA, and NTU RGB+D 120, demonstrate competitive state-of-the-art performance in pose-based action recognition and real-time performance running at 30 frames per second on a live camera. Code and dataset are made available to the public at: https://github.com/RaiseLab/HAPtics. © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2025.