Realizing physical AI through a proprioceptive wearable interface: Semantic understanding of gestures and objects

Abstract

Replicating the human hand's proprioceptive perception is a key challenge for Physical AI, hindered by complex hardware and inherent sensor variability. We introduce a new paradigm through a wearable interface with just ten low-cost strain sensors. Instead of correcting sensor variability, our 1D Convolutional Neural Network (1D-CNN) leverages it to achieve robust, human-like perception. The system accurately recognizes 26 sign language gestures (>98 %), 7 object shapes (80.9 %), and 6 discrete sizes (91.7 %). Demonstrating true generalization, it also predicts the size of a previously unseen 5.5 cm sphere as 5.54 ± 0.49 cm. This confirms the system’s ability to move beyond pattern recognition to semantic understanding. Our study provides a blueprint for intuitive and accessible Physical AI systems, proving that high-level perception unifying communication and manipulation can be achieved with minimal hardware.