Human-computer interaction using radio sensor for people with severe disability

Abstract

Around the world, many people live with total paralysis, which makes it difficult to communicate using speech, text or gestures. Moreover, people with severe disabilities cannot use communication devices including computers, tablets, or mobile phones given their inability to operate conventional input devices such as keyboard, mouse, and interfaces based on speech or gesture recognition. To help paralyzed people to use electronic devices, we propose a human-computer interaction system based on the recognition of breathing patterns acquired through impulse radio ultra-wideband (IR-UWB) sensors. In particular, commands are created through different signal patterns generated by the user's inhalation and exhalation. The signal emitted from the radio sensor is directed to the user's abdomen or chest, and the reflected signal can be analyzed to recognize the breathing patterns. We analyzed features such as the frequency and variance of breathing patterns and identified them using feature extraction and a deep neural network. Considering the target users, we experimentally determined breathing patterns that can be generated with convenience and comfort. Overall, we determined five elementary patterns, whose combination allows to represent several actions for the system to perform. Experiments using the proposed system verify its capabilities for both simple and complex human-computer interaction tasks such as alphabetic writing. Moreover, the breathing principle of the proposed system makes it suitable even for severely disabled people who struggle to use conventional gesture-based interfaces.