To Guide or to Disturb-How to Teach Dexterous Skills Using AI?

Abstract

Learning a skill is a complex cognitive task that requires processing multiple sensorimotor information. To help the skill learners, in this study, we explored and compared how to exploit AI wisely to teach skills. We set an initial target skill of engraving implemented on a haptic-audio-visual (HAV) VR environment. As the haptic device, we used a desktop force-feedback device to render the movement and force profile of the tooltip.To build teaching AI for engraving, we gathered experts' motion and force profile data. Then, we designed an Long-Short Term Memory (LSTM)-based AI model that discriminates the user's behavior and status of the tooltip using the data. With the VR environment and the AI model, we compared and evaluated three teaching strategies for haptic dexterous skill transfer- Guidance, Disturbance, and Hybrid Assistance. Hybrid Assistance alters its force between Guidance and Disturbance based on the user's performance. We conducted a user experiment with seven sessions: one pre-test, three main training, one immediate retention, and two delayed retention sessions. In the results, we found: 1) Guidance showed a steep learning curve during the training sessions, but the participants lost the learning effect in the retention sessions, and 2) the learning with Hybrid Assistance was the slowest but remained longer, even showed a better performance in delayed retention tests. These results seem to follow a guidance hypothesis in learning, which suggests how to design the AI model to determine its policy to provide the best performance for the user's training. © 2025 Copyright held by the owner/author(s).