Sample-efficient reference-free control strategy for multi-legged locomotion

Abstract

Locomotion is one of the fundamental skills that is challenging to simulate in a manner that generalizes across a wide range of speeds and turning capabilities. In this paper, our goal is to develop a versatile locomotion controller applicable to various multi-legged character models (monopod, biped, and quadruped), enabling them to perform a range of tasks such as speed control, steering, moving to target locations, and slope walking. Our method is capable of generating diverse multi-legged locomotions without the need for reference motions, even when faced with the inherent challenge of coordinating multiple legs simultaneously. Based on deep reinforcement learning, we train our policy network to produce desired feet locations and orientations, enhancing sample efficiency and robustness compared to the commonly used joint angles. Utilizing end-effector configurations allows for intuitive adaptation to various locomotion gaits. Additionally, we design a style reward function that is applicable to different types of multi-legged models. The locomotion controller, trained with this reward, effectively performs given tasks in a physically simulated environment while maintaining the naturalness of locomotion.