Reinforcement learning-driven adaptive 3D simulation and visualization of excavator operations

Abstract

Earthwork planning generally relies on expert experience and historical data to estimate operation cycle times. However, this conventional approach assumes that current working conditions resemble those of previous tasks, which is not always accurate. This paper presents a reinforcement learning-based simulation and visualization framework for robust motion planning and cycle time estimation of excavators in 3D virtual environments. A 3D agent was designed to incorporate the mechanical configuration and operational properties of actual excavators. The agent was then trained with the formulated rewards to generate realistic motions under specific working conditions. Experiments were conducted at five sites. These revealed an accuracy of 91.15 % for cycle-time estimation and a discrepancy 10 % smaller than the natural variations observed between trajectories of actual excavators for motion planning. This study can potentially contribute to earthwork planning by providing realistic cycle time estimation and simulation of excavation processes.