Multi-scale multi-resolution reinforcement learning topology optimization: application to elastic-wave metamaterial design

Abstract

Despite the growing potential of reinforcement learning (RL) for solving optimization problems, its application to topology optimization has been hindered by the high computational cost associated with inefficient exploration of the design space, often requiring a large number of episodes. To overcome this limitation, we propose a novel multi-scale multi-resolution reinforcement learning (MMRL) algorithm tailored for topology optimization. Unlike conventional single-scale single-resolution formulations that treat element-wise density values as independent variables, the proposed method transforms them into binary wavelet-based multi-scale variables, thereby enabling simultaneous multi-scale and multi-resolution optimization. Consequently, RL actions are defined in a multi-scale wavelet action space, allowing the agent to explore the design domain more effectively and to converge with a significantly reduced number of episodes. The proposed MMRL framework was applied to the topology optimization of elastic-wave metamaterials designed for perfect mode conversion and full barrier-through transmission. Comparative studies with single-scale single-resolution, single-scale multi-resolution, and multi-scale single-resolution formulations demonstrate the superior efficiency and solution quality of the proposed method. The results highlight its capability to achieve the desired topologic layouts with drastically fewer episodes while maintaining high design performance.