Adaptive reinforcement learning for energy-efficient high-recovery closed-circuit reverse osmosis

Abstract

Closed-circuit reverse osmosis (CCRO) achieves high recovery; however, its semi-batch purge-and-refill cycles complicate control and optimization. Unlike static rule-based operation, reinforcement learning is increasingly used for operational optimization to adaptively select real-time control setpoints under changing conditions. In this study, a data-calibrated dynamic CCRO simulator was integrated with a reinforcement learning control framework and evaluated under realistic plant-variable conditions. Plant behavior was stably reproduced by the simulator, achieving low RMSE across permeate flow, circulated concentrate flow, membrane inlet pressure, and permeate concentration. A proximal policy optimization agent was trained across 24 environmental settings with over 10 million steps, and the best-performing policy was identified through evaluation of 375 predefined scenarios and nonparametric statistical analyses. Across the evaluation scenarios, the resulting agent achieved a mean specific energy consumption (SEC) of 0.489 kWh/m³ and a mean recovery rate of 95.5%, outperforming a static rule-based controller by 13.14% in SEC and 3.92% in water recovery through adaptive control. Interpretability and feasibility were further supported by explainable AI and edge execution-time analyses on representative hardware. Overall, the proposed framework provides a promising alternative to conventional rule-based CCRO operation in small-scale decentralized plants where continuous expert supervision is impractical.