Abstract
Modeling and controlling fluid flows is critical for several fields of science and engineering, including transportation, energy, and medicine. Effective flow control can lead to, e.g., lift increase, drag reduction, mixing enhancement, and noise reduction. However, controlling a fluid faces several significant challenges, including high-dimensional, nonlinear, and multiscale interactions in space and time. Reinforcement learning (RL) has recently shown great success in complex domains, such as robotics and protein folding, but its application to flow control is hindered by a lack of standardized benchmark platforms and the computational demands of fluid simulations. To address these challenges, we introduce HydroGym, a solver-independent RL platform for flow control research. HydroGym integrates sophisticated flow control benchmarks, scalable runtime infrastructure, and state-of-the-art RL algorithms. Our platform includes 42 validated environments spanning from canonical laminar flows to complex three-dimensional turbulent scenarios, validated over a wide range of Reynolds numbers. We provide non-differentiable solvers for traditional RL and differentiable solvers that dramatically improve sample efficiency through gradient-enhanced optimization. Comprehensive evaluation reveals that RL agents consistently discover robust control principles across configurations, such as boundary layer manipulation, acoustic feedback disruption, and wake reorganization. Transfer learning studies demonstrate that controllers learned at one Reynolds number or geometry adapt efficiently to new conditions, requiring approximately 50% fewer training episodes. The HydroGym platform is highly extensible and scalable, providing a framework for researchers in fluid dynamics, machine learning, and control to add environments, surrogate models, and control algorithms to advance science and technology.
Abstract (translated)
流体流动的建模和控制对于包括交通、能源和医学在内的多个科学与工程领域至关重要。有效的流动控制可以带来诸如升力增加、阻力减小、混合增强以及噪音减少等好处。然而，控制流体面临诸多挑战，其中包括高维性、非线性和空间时间多尺度交互作用。近年来，强化学习（RL）在复杂领域的应用如机器人技术和蛋白质折叠方面取得了显著成功，但其在流动控制中的应用受到缺乏标准化基准平台和流体模拟计算需求的限制。为了解决这些问题，我们引入了HydroGym——一个用于流动控制研究的独立求解器强化学习平台。HydroGym整合了复杂的流动控制基准测试、可扩展运行时基础设施以及先进的RL算法。我们的平台包括42个经过验证的环境，从经典的层流到复杂三维湍流场景不等，涵盖广泛的雷诺数范围。我们为传统的RL提供了非微分求解器，并提供可通过梯度增强优化显著提高样本效率的微分求解器。全面评估表明，强化学习代理能够一致地在各种配置中发现有效的控制原则，如边界层操作、声反馈干扰和尾流重组。迁移学习研究表明，在一个雷诺数或几何条件下学到的控制器可以高效适应新的条件，所需的训练回合大约减少了50%。HydroGym平台高度可扩展且具有规模性，为从事流体动力学、机器学习与控制研究的科研人员提供了一个框架，让他们能够添加环境、替代模型和控制算法，从而推进科学和技术的发展。
URL
https://arxiv.org/abs/2512.17534
