DRLFluent: A distributed co-simulation framework coupling deep reinforcement learning with Ansys-Fluent on high-performance computing systems

Zhang, 2021, Artificial intelligence in fluid mechanics, Acta Mech. Sin., 37, 1715, 10.1007/s10409-021-01154-3 Brunton, 2020, Machine learning for fluid mechanics, Annu. Rev. Fluid Mech., 52, 477, 10.1146/annurev-fluid-010719-060214 Duraisamy, 2019, Turbulence modeling in the age of data, Annu. Rev. Fluid Mech., 51, 357, 10.1146/annurev-fluid-010518-040547 Partee, 2022, Using machine learning at scale in numerical simulations with SmartSim: an application to ocean climate modeling, J. Comput. Sci., 62, 10.1016/j.jocs.2022.101707 Wang, 2022, TransFlowNet: a physics-constrained Transformer framework for spatio-temporal super-resolution of flow simulations, J. Comput. Sci., 65, 10.1016/j.jocs.2022.101906 Tang, 2023, Discovering explicit Reynolds-averaged turbulence closures for turbulent separated flows through deep learning-based symbolic regression with non-linear corrections, Phys. Fluids, 35, 10.1063/5.0135638 Mücke, 2021, Reduced order modeling for parameterized time-dependent PDEs using spatially and memory aware deep learning, J. Comput. Sci., 53, 10.1016/j.jocs.2021.101408 Maulik, 2022, PythonFOAM: in-situ data analyses with OpenFOAM and Python, J. Comput. Sci., 62, 10.1016/j.jocs.2022.101750 Bai, 2023, A reduced order modeling approach with Petrov–Galerkin projection based on hybrid snapshot simulation in Semilinear-PDEs, J. Comput. Sci., 67, 10.1016/j.jocs.2023.101969 Li, 2022, Machine learning in aerodynamic shape optimization, Prog. Aerosp. Sci., 134, 10.1016/j.paerosci.2022.100849 Yan, 2019, Aerodynamic shape optimization using a novel optimizer based on machine learning techniques, Aerosp. Sci. Technol., 86, 826, 10.1016/j.ast.2019.02.003 Paris, 2023, Reinforcement-learning-based actuator selection method for active flow control, J. Fluid Mech., 955, 10.1017/jfm.2022.1043 Hosseini, 2023, Flow control with synthetic jets on two tandem airfoils using machine learning, Phys. Fluids, 35, 10.1063/5.0135428 Mu, 2022, Machine learning-based active flutter suppression for a flexible flying-wing aircraft, J. Sound Vib., 529, 10.1016/j.jsv.2022.116916 Lai, 2023, Intelligent controller for unmanned surface vehicles by deep reinforcement learning, Phys. Fluids, 35, 10.1063/5.0139568 Brunton, 2016, Discovering governing equations from data by sparse identification of nonlinear dynamical systems, Proc. Natl. Acad. Sci. USA, 113, 3932, 10.1073/pnas.1517384113 Brunton, 2022 Viquerat, 2022, A review on deep reinforcement learning for fluid mechanics: an update, Phys. Fluids, 34, 10.1063/5.0128446 Garnier, 2021, A review on deep reinforcement learning for fluid mechanics, Comput. Fluids, 225, 10.1016/j.compfluid.2021.104973 Wang, 2022, Deep-learning-based super-resolution reconstruction of high-speed imaging in fluids, Phys. Fluids, 34 Rabault, 2020, Deep reinforcement learning in fluid mechanics: a promising method for both active flow control and shape optimization, J. Hydrodyn., 32, 234, 10.1007/s42241-020-0028-y Vignon, 2023, Recent advances in applying deep reinforcement learning for flow control: perspectives and future directions, Phys. Fluids, 35, 10.1063/5.0143913 L. Guastoni, J. Rabault, P. Schlatter, H. Azizpour, R. Vinuesa, Deep reinforcement learning for turbulent drag reduction in channel flows, ArXiv.Org. (2023). 〈https://arxiv.org/abs/2301.09889v3〉. (Accessed 4 April 2023). Ghraieb, 2022, Single-step deep reinforcement learning for two- and three-dimensional optimal shape design, AIP Adv., 12, 10.1063/5.0097241 Xie, 2022, An active-controlled heaving plate breakwater trained by an intelligent framework based on deep reinforcement learning, Ocean Eng., 244, 10.1016/j.oceaneng.2021.110357 Keramati, 2022, Deep reinforcement learning for heat exchanger shape optimization, Int. J. Heat. Mass Transf., 194, 10.1016/j.ijheatmasstransfer.2022.123112 Viquerat, 2021, Direct shape optimization through deep reinforcement learning, J. Comput. Phys., 428, 10.1016/j.jcp.2020.110080 Hui, 2021, Multi-object aerodynamic design optimization using deep reinforcement learning, AIP Adv., 11, 10.1063/5.0058088 Ren, 2021, Bluff body uses deep-reinforcement-learning trained active flow control to achieve hydrodynamic stealth, Phys. Fluids, 33 Li, 2022, Reinforcement-learning-based control of confined cylinder wakes with stability analyses, J. Fluid Mech., 932, 10.1017/jfm.2021.1045 Chen, 2022, Control of quasi-equilibrium state of annular flow through reinforcement learning, Phys. Fluids, 34, 10.1063/5.0102668 Rabault, 2019, Artificial neural networks trained through deep reinforcement learning discover control strategies for active flow control, J. Fluid Mech., 865, 281, 10.1017/jfm.2019.62 Rabault, 2019, Accelerating deep reinforcement learning strategies of flow control through a multi-environment approach, Phys. Fluids, 31, 10.1063/1.5116415 M. Schaarschmidt, A. Kuhnle, B. Ellis, K. Fricke, F. Gessert, E. Yoneki, LIFT: Reinforcement Learning in Computer Systems by Learning From Demonstrations, (2018). doi: 10.48550/arXiv.1808.07903. Tang, 2020, Robust active flow control over a range of Reynolds numbers using an artificial neural network trained through deep reinforcement learning, Phys. Fluids, 32, 10.1063/5.0006492 Mao, 2022, Active flow control using deep reinforcement learning with time delays in Markov decision process and autoregressive policy, Phys. Fluids, 34, 10.1063/5.0086871 Ren, 2021, Applying deep reinforcement learning to active flow control in weakly turbulent conditions, Phys. Fluids, 33 Ren, 2021, Bluff body uses deep-reinforcement-learning trained active flow control to achieve hydrodynamic stealth, Phys. Fluids, 33 Fan, 2020, Reinforcement learning for bluff body active flow control in experiments and simulations, Proc. Natl. Acad. Sci. USA, 117, 26091, 10.1073/pnas.2004939117 Gazzola, 2016, Learning to school in the presence of hydrodynamic interactions, J. Fluid Mech., 789, 726, 10.1017/jfm.2015.686 Zheng, 2022, Data-efficient deep reinforcement learning with expert demonstration for active flow control, Phys. Fluids, 34, 10.1063/5.0120285 Kim, 2022, Deep reinforcement learning for large-eddy simulation modeling in wall-bounded turbulence, Phys. Fluids, 34, 10.1063/5.0106940 Castellanos, 2022, Machine-learning flow control with few sensor feedback and measurement noise, Phys. Fluids, 34, 10.1063/5.0087208 Wang, 2022, DRLinFluids: an open-source Python platform of coupling deep reinforcement learning and OpenFOAM, Phys. Fluids, 34 Kurz, 2022, Deep reinforcement learning for computational fluid dynamics on HPC systems, J. Comput. Sci., 65, 10.1016/j.jocs.2022.101884 Novati, 2021, Automating turbulence modelling by multi-agent reinforcement learning, Nat. Mach. Intell., 3, 87, 10.1038/s42256-020-00272-0 Bae, 2022, Scientific multi-agent reinforcement learning for wall-models of turbulent flows, Nat. Commun., 13, 10.1038/s41467-022-28957-7 Pawar, 2021, Distributed deep reinforcement learning for simulation control, Mach. Learn. Sci. Technol., 2, 10.1088/2632-2153/abdaf8 Kurz, 2023, Deep reinforcement learning for turbulence modeling in large eddy simulations, Int. J. Heat. Fluid Flow., 99, 10.1016/j.ijheatfluidflow.2022.109094 Krais, 2021, FLEXI: a high order discontinuous Galerkin framework for hyperbolic–parabolic conservation laws, Comput. Math. Appl., 81, 186, 10.1016/j.camwa.2020.05.004 Sutton, 2018 J. Schulman, F. Wolski, P. Dhariwal, A. Radford, O. Klimov, Proximal Policy Optimization Algorithms, (2017). doi: 10.48550/arXiv.1707.06347. S. Guadarrama , A. Korattikara , O. Ramirez , P. Castro , E. Holly , S. Fishman , K. Wang , E. Gonina , N. Wu , E. Kokiopoulou , L. Sbaiz , J. Smith , G. Bartók , J. Berent , C. Harris , V. Vanhoucke , E. Brevdo , TF-Agents: a Library for Reinforcement Learning in TensorFlow, 2018. 〈https://github.com/tensorflow/agents〉. M.W. Hoffman, B. Shahriari, J. Aslanides, G. Barth-Maron, N. Momchev, D. Sinopalnikov, P. Stańczyk, S. Ramos, A. Raichuk, D. Vincent, Acme: a research framework for distributed reinforcement learning, ArXiv Preprint ArXiv:2006.00979. (2020). M. Lanctot, E. Lockhart, J.-B. Lespiau, V. Zambaldi, S. Upadhyay, J. Pérolat, S. Srinivasan, F. Timbers, K. Tuyls, S. Omidshafiei, OpenSpiel: A framework for reinforcement learning in games, ArXiv Preprint ArXiv:1908.09453. (2019). Weng, 2021, Tianshou: a highly modularized deep reinforcement learning library, ArXiv Preprint ArXiv, 2107, 14171 A. Kuhnle , M. Schaarschmidt , K. Fricke , Tensorforce: A TensorFlow Library for Applied Reinforcement Learning, 2017. 〈https://github.com/tensorforce/tensorforce〉. M. Schäfer, S. Turek, F. Durst, E. Krause, R. Rannacher, Benchmark computations of laminar flow around a cylinder, in: Proceedings of the Flow Simulation with HighPerformance Computers II: DFG Priority Research Programme Results 1993–1995, Vieweg Teubner Verlag, Wiesbaden, 1996, 547–566. https://doi.org/10.1007/978–3-322–89849-4_39.