A deep reinforcement learning optimization framework for supercritical airfoil aerodynamic shape design
Tóm tắt
In the context of traditional aerodynamic shape optimization design methods, the necessity to re-execute the complete optimization process when the initial shape changes poses significant challenges in engineering applications. These challenges encompass problems like data wastage and restricted ability for experience learning. We propose a policy learning-based optimization method that can automatically learn optimization experience through interactions with the environment. This optimization framework is based on deep reinforcement learning and consists of the policy learning process and the policy execution process. The action network, trained during the policy learning process, serves as a black box model of optimization experience and can directly and efficiently participate in guiding the actual optimization process. The optimization framework is validated through two-dimensional Rosenbrock function optimization, demonstrating its exceptional performance in achieving high-precision optimal solutions. Then, the effectiveness of this optimization method is demonstrated in the multi-point optimization design of supercritical airfoils, which aims to improve the buffet onset lift within predefined design constraints while maintaining the cruise lift-drag ratio. With the datum-coupled state format, the optimization experience can be tailored to the optimization requirements of different initial states during the learning process, leading to an optimization success rate in the optimization space that can exceed 90%.
Từ khóa
Tài liệu tham khảo
Crouch JD, Garbaruk A, Magidov D, Travin A (2009) Origin of transonic buffet on aerofoils. J Fluid Mech 628:357–369. https://doi.org/10.1017/S0022112009006673
François-Lavet V, Henderson P, Islam R, Bellemare MG, Pineau J (2018) An introduction to deep reinforcement learning. Found Trends Mach Learn 11(3–4):219–354. https://doi.org/10.1561/2200000071
Geist M, Scherrer B, Pietquin O (2019) A theory of regularized markov decision processes. Proceedings of the 36th International Conference on Machine Learning, PMLR 97:2160–2169
Han R, Wang Y, Qian W, Wang W, Zhang M, Chen G (2022) Deep neural network based reduced-order model for fluid-structure interaction system. Phys Fluids 10(1063/5):0096432
Harris CD (1990) NASA supercritical airfoils: A matrix of family-related airfoils. No. NASA-TP-2969
Jofre L, Doostan A (2022) Rapid aerodynamic shape optimization under uncertainty using a stochastic gradient approach. Struct Multidisc Optim 65(7):196. https://doi.org/10.1007/s00158-022-03293-y
Kaelbling LP, Littman ML, Moore AW (1996) Reinforcement learning: a survey. J Artif Intell Res 4:237–285. https://doi.org/10.1613/jair.301
Kang YE, Yang S, Yee K (2022) Physics-aware reduced-order modeling of transonic flow via β-variational autoencoder. Phys Fluids 34(7):076103. https://doi.org/10.1063/5.0097740
Kim S, Kim I, You D (2022) Multi-condition multi-objective optimization using deep reinforcement learning. J Comput Phys 462:111263. https://doi.org/10.1016/j.jcp.2022.111263
Kingma DP, Ba. J (2014) Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980. https://doi.org/10.48550/arXiv.1412.6980
Konda V, Tsitsiklis J (1999) Actor-Critic algorithms. Adv neural inf process syst 12
Krist SL (1998) CFL3D User's Manual (Version 5.0). National Aeronautics and Space Administration, Langley Research Center
Lee BHK (2001) Self-sustained shock oscillations on airfoils at transonic speeds. Prog Aerosp Sci 37(2):1s47-196. https://doi.org/10.1016/S03760421(01)00003-3
Li R, Zhang Y, Chen H (2021) Learning the aerodynamic design of supercritical airfoils through deep reinforcement learning. AIAA J 59(10):3988–4001. https://doi.org/10.2514/1.J060189
Li R, Zhang Y, Chen H (2022) Pressure distribution feature-oriented sampling for statistical analysis of supercritical airfoil aerodynamics. Chin J Aeronaut 35(4):134–147. https://doi.org/10.1016/j.cja.2021.10.028
Liao P, Song W, Du P, Zhao H (2021) Multi-fidelity convolutional neural network surrogate model for aerodynamic optimization based on transfer learning. Phys Fluids 33:127121. https://doi.org/10.1063/5.0076538
Lim HW, Kim H (2019) Multi-objective airfoil shape optimization using an adaptive hybrid evolutionary algorithm. Aerosp Sci Technol 87:141–153. https://doi.org/10.1016/j.ast.2019.02.016
Liu J, Song WP, Han ZH, Zhang Y (2017) Efficient aerodynamic shape optimization of transonic wings using a parallel infilling strategy and surrogate models. Struct Multidisc Optim 55:925–943. https://doi.org/10.1007/s00158-016-1546-7
Mao Y, Zhong S, Yin H (2022) Active flow control using deep reinforcement learning with time delays in Markov decision process and autoregressive policy. Phys Fluids. https://doi.org/10.1063/5.0086871
Oyama A, Obayashi S, Nakahashi K (2000) Real-coded adaptive range genetic algorithm and its application to aerodynamic design. JSME Int J A-Solid M 43(2):124–129. https://doi.org/10.1299/jsmea.43.124
Qian J, Yi J, Cheng Y, Liu J, Zhou Q (2020) A sequential constraint updating approach for Kriging surrogate model-assisted engineering optimization design problem. Eng Comput 36:993–1009. https://doi.org/10.1007/s00366-019-00745-w
Rabault J, Kuchta M, Jensen A, Réglade U, Cerardi N (2019) Artificial neural networks trained through deep reinforcement learning discover control strategies for active flow control. J Fluid Mech 865:281–302. https://doi.org/10.1017/jfm.2019.62
Raul V, Leifsson L (2021) Surrogate-based aerodynamic shape optimization for delaying airfoil dynamic stall using Kriging regression and infill criteria. Aerosp Sci Technol 111:106555. https://doi.org/10.1016/j.ast.2021.106555
Schulman J, Wolski F, Dhariwal P, Radford A, Klimov O (2017) Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347. https://doi.org/10.48550/arXiv.1707.06347
Shi Y, Mader CA, Martins JRRA (2021) Natural laminar flow wing optimization using a discrete adjoint approach. Struct Multidisc Optim 64(2):541–562. https://doi.org/10.1007/s00158-021-02936-w
Silver D, Huang A, Maddison CJ, Guez A, Sifre L, Driessche G, Schrittwieser J, Antonoglou I, Panneershelvam V, Lanctot M, Dieleman S, Grewe D, Nham J, Kalchbrenner N, Sutskever I, Lillicrap T, Leach M, Kavukcuoglu K, Graepel T, Hassabis D (2016) Mastering the game of Go with deep neural networks and tree search. Nature 529:484–489. https://doi.org/10.1038/nature16961
Su SJ, Chow CY (1995) Improvement of transonic wing buffet by geometric Modifications. J Aircr 32(4):901–903. https://doi.org/10.2514/3.46815
Tao J, Sun G (2019) Application of deep learning based multi-fidelity surrogate model to robust aerodynamic design optimization. Aerosp Sci Technol 92:722–737. https://doi.org/10.1016/j.ast.2019.07.002
Tian X, Li J (2020) Robust aerodynamic shape optimization using a novel multi-objective evolutionary algorithm coupled with surrogate model. Struct Multidisc Optim 62:1969–1987. https://doi.org/10.1007/s00158-020-02589-1
Viquerat J, Rabault J, Kuhnle A, Ghraieb H, Larcher A, Hachem E (2021) Direct shape optimization through deep reinforcement learning. J Comput Phys 428:110080. https://doi.org/10.1016/j.jcp.2020.110080
Wang J, He C, Li R, Chen H, Zhai C, Zhang M (2021) Flow field prediction of supercritical airfoils via variational autoencoder based deep learning framework. Phys Fluids 33:086108. https://doi.org/10.1063/5.0053979
Wang J, Xie H, Zhang M, Xu H (2023) Physics-assisted reduced-order modeling for identifying dominant features of transonic buffet. Phys Fluids 35:066124. https://doi.org/10.1063/5.0152127
Xie H, Wang J, Zhang M (2023) Parametric generative schemes with geometric constraints for encoding and synthesizing airfoils. arXiv:2205.02458. https://doi.org/10.48550/arXiv.2205.02458
Xu Z, Saleh JH, Yang V (2019) Optimization of supercritical airfoil design with buffet effect. AIAA J 57:4343–4353. https://doi.org/10.2514/1.j057573s
Yilmaz E, German B (2017) A convolutional neural network approach to training predictors for airfoil performance. 18th AIAA/ISSMO MA&O conference 3660. https://doi.org/10.2514/6.2017-3660
Yonekura K, Hattori H (2019) Framework for design optimization using deep reinforcement learning. Struct Multidisc Optim 60:1709–1713. https://doi.org/10.1007/s00158-019-02276-w