Failure-Informed Adaptive Sampling for PINNs, Part II: Combining with Re-sampling and Subset Simulation
Communications on Applied Mathematics and Computation - Trang 1-22 - 2023
Tóm tắt
This is the second part of our series works on failure-informed adaptive sampling for physic-informed neural networks (PINNs). In our previous work (SIAM J. Sci. Comput. 45: A1971–A1994), we have presented an adaptive sampling framework by using the failure probability as the posterior error indicator, where the truncated Gaussian model has been adopted for estimating the indicator. Here, we present two extensions of that work. The first extension consists in combining with a re-sampling technique, so that the new algorithm can maintain a constant training size. This is achieved through a cosine-annealing, which gradually transforms the sampling of collocation points from uniform to adaptive via the training progress. The second extension is to present the subset simulation (SS) algorithm as the posterior model (instead of the truncated Gaussian model) for estimating the error indicator, which can more effectively estimate the failure probability and generate new effective training points in the failure region. We investigate the performance of the new approach using several challenging problems, and numerical experiments demonstrate a significant improvement over the original algorithm.
Tài liệu tham khảo
Au, S.K., Beck, J.L.: Estimation of small failure probabilities in high dimensions by subset simulation. Probab. Eng. Mech. 16, 263–277 (2001)
Bischof, R., Kraus, M.: Multi-objective loss balancing for physics-informed deep learning. arXiv:2110.09813 (2021)
Daw, A., Bu, J., Wang, S., Perdikaris, P., Karpatne, A.: Rethinking the importance of sampling in physics-informed neural networks. arXiv:2207.02338 (2022)
E, W.N.: Machine learning and computational mathematics. Commun. Comput. Phys. 28, 1639–1670 (2020)
E, W.N., Yu, B.: The deep Ritz method: a deep learning-based numerical algorithm for solving variational problems. Commun. Math. Stat. 6, 1–12 (2018)
Gao, Z., Yan, L., Zhou, T.: Failure-informed adaptive sampling for PINNs. SIAM J. Sci. Comput. 45, A1971–A1994 (2023)
Krishnapriyan, A., Gholami, A., Zhe, S., Kirby, R., Michael, W.M.: Characterizing possible failure modes in physics-informed neural networks. Adv. Neural Inform. Process. Syst. 34, 26548–26560 (2021)
Lu, L., Meng, X., Mao, Z., Karniadakis, G.E.: DeepXDE: a deep learning library for solving differential equations. SIAM Rev. 63, 208–228 (2021)
McClenny, L., Braga-Neto, U.: Self-adaptive physics-informed neural networks using a soft attention mechanism. arXiv:2009.04544 (2020)
Peng, W., Zhou, W., Zhang, X., Yao, W., Liu, Z.: RANG: a residual-based adaptive node generation method for physics-informed neural networks. arXiv:2205.01051 (2022)
Raissi, M., Perdikaris, P., George, E.K.: Physics-informed neural networks: a deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. J. Comput. Phys. 378, 686–707 (2019)
Sirignano, J., Spiliopoulos, K.: DGM: a deep learning algorithm for solving partial differential equations. J. Comput. Phys. 375, 1339–1364 (2018)
Subramanian, S., Kirby, R.M., Mahoney, M.W., Gholami, A.: Adaptive self-supervision algorithms for physics-informed neural networks. arXiv:2207.04084 (2022)
Tang, K., Wan, X., Yang, C.: DAS: a deep adaptive sampling method for solving partial differential equations. arXiv:2112.14038 (2021)
Wang, S., Sankaran, S., Perdikaris, P.: Respecting causality is all you need for training physics-informed neural networks. arXiv:2203.07404 (2022)
Wang, S., Teng, Y., Perdikaris, P.: Understanding and mitigating gradient flow pathologies in physics-informed neural networks. SIAM J. Sci. Comput. 43, A3055–A3081 (2021)
Wang, S., Yu, X.L., Perdikaris, P.: When and why PINNs fail to train: a neural tangent kernel perspective. J. Comput. Phys. 449, 110768 (2022)
Wu, C.X., Zhu, M., Tan, Q., Kartha, Y., Lu, L.: A comprehensive study of non-adaptive and residual-based adaptive sampling for physics-informed neural networks. Comput. Methods Appl. Mech. Eng. 403, 115671 (2023)
Xiang, Z., Wei, P., Liu, X., Yao, W.: Self-adaptive loss balanced physics-informed neural networks. Neurocomputing 496, 11–34 (2022)
Zang, Y., Bao, G., Ye, X., Zhou, H.: Weak adversarial networks for high-dimensional partial differential equations. J. Comput. Phys. 411, 109409 (2020)
Zuev, K.: Subset simulation method for rare event estimation: an introduction. arXiv:1505.03506 (2015)