Operator compression with deep neural networks
Tóm tắt
This paper studies the compression of partial differential operators using neural networks. We consider a family of operators, parameterized by a potentially high-dimensional space of coefficients that may vary on a large range of scales. Based on the existing methods that compress such a multiscale operator to a finite-dimensional sparse surrogate model on a given target scale, we propose to directly approximate the coefficient-to-surrogate map with a neural network. We emulate local assembly structures of the surrogates and thus only require a moderately sized network that can be trained efficiently in an offline phase. This enables large compression ratios and the online computation of a surrogate based on simple forward passes through the network is substantially accelerated compared to classical numerical upscaling approaches. We apply the abstract framework to a family of prototypical second-order elliptic heterogeneous diffusion operators as a demonstrating example.
Tài liệu tham khảo
Abadi, M., Agarwal, A., Barham, P., Brevdo, E., Chen, Z., Citro, C., Corrado, G.S., Davis, A., Dean, J., Devin, M., Ghemawat, S., Goodfellow, I., Harp, A., Irving, G., Isard, M., Jia, Y., Jozefowicz, R., Kaiser, L., Kudlur, M., Levenberg, J., Mané, D., Monga, R., Moore, S., Murray, D., Olah, C., Schuster, M., Shlens, J., Steiner, B., Sutskever, I., Talwar, K., Tucker, P., Vanhoucke, V., Vasudevan, V., Viégas, F., Vinyals, O., Warden, P., Wattenberg, M., Wicke, M., Yu, Y., TensorFlow, X.Z.: Large-Scale Machine Learning on Heterogeneous Systems (2015) Software available from tensorflow.org
Abdulle, A., E, W., Engquist, B., Vanden-Eijnden, E.: The heterogeneous multiscale method. Acta Numer. 21, 1–87 (2012)
Abdulle, A., Henning, P.: A reduced basis localized orthogonal decomposition. J. Comput. Phys. 295, 379–401 (2015)
Abdulle, A., Henning, P.: Localized orthogonal decomposition method for the wave equation with a continuum of scales. Math. Comput. 86(304), 549–587 (2017)
Altmann, R., Henning, P., Peterseim, D.: Numerical homogenization beyond scale separation. Acta Numer. 30, 1–86 (2021)
Arbabi, H., Bunder, J.E., Samaey, G., Roberts, A.J., Kevrekidis, I.G.: Linking machine learning with multiscale numerics: data-driven discovery of homogenized equations. JOM 72(12), 4444–4457 (2020)
Babuška, I., Lipton, R.: Optimal local approximation spaces for generalized finite element methods with application to multiscale problems. Multiscale Model. Simul. 9(1), 373–406 (2011)
Babuška, I., Osborn, J.E.: Can a finite element method perform arbitrarily badly? Math. Comput. 69(230), 443–462 (2000)
Berner, J., Dablander, M., Grohs, P.: Numerically solving parametric families of high-dimensional Kolmogorov partial differential equations via deep learning. In: Larochelle, H., Ranzato, M., Hadsell, R., Balcan, M.F., Lin, H. (eds.) Advances in Neural Information Processing Systems, vol. 33, pp. 16615–16627. Curran Associates, Red Hook (2020)
Bezanson, J., Edelman, A., Karpinski, S., Shah, V.B.: Julia: a fresh approach to numerical computing. SIAM Rev. 59(1), 65–98 (2017)
Bhattacharya, K., Hosseini, B., Kovachki, N.B., Stuart, A.M.: Model reduction and neural networks for parametric PDEs. SMAI J. Comput. Math. 7, 121–157 (2021)
Caiazzo, A., Maier, R., Peterseim, D.: Reconstruction of quasi-local numerical effective models from low-resolution measurements. J. Sci. Comput. 85(1), Article ID 10 (2020)
Chan, S., Elsheikh, A.H.: A machine learning approach for efficient uncertainty quantification using multiscale methods. J. Comput. Phys. 354, 493–511 (2018)
De Giorgi, E.: Sulla convergenza di alcune successioni d’integrali del tipo dell’area. Rend. Mat. 6(8), 277–294 (1975)
E, W., Engquist, B.: The heterogeneous multiscale methods. Commun. Math. Sci. 1(1), 87–132 (2003)
E, W., Engquist, B.: The heterogeneous multi-scale method for homogenization problems. In: Multiscale Methods in Science and Engineering. Lect. Notes Comput. Sci. Eng., vol. 44, pp. 89–110. Springer, Berlin (2005)
E, W., Han, J., Jentzen, A.: Deep learning-based numerical methods for high-dimensional parabolic partial differential equations and backward stochastic differential equations. Commun. Math. Stat. 5(4), 349–380 (2017)
E, W., Han, J., Jentzen, A.: Algorithms for solving high dimensional PDEs: from nonlinear Monte Carlo to machine learning. Nonlinearity 35(1), 278–310 (2021)
E, W., Yu, B.: The deep Ritz method: a deep learning-based numerical algorithm for solving variational problems. Commun. Math. Stat. 6(1), 1–12 (2018)
Efendiev, Y.R., Galvis, J., Wu, X.-H.: Multiscale finite element methods for high-contrast problems using local spectral basis functions. J. Comput. Phys. 230(4), 937–955 (2011)
Efendiev, Y.R., Hou, T.Y.: Multiscale Finite Element Methods: Theory and Applications. Surveys and Tutorials in the Applied Mathematical Sciences, vol. 4. Springer, New York (2009)
Elfverson, D., Ginting, V., Henning, P.: On multiscale methods in Petrov-Galerkin formulation. Numer. Math. 131(4), 643–682 (2015)
Engwer, C., Henning, P., Målqvist, A., Peterseim, D.: Efficient implementation of the localized orthogonal decomposition method. Comput. Methods Appl. Mech. Eng. 350, 123–153 (2019)
Ern, A., Guermond, J.-L.: Finite element quasi-interpolation and best approximation. ESAIM: Math. Model. Numer. Anal. 51(4), 1367–1385 (2017)
Feischl, M., Peterseim, D.: Sparse compression of expected solution operators. SIAM J. Numer. Anal. 58(6), 3144–3164 (2020)
Gallistl, D., Henning, P., Verfürth, B.: Numerical homogenization of H(curl)-problems. SIAM J. Numer. Anal. 56(3), 1570–1596 (2018)
Gallistl, D., Peterseim, D.: Stable multiscale Petrov–Galerkin finite element method for high frequency acoustic scattering. Comput. Methods Appl. Math. 295, 1–17 (2015)
Gao, H., Sun, L., Wang, J.-X.: Phygeonet: physics-informed geometry-adaptive convolutional neural networks for solving parameterized steady-state PDEs on irregular domain. J. Comput. Phys. 428, 110079 (2021)
Geevers, S., Maier, R.: Fast mass lumped multiscale wave propagation modelling. IMA J. Numer. Anal. (2022) To appear
Geist, M., Petersen, P., Raslan, M., Schneider, R., Kutyniok, G.: Numerical solution of the parametric diffusion equation by deep neural networks. J. Sci. Comput. (2022) To appear
Ghavamian, F., Simone, A.: Accelerating multiscale finite element simulations of history-dependent materials using a recurrent neural network. Comput. Methods Appl. Math. 357, 112594 (2019)
Glorot, X., Bengio, Y.: Understanding the difficulty of training deep feedforward neural networks. In: Proceedings of the 13th International Conference on Artificial Intelligence and Statistics. JMLR Workshop and Conference Proceedings, pp. 249–256 (2010)
Han, J., Jentzen, A., E, W.: Solving high-dimensional partial differential equations using deep learning. Proc. Natl. Acad. Sci. 115(34), 8505–8510 (2018)
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)
Hellman, F., Keil, T., Målqvist, A.: Numerical upscaling of perturbed diffusion problems. SIAM J. Sci. Comput. 42(4), A2014–A2036 (2020)
Henning, P., Peterseim, D.: Oversampling for the multiscale finite element method. Multiscale Model. Simul. 11(4), 1149–1175 (2013)
Henning, P., Wärnegård, J.: Superconvergence of time invariants for the Gross-Pitaevskii equation. Math. Comput. 91(334), 509–555 (2022)
Hou, T.Y., Wu, X.-H.: A multiscale finite element method for elliptic problems in composite materials and porous media. J. Comput. Phys. 134(1), 169–189 (1997)
Hutzenthaler, M., Jentzen, A., Kruse, T., Nguyen, T.A.: A proof that rectified deep neural networks overcome the curse of dimensionality in the numerical approximation of semilinear heat equations. Ser. Partial Differ. Equ. Appl. 1(2), 1–34 (2020)
Innes, M.: Flux: elegant machine learning with Julia. J. Open Sour. Softw. (2018)
Khoo, Y., Lu, J., Ying, L.: Solving parametric PDE problems with artificial neural networks. Eur. J. Appl. Math. 32(3), 421–435 (2021)
Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization (2014). Preprint arXiv:1412.6980
Kutyniok, G., Petersen, P., Raslan, M., Schneider, R.: A theoretical analysis of deep neural networks and parametric PDEs. Constr. Approx. (2022) To appear
Maier, R., Peterseim, D.: Explicit computational wave propagation in micro-heterogeneous media. BIT Numer. Math. 59(2), 443–462 (2019)
Maier, R., Verfürth, B.: Multiscale scattering in nonlinear Kerr-type media. Math. Comput. (2022) To appear
Målqvist, A., Peterseim, D.: Localization of elliptic multiscale problems. Math. Comput. 83(290), 2583–2603 (2014)
Målqvist, A., Peterseim, D.: Computation of eigenvalues by numerical upscaling. Numer. Math. 130(2), 337–361 (2015)
Målqvist, A., Peterseim, D.: Generalized finite element methods for quadratic eigenvalue problems. ESAIM: Math. Model. Numer. Anal. 51(1), 147–163 (2017)
Målqvist, A., Peterseim, D.: Numerical Homogenization by Localized Orthogonal Decomposition. SIAM Spotlights, vol. 5. SIAM, Philadelphia (2020)
Målqvist, A., Verfürth, B.: An offline-online strategy for multiscale problems with random defects. ESAIM: Math. Model. Numer. Anal. (2022) To appear
Murat, F., Tartar, L.: H-convergence. In: Séminaire d’Analyse Fonctionnelle et Numérique de l’Université d’Alger (1978)
Owhadi, H.: Multigrid with rough coefficients and multiresolution operator decomposition from hierarchical information games. SIAM Rev. 59(1), 99–149 (2017)
Owhadi, H., Zhang, L., Berlyand, L.: Polyharmonic homogenization, rough polyharmonic splines and sparse super-localization. ESAIM: Math. Model. Numer. Anal. 48(2), 517–552 (2014)
Padmanabha, G.A., Zabaras, N.: A Bayesian multiscale deep learning framework for flows in random media. Found. Data Sci. 3(2), 251–303 (2021)
Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., Antiga, L., Desmaison, A., Kopf, A., Yang, E., DeVito, Z., Raison, M., Tejani, A., Chilamkurthy, S., Steiner, B., Fang, L., Bai, J., PyTorch, S.C.: An imperative style, high-performance deep learning library. In: Wallach, H., Larochelle, H., Beygelzimer, A., Alché-Buc, F., Fox, E., Garnett, R. (eds.) Advances in Neural Information Processing Systems, vol. 32, pp. 8024–8035. Curran Associates, Red Hook (2019)
Peterseim, D.: Variational multiscale stabilization and the exponential decay of fine-scale correctors. In: Building Bridges: Connections and Challenges in Modern Approaches to Numerical Partial Differential Equations. Lect. Notes Comput. Sci. Eng., vol. 114, pp. 341–367. Springer, Cham (2016)
Peterseim, D.: Eliminating the pollution effect in Helmholtz problems by local subscale correction. Math. Comput. 86(305), 1005–1036 (2017)
Peterseim, D., Sauter, S.A.: Finite elements for elliptic problems with highly varying, nonperiodic diffusion matrix. Multiscale Model. Simul. 10(3), 665–695 (2012)
Peterseim, D., Schedensack, M.: Relaxing the CFL condition for the wave equation on adaptive meshes. J. Sci. Comput. 72(3), 1196–1213 (2017)
Raissi, M., Perdikaris, P., Karniadakis, G.E.: Physics-informed neural networks: a deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. J. Comput. Phys. 378, 686–707 (2019)
Ren, X., Hannukainen, A., Belahcen, A.: Homogenization of multiscale Eddy current problem by localized orthogonal decomposition method. IEEE Trans. Magn. 55(9), 1–4 (2019)
Schwab, C., Zech, J.: Deep learning in high dimension: neural network expression rates for generalized polynomial chaos expansions in UQ. Anal. Appl. 17(01), 19–55 (2019)
Sirignano, J., Spiliopoulos, K.: DGM: a deep learning algorithm for solving partial differential equations. J. Comput. Phys. 375, 1339–1364 (2018)
Spagnolo, S.: Sulla convergenza di soluzioni di equazioni paraboliche ed ellittiche. Ann. Sc. Norm. Super. Pisa, Cl. Sci. 3(22), 571–597 (1968)
Wang, Y., Cheung, S.W., Chung, E.T., Efendiev, Y., Wang, M.: Deep multiscale model learning. J. Comput. Phys. 406, 109071 (2020)