Levenberg-marquardt revisited and parameter tuning of river regression models
Tóm tắt
The Levenberg-Marquardt method is well known for solving nonlinear least squares problems. This method is mostly used in the context of overdetermined systems. In this paper it is shown that suitable implementations with respect to underdetermined systems can be defined. The resulting method is equipped with a new effective non-monotone strategy and it is proved that, when the residual tends to zero, a sufficient descent condition is obtained with minimal computational cost. The method is applied to the problem of parameter fitting of regression models based on Neural Networks for natural rivers.
Tài liệu tham khảo
Birgin EG, Correa MR, González-López V, Martínez JM, Rodrigues DS (2023) Randomly supported models for the prediction of flows in channels, Technical Report. University of Campinas
Birgin EG, Martínez JM (2023) A PDE-informed optimization algorithm for river flow predictions. Numer Algorithms. https://doi.org/10.1007/s11075-023-01647-1
Bottou L, Curtis FE, Nocedal J (2018) Optimization methods for large-scale machine learning. SIAM Rev 60:223–311
Brunton SL, Kutz JL (2020) Data-driven science and engineering. Cambridge University Press, Cambridge
Conn AR, Conn AR, Gould NIM, Toint PhL (2000) Trust-region methods, MPS/SIAM series on optimization. SIAM, Philadelphia
Dennis JE, Schnabel RS (1983) Numerical methods for unconstrained optimization and nonlinear equations. Prentice Hall, Englewood Cliffs
Emmett WW, Myrick WW, Meade RH (1979) Field data describing the movement and storage of sediment in the East Fork River, Wyoming, Part 1, River Hydraulics and Sediment Transport, Report No. 1
Golub GH, Van Loan Ch (1996) Matrix computations, 3rd edn. Johns Hopkins University Press, Baltimore
Grippo L, Lampariello F, Lucidi S (1986) Nonmonotone line search technique for Newton’s method. SIAM J Numer Anal 23:707–716
Karas EW, Santos SA, Svaiter BF (2016) Algebraic rules for computing the regularization parameter of the Levenberg-Marquardt method. Comput Optim Appl 65:723–751
Levenberg K (1944) A method for the solution of certain nonlinear problems in least squares. Q Appl Math 2:164–168
LeVeque RJ (1992) Numerical methods for conservation laws. Lectures in mathematics. Birkäuser, ETH Zürich
Marquardt DW (1963) An algorithm for least squares estimation of nonlinear parameters. SIAM J Appl Math 11:431–441
Martínez JM (1986) Solution of nonlinear systems of equations by an optimal projection method. Computing 37:59–70
Martínez JM (1991) Quasi-Newton methods for solving underdetermined nonlinear simultaneous equations. J Comput Appl Math 34:171–190
Meade RH, Myrick WW, Emmett WW (1979) Field data describing the movement and storage of sediment in the East Fork River, Wyoming, Part 2, River Hydraulics and Sediment Transport, Report No. 2
Meyn K-H (1983) Solution of underdetermined nonlinear equations by stationary iteration methods. Numer Math 42:161–172
Moré JJ (1978) The Levenberg-Marquardt algorithm: implementation and theory. In: Watson GA (ed) Numerical analysis. Lecture notes in mathematics, vol 630. Springer, Berlin
Nocedal J, Wright SJ (1999) Numerical optimization. Springer, New York
Saint-Venant AJC (1871) Théorie du mouvement non-permanent des eaux, avec application aux crues des rivière at à l’introduction des marées dans leur lit. C R Séances Acad Sci 73:147–154
Sra S, Nowozin S, Wright SJ (2011) Optimization for machine learning. The MIT Press, Cambridge
Walker H, Watson LT (1990) Least-change secant update methods for under-determined systems. SIAM J Numer Anal 27:1227–1262
Wright SJ (2023) Optimization in machine learning and data science. SIAM News 56:1–2