An accurate active set newton algorithm for large scale bound constrained optimization
Tóm tắt
A new algorithm for solving large scale bound constrained minimization problems is proposed. The algorithm is based on an accurate identification technique of the active set proposed by Facchinei, Fischer and Kanzow in 1998. A further division of the active set yields the global convergence of the new algorithm. In particular, the convergence rate is superlinear without requiring the strict complementarity assumption. Numerical tests demonstrate the efficiency and performance of the present strategy and its comparison with some existing active set strategies.
Tài liệu tham khảo
T.F. Coleman, Y. Li: On the convergence of interior-reflective Newton methods for nonlinear minimization subject to bounds. Math. Program. 67 (1994), 189–224.
T.F. Coleman, Y. Li: An interior trust region approach for nonlinear minimization subject to bounds. SIAM J. Optim. 6 (1996), 418–445.
A.R. Conn, N. I.M. Gould, Ph. L. Toint: Testing a class of methods for solving minimization problems with simple bounds on the variables. ACM Trans. Math. Softw. 7 (1981), 17–41.
A.R. Conn, N. I.M. Gould, Ph. L. Toint: Global convergence of a class of trust region algorithms for optimization with simple bounds. SIAM J. Numer. Anal. 25 (1988), 433–460.
A.R. Conn, N. I.M. Gould, Ph. L. Toint: Correction to the paper on global convergence of a class of trust region algorithms for optimization with simple bounds. SIAM J. Numer. Anal. 26 (1989), 764–767.
J.E. Dennis, J. J. Moré: A characterization of superlinear convergence and its application to quasi-Newton methods. Math. Comput. 28 (1974), 549–560.
Z. Dostál: A proportioning based algorithm with rate of convergence for bound constrained quadratic programming. Numer. Algorithms 34 (2003), 293–302.
F. Facchinei: Minimization of SC1 functions and the Maratos effect. Oper. Res. Lett. 17 (1995), 131–137.
F. Facchinei, A. Fischer, C. Kanzow: On the accurate identification of active constraints. SIAM J. Optim. 9 (1998), 14–32.
F. Facchinei, J. Júdice, J. Soares: An active set Newton algorithm for large-scale nonlinear programs with box constraints. SIAM J. Optim. 8 (1998), 158–186.
F. Facchinei, J. Júdice, J. Soares: Generating box-constrained optimization problems. ACM Trans. Math. Softw. 23 (1997), 443–447.
F. Facchinei, S. Lucidi: Quadratically and superlinearly convergent algorithms for the solution of inequality constrained minimization problems. J. Optimization Theory Appl. 85 (1995), 265–289.
F. Facchinei, S. Lucidi, L. Palagi: A truncated Newton algorithm for large scale box constrained optimization. SIAM J. Optim. 12 (2002), 1100–1125.
R.E. Fan, P.H. Chen, C. J. Lin: Working set selection using second order information for training support vector machines. J. Mach. Learn. Res. 6 (2005), 1889–1918.
P.E. Gill, W. Murray, M.H. Wright: Practical Optimization. Academic Press, London, 1981.
W.W. Hager, H. Zhang: A new active set algorithm for box constrained optimization. SIAM J. Optim. 17 (2006), 526–557.
M. Heinkenschloss, M. Ulbrich, S. Ulbrich: Superlinear and quadratic convergence of affine-scaling interior-point Newton methods for problems with simple bounds without strict complementarity assumption. Math. Program. 86 (1999), 615–635.
S. S. Keerthi, E.G. Gilbert: Convergence of a generalized SMO algorithm for SVM classifier design. Mach. Learn. 46 (2002), 351–360.
M. Lescrenier: Convergence of trust region algorithms for optimization with bounds when strict complementarity does not hold. SIAM J. Numer. Anal. 28 (1991), 476–495.
C. J. Lin, J. J. Moré: Newton’s method for large bound-constrained optimization problems. SIAM J. Optim. 9 (1999), 1100–1127.
J. J. Moré, G. Toraldo: On the solution of large quadratic programming problems with bound constraints. SIAM J. Optim. 1 (1991), 93–113.
Q. Ni, Y. Yuan: A subspace limited memory quasi-Newton algorithm for large-scale nonlinear bound constrained optimization. Math. Comput. 66 (1997), 1509–1520.
J. Nocedal, S. J. Wright: Numerical Optimization. Springer, New York, 2006.
C. Oberlin, S. J. Wright: Active set identification in nonlinear programming. SIAM J. Optim. 17 (2006), 577–605.
G.Di Pillo, F. Facchinei, L. Grippo: An RQP algorithm using a differentiable exact penalty function for inequality constrained problems. Math. Program. 55 (1992), 49–68.
B.T. Polyak: The conjugate gradient method in extremal problems. U.S.S.R. Comput. Math. Math. Phys. 9 (1969), 94–112.
K. Schittkowski: More Test Examples for Nonlinear Programming Codes. Lecture Notes in Economics and Mathematical Systems, Vol. 282. Springer, Berlin, 1987.
L. Sun, G. P. He, Y. L. Wang, L. Fang: An active set quasi-Newton method with projected search for bound constrained minimization. Comput. Math. Appl. 58 (2009), 161–170.
L. Sun, L. Fang, G. P. He: An active set strategy based on the multiplier function or the gradient. Appl. Math. 55 (2010), 291–304.
Y.H. Xiao, Z.X. Wei: A new subspace limited memory BFGS algorithm for large-scale bound constrained optimization. Appl. Math. Comput. 185 (2007), 350–359.