Random block-coordinate methods for inconsistent convex optimisation problems

Mathias Staudigl1, Paulin Jacquot2
1Department of Business Informatics and Mathematics, University of Mannheim, Mannheim, Germany
2OSIRIS Department, EDF Lab Saclay, Palaisseau, France

Tóm tắt

AbstractWe develop a novel randomised block-coordinate primal-dual algorithm for a class of non-smooth ill-posed convex programs. Lying midway between the celebrated Chambolle–Pock primal-dual algorithm and Tseng’s accelerated proximal gradient method, we establish global convergence of the last iterate as well as optimal $O(1/k)$ O ( 1 / k ) and $O(1/k^{2})$ O ( 1 / k 2 ) complexity rates in the convex and strongly convex case, respectively, k being the iteration count. Motivated by the increased complexity in the control of distribution-level electric-power systems, we test the performance of our method on a second-order cone relaxation of an AC-OPF problem. Distributed control is achieved via the distributed locational marginal prices (DLMPs), which are obtained as dual variables in our optimisation framework.

Từ khóa


Tài liệu tham khảo

Facchinei, F., Kanzow, C.: Generalized Nash equilibrium problems. 4OR 5, 173–210 (2007)

Grasmair, M., Haltmeier, M., Scherzer, O.: The residual method for regularizing ill-posed problems. Appl. Math. Comput. 218(6), 2693–2710 (2011). https://doi.org/10.1016/j.amc.2011.08.009

Wood, A.J., Wollenberg, B.F., Sheblé, G.B.: Power Generation, Operation, and Control. Wiley, New York (2013)

Bienstock, D.: Electrical Transmission System Cascades and Vulnerability: An Operations Research Viewpoint. MOS-SIAM Series on Optimization (2015)

Papavasiliou, A.: Analysis of distribution locational marginal prices. IEEE Trans. Smart Grid 9(5), 4872–4882 (2018). https://doi.org/10.1109/TSG.2017.2673860

Nesterov, Y.: Efficiency of coordinate descent methods on huge-scale optimization problems. SIAM J. Optim. 22(2), 341–362 (2012)

Lu, Z., Xiao, L.: On the complexity analysis of randomized block-coordinate descent methods. Math. Program. 152(1), 615–642 (2015). https://doi.org/10.1007/s10107-014-0800-2

Richtárik, P., Takáč, M.: Iteration complexity of randomized block-coordinate descent methods for minimizing a composite function. Math. Program. 144(1), 1–38 (2014). https://doi.org/10.1007/s10107-012-0614-z

Richtárik, P., Takáč, M.: Parallel coordinate descent methods for big data optimization. Math. Program. 156(1–2), 433–484 (2016)

Glowinski, R., Marroco, A.: Sur l’approximation, par éléments finis d’ordre un, et la résolution, par pénalisation-dualitéd’une classe de problèmes de dirichlet non linéaires. Rev. Fr. Autom. Inform. Rech. Opér., Anal. Numér. 9(R2), 41–76 (1975)

Gabay, D., Mercier, B.: A dual algorithm for the solution of nonlinear variational problems via finite element approximation. Comput. Math. Appl. 2(1), 17–40 (1976). https://doi.org/10.1016/0898-1221(76)90003-1

Han, D., Sun, D., Zhang, L.: Linear rate convergence of the alternating direction method of multipliers for convex composite programming. Math. Oper. Res. 43(2), 622–637 (2017). https://doi.org/10.1287/moor.2017.0875

Eckstein, J., Yao, W.: Understanding the convergence of the alternating direction method of multipliers: theoretical and computational perspectives. Pac. J. Optim. 11(4), 619–644 (2015)

Chen, C., He, B., Ye, Y., Yuan, X.: The direct extension of admm for multi-block convex minimization problems is not necessarily convergent. Math. Program. 155(1), 57–79 (2016). https://doi.org/10.1007/s10107-014-0826-5

Gao, X., Xu, Y.-Y., Zhang, S.-Z.: Randomized primal–dual proximal block coordinate updates. J. Oper. Res. Soc. China 7(2), 205–250 (2019)

Gao, X., Zhang, S.-Z.: First-order algorithms for convex optimization with nonseparable objective and coupled constraints. J. Oper. Res. Soc. China 5(2), 131–159 (2017). https://doi.org/10.1007/s40305-016-0131-5

Deng, W., Lai, M.-J., Peng, Z., Yin, W.: Parallel multi-block ADMM with $o(1 / k)$ convergence. J. Sci. Comput. 71(2), 712–736 (2016). https://doi.org/10.1007/s10915-016-0318-2

Nesterov, Y.: A method of solving a convex programming problem with convergence rate $o(1/k^{2})$. Sov. Math. Dokl. 27(2), 372–376 (1983)

Beck, A., Teboulle, M.: A fast iterative shrinkage-thresholding algorithm for linear inverse problems. SIAM J. Imaging Sci. 2(1), 183–202 (2009). https://doi.org/10.1137/080716542

Nesterov, Y.: Gradient methods for minimizing composite functions. Math. Program. 140(1), 125–161 (2013). https://doi.org/10.1007/s10107-012-0629-5

Kang, M., Kang, M., Jung, M.: Inexact accelerated augmented Lagrangian methods. Comput. Optim. Appl. 62(2), 373–404 (2015). https://doi.org/10.1007/s10589-015-9742-8

Malitsky, Y.: The primal-dual hybrid gradient method reduces to a primal method for linearly constrained optimization problems (2019)

Chambolle, A., Pock, T.: A first-order primal-dual algorithm for convex problems with applications to imaging. J. Math. Imaging Vis. 40(1), 120–145 (2011). https://doi.org/10.1007/s10851-010-0251-1

Tseng, P.: On accelerated proximal gradient methods for convex-concave optimization (2008)

Malitsky, Y.: Chambolle-Pock and Tseng’s methods: relationship and extension to the bilevel optimization (2017)

Luke, D.R., Malitsky, Y.: Block-coordinate primal-dual method for nonsmooth minimization over linear constraints. In: Giselsson, P., Rantzer, A. (eds.) Large-Scale and Distributed Optimization, pp. 121–147. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-97478-1_6

Tran-Dinh, Q., Liu, D.: Faster randomized primal-dual algorithms for nonsmooth composite convex minimization. arXiv preprint (2020) arXiv:2003.01322

Xu, Y., Zhang, S.: Accelerated primal-dual proximal block coordinate updating methods for constrained convex optimization. Comput. Optim. Appl. 70(1), 91–128 (2018). https://doi.org/10.1007/s10589-017-9972-z

Tran-Dinh, Q., Liu, D.: A new randomized primal-dual algorithm for convex optimization with fast last iterate convergence rates. Optim. Methods Softw. 38(1), 184–217 (2023). https://doi.org/10.1080/10556788.2022.2119233

Farivar, M., Low, S.H.: Branch flow model: relaxations and convexification—part I. IEEE Trans. Power Syst. 28(3), 2554–2564 (2013). https://doi.org/10.1109/TPWRS.2013.2255317

Peng, Q., Low, S.H.: Distributed optimal power flow algorithm for radial networks, I: balanced single phase case. IEEE Trans. Smart Grid 9(1), 111–121 (2018). https://doi.org/10.1109/TSG.2016.2546305

Fercoq, O., Richtárik, P.: Accelerated, parallel, and proximal coordinate descent. SIAM J. Optim. 25(4), 1997–2023 (2015). https://doi.org/10.1137/130949993

Ryu, E.K., Yin, W.: Large-Scale Convex Optimization: Algorithms & Analyses via Monotone Operators. Cambridge University Press, Cambridge (2022). https://doi.org/10.1017/9781009160865

Qu, Z., Richtárik, P.: Coordinate descent with arbitrary sampling I: algorithms and complexity. Optim. Methods Softw. 31(5), 829–857 (2016). https://doi.org/10.1080/10556788.2016.1190360

Qu, Z., Richtárik, P.: Coordinate descent with arbitrary sampling II: expected separable overapproximation. Optim. Methods Softw. 31(5), 858–884 (2016). https://doi.org/10.1080/10556788.2016.1190361

Boyd, S., Parikh, N., Chu, E., Peleato, B., Eckstein, J.: Distributed optimization and statistical learning via the alternating direction method of multipliers. Found. Trends Mach. Learn. 3(1), 1–122 (2011)

Necoara, I., Clipici, D.: Efficient parallel coordinate descent algorithm for convex optimization problems with separable constraints: application to distributed MPC. J. Process Control 23(3), 243–253 (2013)

Bianchi, P., Hachem, W., Iutzeler, F.: A coordinate descent primal-dual algorithm and application to distributed asynchronous optimization. IEEE Trans. Autom. Control 61(10), 2947–2957 (2015)

Fercoq, O., Bianchi, P.: A coordinate-descent primal-dual algorithm with large step size and possibly nonseparable functions. SIAM J. Optim. 29(1), 100–134 (2019). https://doi.org/10.1137/18M1168480

Latafat, P., Freris, N.M., Patrinos, P.: A new randomized block-coordinate primal-dual proximal algorithm for distributed optimization. IEEE Trans. Autom. Control 64(10), 4050–4065 (2019)

Nesterov, Y.: Lectures on Convex Optimization. Springer Optimization and Its Applications, vol. 137. Springer, Berlin (2018)

d’Aspremont, A., Scieur, D., Taylor, A.: Acceleration methods. Found. Trends Optim. 5(1–2), 1–245 (2021)

Bilenne, O., Jacquot, P., Oudjane, N., Staudigl, M., Wan, C.: A privacy-preserving distributed computational approach for distributed locational marginal prices. In: 61st IEEE Conference on Decision and Control (2022)

Baran, M.E., Wu, F.F.: Optimal capacitor placement on radial distribution systems. IEEE Trans. Power Deliv. 4(1), 725–734 (1989). https://doi.org/10.1109/61.19265

Molzahn, D.K., Hiskens, I.A.: A survey of relaxations and approximations of the power flow equations. Found. Trends Electr. Energy Syst. 4(1–2), 1–221 (2019). https://doi.org/10.1561/3100000012

Gan, L., Li, N., Topcu, U., Low, S.H.: Exact convex relaxation of optimal power flow in radial networks. IEEE Trans. Autom. Control 60(1), 72–87 (2015). https://doi.org/10.1109/TAC.2014.2332712

Mezghani, I., Misra, S., Deka, D.: Stochastic ac optimal power flow: a data-driven approach. Electr. Power Syst. Res. 189, 106567 (2020)

Bauschke, H.H., Combettes, P.L.: Convex Analysis and Monotone Operator Theory in Hilbert Spaces. Springer, Berlin (2016)

Robbins, H., Siegmund, D.: A convergence theorem for non negative almost supermartingales and some applications. In: Rustagi, J.S. (ed.) Optimizing Methods in Statistics, pp. 233–257. Academic Press, San Diego (1971). https://doi.org/10.1016/B978-0-12-604550-5.50015-8. https://www.sciencedirect.com/science/article/pii/B9780126045505500158