Decentralized optimization with affine constraints over time-varying networks

Computational Management Science - Tập 21 - Trang 1-23 - 2023
Demyan Yarmoshik1,2, Alexander Rogozin1, Alexander Gasnikov1,3,2
1Moscow Institute of Physics and Technology, Dolgoprudny, Russia
2Institute for Information Transmission Problems, Moscow, Russia
3Skoltech, Moscow, Russia

Tóm tắt

The decentralized optimization paradigm assumes that each term of a finite-sum objective is privately stored by the corresponding agent. Agents are only allowed to communicate with their neighbors in the communication graph. We consider the case when the agents additionally have local affine constraints and the communication graph can change over time. We provide the first linearly convergent decentralized algorithm for time-varying networks by generalizing the optimal decentralized algorithm ADOM to the case of affine constraints. We show that its rate of convergence is optimal for first-order methods by providing the lower bounds for the number of communications and oracle calls.

Tài liệu tham khảo

Alghunaim S.A, Yuan K, Sayed A.H (2018) Dual coupled diffusion for distributed optimization with affine constraints. In: 2018 IEEE Conference on decision and control (CDC). IEEE, pp. 829–834 Aybat NS, Hamedani EY (2019) A distributed admm-like method for resource sharing over time-varying networks. SIAM J Optim 29(4):3036–3068 Carli R, Dotoli M (2019) Distributed alternating direction method of multipliers for linearly constrained optimization over a network. IEEE Control Syst Lett 4(1):247–252 Chang T-H (2016) A proximal dual consensus admm method for multi-agent constrained optimization. IEEE Trans Signal Process 64(14):3719–3734 Gong K, Zhang L (2023) Push-pull based distributed primal-dual algorithm for coupled constrained convex optimization in multi-agent networks. Available at SSRN 4109852 Huang Y, Cheng Y, Bapna A, Firat O, Chen D, Chen M, Lee H, Ngiam J, Le Q.V, Wu Y, et al.: (2019) Gpipe: efficient training of giant neural networks using pipeline parallelism. Advances in neural information processing systems, 32 Hu T.-K, Gama F, Chen T, Wang Z, Ribeiro A, Sadler B.M (2021) Vgai: End-to-end learning of vision-based decentralized controllers for robot swarms. In: ICASSP 2021-2021 IEEE international conference on acoustics, speech and signal processing (ICASSP). IEEE, pp. 4900–4904 Kovalev D, Gasanov E, Gasnikov A, Richtarik P (2021) Lower bounds and optimal algorithms for smooth and strongly convex decentralized optimization over time-varying networks. Advances in Neural Information Processing Systems, 34 Kovalev D, Shulgin E, Richtárik P, Rogozin A, Gasnikov A (2021) Adom: accelerated decentralized optimization method for time-varying networks. arXiv preprint arXiv:2102.09234 Li W, Tang R, Wang S, Zheng Z (2023) An optimal design method for communication topology of wireless sensor networks to implement fully distributed optimal control in iot-enabled smart buildings. Appl Energy 349:121539 Liang S, Yin G et al (2019) Distributed smooth convex optimization with coupled constraints. IEEE Trans Autom Control 65(1):347–353 Lian X, Zhang C, Zhang H, Hsieh C.-J, Zhang W, Liu J (2017) Can decentralized algorithms outperform centralized algorithms? A case study for decentralized parallel stochastic gradient descent. In: Advances in neural information processing systems, pp. 5330–5340 Molzahn DK, Dörfler F, Sandberg H, Low SH, Chakrabarti S, Baldick R, Lavaei J (2017) A survey of distributed optimization and control algorithms for electric power systems. IEEE Trans Smart Grid 8(6):2941–2962 Necoara I, Nedelcu V, Dumitrache I (2011) Parallel and distributed optimization methods for estimation and control in networks. J Process Control 21(5):756–766 Nedic A, Ozdaglar A, Parrilo PA (2010) Constrained consensus and optimization in multi-agent networks. IEEE Trans Autom Control 55(4):922–938 Nesterov Y (2004) Introductory lectures on convex optimization: a basic course. Kluwer Academic Publishers, Amsterdam Rogozin A, Yarmoshik D, Kopylova K, Gasnikov, A (2022) Decentralized strongly-convex optimization with affine constraints: Primal and dual approaches. arXiv preprint arXiv:2207.04555 Salim A, Condat L, Kovalev D, Richtárik P (2022) An optimal algorithm for strongly convex minimization under affine constraints. In: International conference on artificial intelligence and statistics, pp. 4482–4498 . PMLR Scaman K, Bach F, Bubeck S, Lee Y.T, Massoulié L (2017) Optimal algorithms for smooth and strongly convex distributed optimization in networks. In: Proceedings of the 34th international conference on machine learning-Volume 70 JMLR. org, pp. 3027–3036 Scutari G, Sun Y (2019) Distributed nonconvex constrained optimization over time-varying digraphs. Math Program 176(1):497–544 Scutari G, Facchinei F, Lampariello L (2016) Parallel and distributed methods for constrained nonconvex optimization-part i: theory. IEEE Trans Signal Process 65(8):1929–1944 Silva-Rodriguez J, Li X (2023) Privacy-preserving decentralized energy management for networked microgrids via objective-based admm. arXiv preprint arXiv:2304.03649 Wang J, Hu G (2022) Distributed optimization with coupling constraints in multi-cluster networks based on dual proximal gradient method. arXiv preprint arXiv:2203.00956 Wu X, Wang H, Lu J (2022) Distributed optimization with coupling constraints. IEEE Trans Autom Control 8(3):1847–1854 Yarmoshik D, Rogozin A, Khamisov O, Dvurechensky P, Gasnikov A, et al (2022) Decentralized convex optimization under affine constraints for power systems control. arXiv preprint arXiv:2203.16686 Zhou H, Lange K (2013) A path algorithm for constrained estimation. J Comput Graph Stat 22(2):261–283 Zhu M, Martinez S (2011) On distributed convex optimization under inequality and equality constraints. IEEE Trans Autom Control 57(1):151–164 Zhu F, Ren Y, Kong F, Wu H, Liang S, Chen N, Xu W, Zhang F (2023) Swarm-lio: Decentralized swarm lidar-inertial odometry. In: 2023 IEEE International conference on robotics and automation (ICRA). IEEE, pp. 3254–3260