Decentralized optimization with affine constraints over time-varying networks
Tóm tắt
The decentralized optimization paradigm assumes that each term of a finite-sum objective is privately stored by the corresponding agent. Agents are only allowed to communicate with their neighbors in the communication graph. We consider the case when the agents additionally have local affine constraints and the communication graph can change over time. We provide the first linearly convergent decentralized algorithm for time-varying networks by generalizing the optimal decentralized algorithm ADOM to the case of affine constraints. We show that its rate of convergence is optimal for first-order methods by providing the lower bounds for the number of communications and oracle calls.
Tài liệu tham khảo
Alghunaim S.A, Yuan K, Sayed A.H (2018) Dual coupled diffusion for distributed optimization with affine constraints. In: 2018 IEEE Conference on decision and control (CDC). IEEE, pp. 829–834
Aybat NS, Hamedani EY (2019) A distributed admm-like method for resource sharing over time-varying networks. SIAM J Optim 29(4):3036–3068
Carli R, Dotoli M (2019) Distributed alternating direction method of multipliers for linearly constrained optimization over a network. IEEE Control Syst Lett 4(1):247–252
Chang T-H (2016) A proximal dual consensus admm method for multi-agent constrained optimization. IEEE Trans Signal Process 64(14):3719–3734
Gong K, Zhang L (2023) Push-pull based distributed primal-dual algorithm for coupled constrained convex optimization in multi-agent networks. Available at SSRN 4109852
Huang Y, Cheng Y, Bapna A, Firat O, Chen D, Chen M, Lee H, Ngiam J, Le Q.V, Wu Y, et al.: (2019) Gpipe: efficient training of giant neural networks using pipeline parallelism. Advances in neural information processing systems, 32
Hu T.-K, Gama F, Chen T, Wang Z, Ribeiro A, Sadler B.M (2021) Vgai: End-to-end learning of vision-based decentralized controllers for robot swarms. In: ICASSP 2021-2021 IEEE international conference on acoustics, speech and signal processing (ICASSP). IEEE, pp. 4900–4904
Kovalev D, Gasanov E, Gasnikov A, Richtarik P (2021) Lower bounds and optimal algorithms for smooth and strongly convex decentralized optimization over time-varying networks. Advances in Neural Information Processing Systems, 34
Kovalev D, Shulgin E, Richtárik P, Rogozin A, Gasnikov A (2021) Adom: accelerated decentralized optimization method for time-varying networks. arXiv preprint arXiv:2102.09234
Li W, Tang R, Wang S, Zheng Z (2023) An optimal design method for communication topology of wireless sensor networks to implement fully distributed optimal control in iot-enabled smart buildings. Appl Energy 349:121539
Liang S, Yin G et al (2019) Distributed smooth convex optimization with coupled constraints. IEEE Trans Autom Control 65(1):347–353
Lian X, Zhang C, Zhang H, Hsieh C.-J, Zhang W, Liu J (2017) Can decentralized algorithms outperform centralized algorithms? A case study for decentralized parallel stochastic gradient descent. In: Advances in neural information processing systems, pp. 5330–5340
Molzahn DK, Dörfler F, Sandberg H, Low SH, Chakrabarti S, Baldick R, Lavaei J (2017) A survey of distributed optimization and control algorithms for electric power systems. IEEE Trans Smart Grid 8(6):2941–2962
Necoara I, Nedelcu V, Dumitrache I (2011) Parallel and distributed optimization methods for estimation and control in networks. J Process Control 21(5):756–766
Nedic A, Ozdaglar A, Parrilo PA (2010) Constrained consensus and optimization in multi-agent networks. IEEE Trans Autom Control 55(4):922–938
Nesterov Y (2004) Introductory lectures on convex optimization: a basic course. Kluwer Academic Publishers, Amsterdam
Rogozin A, Yarmoshik D, Kopylova K, Gasnikov, A (2022) Decentralized strongly-convex optimization with affine constraints: Primal and dual approaches. arXiv preprint arXiv:2207.04555
Salim A, Condat L, Kovalev D, Richtárik P (2022) An optimal algorithm for strongly convex minimization under affine constraints. In: International conference on artificial intelligence and statistics, pp. 4482–4498 . PMLR
Scaman K, Bach F, Bubeck S, Lee Y.T, Massoulié L (2017) Optimal algorithms for smooth and strongly convex distributed optimization in networks. In: Proceedings of the 34th international conference on machine learning-Volume 70 JMLR. org, pp. 3027–3036
Scutari G, Sun Y (2019) Distributed nonconvex constrained optimization over time-varying digraphs. Math Program 176(1):497–544
Scutari G, Facchinei F, Lampariello L (2016) Parallel and distributed methods for constrained nonconvex optimization-part i: theory. IEEE Trans Signal Process 65(8):1929–1944
Silva-Rodriguez J, Li X (2023) Privacy-preserving decentralized energy management for networked microgrids via objective-based admm. arXiv preprint arXiv:2304.03649
Wang J, Hu G (2022) Distributed optimization with coupling constraints in multi-cluster networks based on dual proximal gradient method. arXiv preprint arXiv:2203.00956
Wu X, Wang H, Lu J (2022) Distributed optimization with coupling constraints. IEEE Trans Autom Control 8(3):1847–1854
Yarmoshik D, Rogozin A, Khamisov O, Dvurechensky P, Gasnikov A, et al (2022) Decentralized convex optimization under affine constraints for power systems control. arXiv preprint arXiv:2203.16686
Zhou H, Lange K (2013) A path algorithm for constrained estimation. J Comput Graph Stat 22(2):261–283
Zhu M, Martinez S (2011) On distributed convex optimization under inequality and equality constraints. IEEE Trans Autom Control 57(1):151–164
Zhu F, Ren Y, Kong F, Wu H, Liang S, Chen N, Xu W, Zhang F (2023) Swarm-lio: Decentralized swarm lidar-inertial odometry. In: 2023 IEEE International conference on robotics and automation (ICRA). IEEE, pp. 3254–3260