LayerCFL: an efficient federated learning with layer-wised clustering

citation_journal_title=Big Data Res; citation_title=Efficient machine learning for big data: a review; citation_author=OY Al-Jarrah, PD Yoo, S Muhaidat, GK Karagiannidis, K Taha; citation_volume=2; citation_issue=3; citation_publication_date=2015; citation_pages=87-93; citation_doi=10.1016/j.bdr.2015.04.001; citation_id=CR1 Chandran P, Bhat R, Chakravarthi A, Chandar S (2021) Weight divergence driven divide-and-conquer approach for optimal federated learning from non-iid data. arXiv:2106.14503 Cohen G, Afshar S, Tapson J, Van Schaik A (2017) Emnist: Extending mnist to handwritten letters. In: 2017 international joint conference on neural networks (IJCNN), IEEE, pp 2921–2926 citation_journal_title=Inform Fusion; citation_title=Non-iid data and continual learning processes in federated learning: a long road ahead; citation_author=MF Criado, FE Casado, R Iglesias, CV Regueiro, S Barro; citation_volume=88; citation_publication_date=2022; citation_pages=263-280; citation_doi=10.1016/j.inffus.2022.07.024; citation_id=CR4 Dennis DK, Li T, Smith V (2021) Heterogeneity for the win: One-shot federated clustering. In: International conference on machine learning, PMLR, pp 2611–2620 Gao C, Wang X, He X, Li Y (2022) Graph neural networks for recommender system. In: WSDM ’22: The fifteenth ACM international conference on Web search and data mining citation_journal_title=Adv Neural Inf Process Syst; citation_title=An efficient framework for clustered federated learning; citation_author=A Ghosh, J Chung, D Yin, K Ramchandran; citation_volume=33; citation_publication_date=2020; citation_pages=19,586-19,597; citation_id=CR7 citation_journal_title=Mobile Networks Appl; citation_title=Adaptive clustered federated learning for heterogeneous data in edge computing; citation_author=B Gong, T Xing, Z Liu, J Wang, X Liu; citation_volume=27; citation_issue=4; citation_publication_date=2022; citation_pages=1520-1530; citation_doi=10.1007/s11036-022-01978-8; citation_id=CR8 citation_journal_title=Foundations and Trends® in Machine Learning; citation_title=Advances and open problems in federated learning; citation_author=P Kairouz, HB McMahan, B Avent, A Bellet, M Bennis, AN Bhagoji, K Bonawitz, Z Charles, G Cormode, R Cummings; citation_volume=14; citation_issue=1–2; citation_publication_date=2021; citation_pages=1-210; citation_doi=10.1561/2200000083; citation_id=CR9 Kemp S (2022) Digital 2022: Global overview report, datareportal, 2022. https://datareportal.com/reports/digital-2022-global-overview-report Kim H, Kim Y, Park H (2021) Reducing model cost based on the weights of each layer for federated learning clustering. In: 2021 Twelfth International Conference on Ubiquitous and Future Networks (ICUFN), IEEE, pp 405–408 Krizhevsky A, Hinton G (2009) Learning multiple layers of features from tiny images. University of Toronto, Tech Rep, Computer Science Department, p 1 Li Q, Diao Y, Chen Q, He B (2022) Federated learning on non-iid data silos: An experimental study. In: 2022 IEEE 38th International Conference on Data Engineering (ICDE), pp 965–978, https://doi.org/10.1109/ICDE53745.2022.00077 citation_journal_title=IEEE Signal Process Mag; citation_title=Federated learning: challenges, methods, and future directions; citation_author=T Li, AK Sahu, A Talwalkar, V Smith; citation_volume=37; citation_issue=3; citation_publication_date=2020; citation_pages=50-60; citation_doi=10.1109/MSP.2020.2975749; citation_id=CR14 citation_journal_title=Proc Mach Learn Syst; citation_title=Federated optimization in heterogeneous networks; citation_author=T Li, AK Sahu, M Zaheer, M Sanjabi, A Talwalkar, V Smith; citation_volume=2; citation_publication_date=2020; citation_pages=429-450; citation_id=CR15 Li X, Huang K, Yang W, Wang S, Zhang Z (2019) On the convergence of fedavg on non-iid data. arXiv:1907.02189 citation_journal_title=ACM Comput Surv (CSUR); citation_title=When machine learning meets privacy: a survey and outlook; citation_author=B Liu, M Ding, S Shaham, W Rahayu, F Farokhi, Z Lin; citation_volume=54; citation_issue=2; citation_publication_date=2021; citation_pages=1-36; citation_doi=10.1145/3436755; citation_id=CR17 citation_journal_title=Proc AAAI Conf Artif Intell; citation_title=Federated learning for vision-and-language grounding problems; citation_author=F Liu, X Wu, S Ge, W Fan, Y Zou; citation_volume=34; citation_publication_date=2020; citation_pages=11572-11579; citation_id=CR18 M L, Y C, Z C (2018) Transferable representation learning with deep adaptation networks. IEEE Trans Pattern Anal Mach Intell Ma X, Zhang J, Guo S, Xu W (2022a) Layer-wised model aggregation for personalized federated learning citation_title=A state-of-the-art survey on solving non-iid data in federated learning; citation_publication_date=2022; citation_id=CR21; citation_author=X Ma; citation_author=J Zhu; citation_author=Z Lin; citation_author=S Chen; citation_author=Y Qin; citation_publisher=FGCS Mansour Y, Mohri M, Ro J, Suresh AT (2020) Three approaches for personalization with applications to federated learning. Computer Science McMahan B, Moore E, Ramage D, Hampson S, y Arcas BA (2017) Communication-efficient learning of deep networks from decentralized data. In: Artificial intelligence and statistics, PMLR, pp 1273–1282 McMahan HB, Moore E, Ramage D, y Arcas BA (2016) Federated learning of deep networks using model averaging. arXiv preprint arXiv:1602.05629 2 citation_journal_title=Futur Gener Comput Syst; citation_title=A survey on security and privacy of federated learning; citation_author=V Mothukuri, RM Parizi, S Pouriyeh, Y Huang, A Dehghantanha, G Srivastava; citation_volume=115; citation_publication_date=2021; citation_pages=619-640; citation_doi=10.1016/j.future.2020.10.007; citation_id=CR25 OpenAI (2022) Chatgpt: optimizing language models for dialogue. https://openai.com/blog/chatgpt/ Ouyang X, Xie Z, Zhou J, Huang J, Xing G (2021) Clusterfl: a similarity-aware federated learning system for human activity recognition. In: MobiSys ’21: The 19th Annual International Conference on Mobile Systems, Applications, and Services Papernot N, McDaniel P, Sinha A, Wellman M (2016) Towards the science of security and privacy in machine learning. arXiv preprint arXiv:1611.03814 citation_journal_title=IEEE Transactions on Neural Networks and Learning Systems; citation_title=Clustered federated learning: Model-agnostic distributed multitask optimization under privacy constraints; citation_author=F Sattler, KR Muller, W Samek; citation_volume=PP; citation_issue=99; citation_publication_date=2020; citation_pages=1-13; citation_id=CR29 Silva S, Gutman BA, Romero E, Thompson PM, Altmann A, Lorenzi M (2019) Federated learning in distributed medical databases: Meta-analysis of large-scale subcortical brain data. In: 2019 IEEE 16th international symposium on biomedical imaging (ISBI 2019), IEEE, pp 270–274 Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. Computer Science Voigt P, Von dem Bussche A (2017) The eu general data protection regulation (gdpr). A Practical Guide, 1st Ed, Cham: Springer International Publishing 10(3152676):10–5555 Wang H, Kaplan Z, Niu D, Li B (2020) Optimizing federated learning on non-iid data with reinforcement learning. In: IEEE INFOCOM 2020 - IEEE Conference on Computer Communications citation_journal_title=IEEE Network; citation_title=In-edge ai: Intelligentizing mobile edge computing, caching and communication by federated learning; citation_author=X Wang, Y Han, C Wang, Q Zhao, M Chen; citation_volume=PP; citation_issue=99; citation_publication_date=2019; citation_pages=1-10; citation_id=CR34 Wu C, Wu F, Cao Y, Huang Y, Xie X (2021) Fedgnn: Federated graph neural network for privacy-preserving recommendation. arXiv preprint arXiv:2102.04925 citation_journal_title=ACM Trans Knowl Discov Data just Accepted; citation_title=Personalized federated learning on non-iid data via group-based meta-learning; citation_author=L Yang, J Huang, W Lin, J Cao; citation_publication_date=2022; citation_doi=10.1145/3558005; citation_id=CR36 citation_journal_title=ACM Transactions on Intelligent Systems and Technology; citation_title=Federated machine learning: Concept and applications; citation_author=Q Yang, Y Liu, T Chen, Y Tong; citation_volume=10; citation_issue=2; citation_publication_date=2019; citation_pages=1-19; citation_doi=10.1145/3298981; citation_id=CR37 Yang W, Zhang Y, Ye K, Li L, Xu CZ (2019b) Ffd: A federated learning based method for credit card fraud detection. In: Big Data–BigData 2019: 8th International Congress, Held as Part of the Services Conference Federation, SCF 2019, San Diego, CA, USA, June 25–30, 2019, Proceedings 8, Springer, pp 18–32 citation_title=How transferable are features in deep neural networks?; citation_publication_date=2014; citation_id=CR39; citation_author=J Yosinski; citation_author=J Clune; citation_author=Y Bengio; citation_author=H Lipson; citation_publisher=MIT Press Zeiler M, Fergus R (2014) Visualizing and understanding convolutional networks. In: ECCV 2014 Zhao Y, Li M, Lai L, Suda N, Civin D, Chandra V (2018) Federated learning with non-iid data. arXiv preprint arXiv:1806.00582 citation_journal_title=Neurocomputing; citation_title=Machine learning on big data: Opportunities and challenges; citation_author=L Zhou, S Pan, J Wang, AV Vasilakos; citation_volume=237; citation_publication_date=2017; citation_pages=350-361; citation_doi=10.1016/j.neucom.2017.01.026; citation_id=CR42 Zhu H, Xu J, Liu S, Jin Y (2021) Federated learning on non-iid data: A survey. Neurocomputing 465:371–390 https://doi.org/10.1016/j.neucom.2021.07.098 , www.sciencedirect.com/science/article/pii/S0925231221013254