MistNet: A superior edge-cloud privacy-preserving training framework with one-shot communication

K. He, X. Zhang, S. Ren, J. Sun, Identity mappings in deep residual networks, in: European Conference on Computer Vision, 2016, pp. 630–645. A. Graves, A.R. Mohamed, G. Hinton, Speech recognition with deep recurrent neural networks, in: 2013 IEEE international conference on acoustics, speech and signal processing, 2013, pp. 6645–6649. Bengio, 2003, A neural probabilistic language model, J. Mach. Learn. Res., 1137 Liu, 2021, Adaptive asynchronous federated learning in resource-constrained edge computing, IEEE Trans. Mob. Comput., 1 Voigt, 2017, The EU general data protection regulation (GDPR), 10 Sattler, 2019, Robust and communication-efficient federated learning from non-IID data, IEEE Trans. Neural Netw. Learn. Syst., 3400 Sezer, 2023, PPFchain: A novel framework privacy-preserving blockchain-based federated learning method for sensor networks, Internet Things, 10.1016/j.iot.2023.100781 Wang, 2022, Accelerating federated learning with cluster construction and hierarchical aggregation, IEEE Trans. Mob. Comput., 1 Konečnỳ, 2016 McMahan, 2017, Communicationefficient learning of deep networks from decentralized data, 1273 Yang, 2019, Federated machine learning: Concept and applications, ACM Trans. Intell. Syst. Technol., 1 Gupta, 2018, Distributed learning of deep neural network over multiple agents, J. Netw. Comput. Appl., 1 Vepakomma, 2018 J. Jeon, J. Kim, Privacy-sensitive parallel split learning, in: 2020 International Conference on Information Networking, ICOIN, 2020, pp. 7–9. K. Hsieh, A. Phanishayee, O. Mutlu, P. Gibbons, The non-IID data quagmire of decentralized machine learning, in: Proceedings of the 37th International Conference on Machine Learning, 2020, pp. 4387–4398. Y. Gao, M. Kim, S. Abuadbba, Y. Kim, C. Thapa, K. Kim, S.A. Camtep, H. Kim, S. Nepal, End-to-end evaluation of federated learning and split learning for internet of things, in: 2020 International Symposium on Reliable Distributed Systems, SRDS, 2020, pp. 91–100. Z. He, T. Zhang, R. Lee, Model Inversion Attacks against Collaborative Inference, in: Proceedings of the 35th Annual Computer Security Applications Conference, 2019, pp. 148–162. Park, 2019, An attack-based evaluation method for differentially private learning against model inversion attack, IEEE Access, 124988, 10.1109/ACCESS.2019.2938759 Yosinski, 2014, How transferable are features in deep neural networks?, 3320 J. Deng, W. Dong, R. Socher, L.J. Li, K. Li, Imagenet: A large-scale hierarchical image database, in: 2009 IEEE Conference on Computer Vision and Pattern Recognition, 2009, pp. 248–255. Warner, 1965, Randomized response: A survey technique for eliminating evasive answer bias, J. Amer. Statist. Assoc., 63, 10.1080/01621459.1965.10480775 Ú. Erlingsson, V. Pihur, A. Korolova, Rappor: Randomized aggregatable privacy-preserving ordinal response, in: Proceedings of the 2014 ACM SIGSAC Conference on Computer and Communications Security, 2014, pp. 1054–1067. 2019 Jiang, 2022, SignDS-FL: Local differentially private federated learning with sign-based dimension selection, ACM Trans. Intell. Syst. Technol., 1 Agarwal, 2021, The skellam mechanism for differentially private federated learning, Adv. Neural Inf. Process. Syst., 5052 Noble, 2022, Differentially private federated learning on heterogeneous data, 10110 Y. **ong, R. Wang, M. Cheng, F. Yu, C.J. Hsieh, Feddm: Iterative distribution matching for communication-efficient federated learning, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 16323–16332. Su, 2023, One-shot federated learning without server-side training, Neural Netw., 203, 10.1016/j.neunet.2023.04.035 X. Cao, J. Jia, Z. Zhang, N.Z. Gong, Fedrecover: Recovering from poisoning attacks in federated learning using historical information, in: 2023 IEEE Symposium on Security and Privacy, 2023, pp. 1366–1383. V. Turina, Z. Zhang, F. Esposito, I. Matta, Federated or Split? A Performance and Privacy Analysis of Hybrid Split and Federated Learning Architectures, in: 2021 IEEE 14th International Conference on Cloud Computing, CLOUD, 2021, pp. 250–260. Wang, 2023, CoopFL: Accelerating federated learning with DNN partitioning and offloading in heterogeneous edge computing, Comput. Netw. Zhang, 2023, Privacy and efficiency of communications in federated split learning, IEEE Trans. Big Data, 1380, 10.1109/TBDATA.2023.3280405 Yang, 2023, Dynamic corrected split federated learning with homomorphic encryption for U-shaped medical image networks, IEEE J. Biomed. Health Inf., 1 Kang, 2017, Neurosurgeon: Collaborative intelligence between the cloud and mobile edge, ACM SIGARCH Comput. Archit. News, 615, 10.1145/3093337.3037698 Arachchige, 2019, Local differential privacy for deep learning, IEEE Internet Things J., 5827 J. Mao, X. Chen, K. Nixon, C. Krieger, Y. Chen, Modnn: Local distributed mobile computing system for deep neural network, in: Design, Automation & Test in Europe Conference & Exhibition, DATE, 2017, pp. 1396–1401. H. Li, C. Hu, Chenghao, J. Jiang, Z. Wang, Y. Wen, W. Zhu, JALAD: Joint accuracy-and latency-aware deep structure decoupling for edge-cloud execution, in: 2018 IEEE 24th International Conference on Parallel and Distributed Systems, ICPADS, 2018, pp. 671–678. J. Ko, T. Na, M. Amir, S. Mukhopadhyay, Edge-host partitioning of deep neural networks with feature space encoding for resource-constrained internet-of-things platforms, in: 2018 15th IEEE International Conference on Advanced Video and Signal Based Surveillance, AVSS, 2018, pp. 1–6. H.J. Jeong, H.J. Lee, C.H. Shin, S.M. Moon, IONN: Incremental offloading of neural network computations from mobile devices to edge servers, in: Proceedings of the ACM Symposium on Cloud Computing, 2018, pp. 401–411. Wu, 2018 F. Mireshghallah, M. Taram, Mohammadkazem, P. Ramrakhyani, A. Jalali, D. Tullsen, H. Esmaeilzadeh, Shredder: Learning noise distributions to protect inference privacy, in: Proceedings of the Twenty-Fifth International Conference on Architectural Support for Programming Languages and Operating Systems, 2020, pp. 3–18. Osia, 2018, Deep private-feature extraction, IEEE Trans. Knowl. Data Eng., 54 J. Wang, J. Zhang, W. Bao, X. Zhu, B. Cao, P. Yu, Not just privacy: Improving performance of private deep learning in mobile cloud, in: Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, 2018, pp. 2407–2416. Y. Mao, S. Yi, Q. Li, Qun, J. Feng, F.Xu, S. Zhong, Learning from differentially private neural activations with edge computing, SEC, 2018, pp. 90–102. Li, 2017 Zhang, 2019 Li, 2019 C.C. Yao, How to generate and exchange secrets, in: 27th Annual Symposium on Foundations of Computer Science, sfcs 1986, 1986, pp. 162–167. P. Mohassel, Y. Zhang, Secureml: A system for scalable privacy-preserving machine learning, in: 2017 IEEE Symposium on Security and Privacy, SP, 2017, pp. 19–38. C. Gentry, Fully homomorphic encryption using ideal lattices, in: Proceedings of the forty-first annual ACM symposium on Theory of computing, 2009, pp. 169–178. R. Gilad-Bachrach, N. Dowlin, K. Laine, Kim, K. Lauter, M. Naehrig, Michael, J. Wernsing, Cryptonets: Applying neural networks to encrypted data with high throughput and accuracy, in: International Conference on Machine Learning, 2016, pp. 201–210. C. Juvekar, V. Vaikuntanathan, A. Chandrakasan, {GAZELLE}: A low latency framework for secure neural network inference, in: 27th {USENIX} Security Symposium, {USENIX} Security 18, 2018, pp. 1651–1699. Z. Gu, H. Jamjoom, D. Su, H. Huang, J. Zhang, T. Ma, D. Pendarakis, I. Molloy, Reaching data confidentiality and model accountability on the caltrain, in: 2019 49th Annual IEEE/IFIP International Conference on Dependable Systems and Networks, DSN, 2019, pp. 336–348. Tramer, 2018 Hunt, 2018 F. McKeen, I. Alexandrovich, A. Berenzon, C. Rozas, Carlos, H. Shafi, V. Shanbhogue, U. Savagaonkar, Innovative instructions and software model for isolated execution, in: HASP@ ISCA, Vol. 10, no. 1, 2013. Alves, 2004, Trustzone: Integrated hardware and software security, White Paper Kasiviswanathan, 2011, What can we learn privately?, SIAM J. Comput., 793, 10.1137/090756090 Bebensee, 2019 M. Fredrikson, S. Jha, T. Ristenpart, Model inversion attacks that exploit confidence information and basic countermeasures, in: Proceedings of the 22nd ACM SIGSAC Conference on Computer and Communications Security, 2015, pp. 1322–1333. R. Shokri, M. Stronati, C. Song, V. Shmatikov, Membership inference attacks against machine learning models, in: 2017 IEEE Symposium on Security and Privacy, SP, 2017, pp. 3–18. Ren, 2015, Faster R-CNN: Towards real-time object detection with region proposal networks, 91 A. Krizhevsky, G. Hinton, Learning multiple layers of features from tiny images, in: 28th {USENIX} Security Symposium, {USENIX} Security 19, 2009. Y. Netzer, T. Wang, A. Coates, A. Bissacco, B. Wu, A.Y. Ng, Reading digits in natural images with unsupervised feature learning, in: NIPS Workshop, 2011. Darlow, 2018 Mark, 2010, The PASCAL Visual Object Classes (VOC) challenge, Int. J. Comput. Vis., 303 T. Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, P. Dollar, C. Zitnick, Microsoft COCO: Common objects in context, in: European Conference on Computer Vision, ECCV, 2014, pp. 740–755. K. He, X. Zhang, S. Ren, J. Sun, Deep residual learning for image recognition, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016, pp. 770–778. Simonyan, 2014 C. Chen, X. Hong, W. Wei, B. Li, B. Li, L. Chen, G. Zhang, Communication-efficient federated learning with adaptive parameter freezing, in: 2021 IEEE 41st International Conference on Distributed Computing Systems, ICDCS, 2021, pp. 1–11. B. Jayaraman, D. Evans, Evaluating differentially private machine learning in practice, in: 28th {USENIX} Security Symposium, {USENIX} Security 19, 2019, pp. 1895–1912. Bernau, 2019