Differential privacy in deep learning: Privacy and beyond

Xu, 2019, EdgeSanitizer: Locally differentially private deep inference at the edge for mobile data analytics, IEEE Internet Things J., 6, 5140, 10.1109/JIOT.2019.2897005 Huang, 2019, An efficient differential privacy logistic classification mechanism, IEEE Internet Things J., 6, 10620, 10.1109/JIOT.2019.2940103 J. Jia, N.Z. Gong, Calibrate: Frequency estimation and heavy hitter identification with local differential privacy via incorporating prior knowledge, in: Proc. of IEEE INFOCOM, 2019. Ghane, 2020, TGM: A generative mechanism for publishing trajectories with differential privacy, IEEE Internet Things J., 7, 2611, 10.1109/JIOT.2019.2943719 Usman, 2020, Paal: A framework based on authentication, aggregation, and local differential privacy for internet of multimedia things, IEEE Internet Things J., 7, 2501, 10.1109/JIOT.2019.2936512 Gao, 2020, Dpdt: A differentially private crowd-sensed data trading mechanism, IEEE Internet Things J., 7, 751, 10.1109/JIOT.2019.2944107 Nie, 2020, Differentially private tensor train decomposition in edge-cloud computing for SDN-based internet of things, IEEE Internet Things J., 7, 5695, 10.1109/JIOT.2019.2960293 R. Shokri, M. Stronati, C. Song, V. Shmatikov, Membership inference attacks against machine learning models, in: Proc. of IEEE S&P, 2017. A. Friedman, A. Schuster, Data mining with differential privacy, in: Proc. of ACM SIGKDD, 2010. C. Li, M. Hay, V. Rastogi, G. Miklau, A. McGregor, Optimizing linear counting queries under differential privacy, in: Proc. of ACM SIGMOD-SIGACT-SIGART Symposium on Principles of Database Systems, 2010. C. Dwork, Differential privacy: A survey of results, in: Proc. of Theory and Applications of Models of Computation, TAMC, 2008. M. Abadi, A. Chu, I. Goodfellow, H.B. McMahan, I. Mironov, K. Talwar, L. Zhang, Deep learning with differential privacy, in: Proc. of ACM CCS, 2016. B. Jayaraman, D. Evans, Evaluating differentially private machine learning in practice, in: Proc. of USENIX Security Symposium, 2019. N. Papernot, M. Abadi, U. Erlingsson, I. Goodfellow, K. Talwar, Semi-supervised knowledge transfer for deep learning from private training data, in: Proc. of ICLR, 2017. M. Lecuyer, V. Atlidakis, R. Geambasu, D. Hsu, S. Jana, Certified robustness to adversarial examples with differential privacy, in: Proc. of IEEE S&P, 2019. E. Bagdasaryan, O. Poursaeed, V. Shmatikov, Differential privacy has disparate impact on model accuracy, in: Proc. of NIPS, 2019. M. Nasr, S. Song, A. Thakurta, N. Papernot, N. Carlini, Adversary instantiation: Lower bounds for differentially private machine learning, in: Proc. of IEEE S&P, 2021. Ú. Erlingsson, V. Pihur, A. Korolova, Rappor: Randomized aggregatable privacy-preserving ordinal response, in: Proc. of ACM CCS, 2014. Google, 2019 T. Farrand, F. Mireshghallah, S. Singh, A. Trask, Neither private nor fair: Impact of data imbalance on utility and fairness in differential privacy, in: Proc. of the Workshop on Privacy-Preserving Machine Learning in Practice, 2020. C. Tran, F. Fioretto, P. Van Hentenryck, Differentially private and fair deep learning: A lagrangian dual approach, in: Proc. of AAAI, 2021. C. Dwork, V. Feldman, M. Hardt, T. Pitassi, O. Reingold, A.L. Roth, Preserving statistical validity in adaptive data analysis, in: Proc. of ACM Symposium on Theory of Computing, 2015. S. Yeom, I. Giacomelli, M. Fredrikson, S. Jha, Privacy risk in machine learning: Analyzing the connection to overfitting, in: Proc. of IEEE Computer Security Foundations Symposium, CSF, 2018. B. Wu, S. Zhao, G. Sun, X. Zhang, Z. Su, C. Zeng, Z. Liu, P3sgd: Patient privacy preserving sgd for regularizing deep CNNs in pathological image classification, in: Proc. of IEEE/CVF CVPR, 2019. N. Phan, M. Vu, Y. Liu, R. Jin, D. Dou, X. Wu, M.T. Thai, Heterogeneous Gaussian mechanism: Preserving differential privacy in deep learning with provable robustness, in: Proc. of IJCAI, 2019. Xu, 2020 M. Du, R. Jia, D. Song, Robust anomaly detection and backdoor attack detection via differential privacy, in: Proc. of ICLR, 2020. J. Giraldo, A. Cardenas, M. Kantarcioglu, J. Katz, Adversarial classification under differential privacy, in: Proc. of NDSS, 2020. Hossain, 2021 Zhou, 2019, Achieving differentially private location privacy in edge-assistant connected vehicles, IEEE Internet Things J., 6, 4472, 10.1109/JIOT.2018.2876419 Li, 2020, PAPU: Pseudonym swap with provable unlinkability based on differential privacy in VANETs, IEEE Internet Things J., 7, 11789, 10.1109/JIOT.2020.3001381 X. Lou, R. Tan, D.K. Yau, P. Cheng, Cost of differential privacy in demand reporting for smart grid economic dispatch, in: Proc. of IEEE INFOCOM, 2017. A. Ghosh, J. Ding, R. Sarkar, J. Gao, Differentially private range counting in planar graphs for spatial sensing, in: Proc. of IEEE INFOCOM, 2020. Wang, 2018, A differentially private unscented Kalman filter for streaming data in IoT, IEEE Access, 6, 6487, 10.1109/ACCESS.2018.2797159 T. Gao, F. Li, PHDP: Preserving persistent homology in differentially private graph publications, in: Proc. of IEEE INFOCOM, 2019. A. Salem, Y. Zhang, M. Humbert, P. Berrang, M. Fritz, M. Backes, Ml-leaks: Model and data independent membership inference attacks and defenses on machine learning models, in: Proc. of NDSS, 2019. M. Fredrikson, E. Lantz, S. Jha, S. Lin, D. Page, T. Ristenpart, Privacy in pharmacogenetics: An end-to-end case study of personalized Warfarin dosing, in: Proc. of USENIX Security Symposium, 2014. S. Mehnaz, S.V. Dibbo, R. De Viti, E. Kabir, B.B. Brandenburg, S. Mangard, N. Li, E. Bertino, M. Backes, E. De Cristofaro, et al., Are your sensitive attributes private? Novel model inversion attribute inference attacks on classification models, in: Proc. of USENIX Security Symposium, 2022. M. Fredrikson, S. Jha, T. Ristenpart, Model inversion attacks that exploit confidence information and basic countermeasures, in: Proc. of ACM CCS, 2015. E. Cohen, H. Kaplan, Y. Mansour, U. Stemmer, E. Tsfadia, Differentially-private clustering of easy instances, in: Proc. of ICML, 2021. M. Bun, M. Eliáš, J. Kulkarni, Differentially private correlation clustering, in: Proc. of ICML, 2021. M. Jones, H.L. Nguyen, T.D. Nguyen, Differentially private clustering via maximum coverage, in: Proc. of AAAI, 2021. H.L. Nguyen, A. Chaturvedi, E.Z. Xu, Differentially private k-Means via exponential mechanism and max cover, in: Proc. of AAAI, 2021. A. Chaturvedi, H. Nguyen, L. Zakynthinou, Differentially private decomposable submodular maximization, in: Proc. of AAAI, 2021. J. Imola, T. Murakami, K. Chaudhuri, Locally differentially private analysis of graph statistics, in: Proc. of USENIX Security Symposium, 2021. A. De, S. Chakrabarti, Differentially private link prediction with protected connections, in: Proc. of AAAI, 2021. D. Nguyen, A. Vullikanti, Differentially private densest subgraph detection, in: Proc. of ICML, 2021. C. Yang, H. Wang, K. Zhang, L. Chen, L. Sun, Secure deep graph generation with link differential privacy, in: Proc. of IJCAI, 2021. S. Gopi, P. Gulhane, J. Kulkarni, J.H. Shen, M. Shokouhi, S. Yekhanin, Differentially private set union, in: Proc. of ICML, 2020. Zhao, 2021, Latent dirichlet allocation model training with differential privacy, IEEE Trans. Inf. Forensics Secur., 16, 1290, 10.1109/TIFS.2020.3032021 H. Kaplan, Y. Mansour, Y. Matias, U. Stemmer, Differentially private learning of geometric concepts, in: Proc. of ICML, 2019. M. Aliakbarpour, I. Diakonikolas, R. Rubinfeld, Differentially private identity and equivalence testing of discrete distributions, in: Proc. of ICML, 2018. M. Huai, D. Wang, C. Miao, J. Xu, A. Zhang, Pairwise learning with differential privacy guarantees, in: Proc. of AAAI, 2020. X. Zhou, J. Tan, Local differential privacy for Bayesian optimization, in: Proc. of AAAI, 2021. Li, 2018, Differentially private distributed online learning, IEEE Trans. Knowl. Data Eng., 30, 1440, 10.1109/TKDE.2018.2794384 J. Abernethy, Y.H. Jung, C. Lee, A. McMillan, A. Tewari, Online learning via the differential privacy lens, in: Proc. of NIPS, 2019. A. Gonen, E. Hazan, S. Moran, Private learning implies online learning: An efficient reduction, in: Proc. of NIPS, 2019. H. Liu, J. Jia, N.Z. Gong, On the intrinsic differential privacy of bagging, in: Proc. of IJCAI, 2021. M. Bun, M.L. Carmosino, J. Sorrell, Efficient, noise-tolerant, and private learning via boosting, in: Proc. of Conference on Learning Theory, COLT, 2020. H. Nori, R. Caruana, Z. Bu, J.H. Shen, J. Kulkarni, Accuracy, interpretability, and differential privacy via explainable boosting, in: Proc. of ICML, 2021. D. Wang, J. Xu, Differentially private empirical risk minimization with smooth non-convex loss functions: A non-stationary view, in: Proc. of AAAI, 2019. Chaudhuri, 2011, Differentially private empirical risk minimization, J. Mach. Learn. Res., 12 D. Wang, J. Xu, On sparse linear regression in the local differential privacy model, in: Proc. of ICML, 2019. A. Rakotomamonjy, R. Liva, Differentially private sliced Wasserstein distance, in: Proc. of ICML, 2021. Ryffel, 2018 Zhao, 2019, Differential privacy preservation in deep learning: Challenges, opportunities and solutions, IEEE Access, 7, 48901, 10.1109/ACCESS.2019.2909559 Yang, 2020 Zhu, 2020 Wang, 2020, Deep domain adaptation with differential privacy, IEEE Trans. Inf. Forensics Secur., 15, 3093, 10.1109/TIFS.2020.2983254 Q. Zheng, J. Dong, Q. Long, W. Su, Sharp composition bounds for Gaussian differential privacy via edgeworth expansion, in: Proc. of ICML, 2021. F. Fioretto, C. Tran, P. Van Hentenryck, Decision making with differential privacy under a fairness lens, in: Proc. of IJCAI, 2021. Ji, 2014 L. Fan, A survey of differentially private generative adversarial networks, in: Proc. of AAAI Workshop on Privacy-Preserving Artificial Intelligence, 2020. Zhao, 2020, A survey of local differential privacy for securing Internet of Vehicles, J. Supercomput., 76, 8391, 10.1007/s11227-019-03104-0 L. Zhao, L. Ni, S. Hu, Y. Chen, P. Zhou, F. Xiao, L. Wu, Inprivate digging: Enabling tree-based distributed data mining with differential privacy, in: Proc. of IEEE INFOCOM, 2018. Y. Qu, S. Yu, L. Gao, S. Peng, Y. Xiang, L. Xiao, FuzzyDP: Fuzzy-based big data publishing against inquiry attacks, in: Proc. of IEEE INFOCOM Workshops, 2017. S. Wang, L. Huang, Y. Nie, P. Wang, H. Xu, W. Yang, Privset: Set-valued data analyses with locale differential privacy, in: Proc. of IEEE INFOCOM, 2018. C. Dwork, Differential privacy, in: Proc. of International Colloquium on Automata, Languages, and Programming, 2006. I. Mironov, Rényi differential privacy, in: Proc. of IEEE Computer Security Foundations Symposium, CSF, 2017. Sarathy, 2011, Evaluating Laplace noise addition to satisfy differential privacy for numeric data, Trans. Data Priv., 4, 1 Liu, 2018, Generalized Gaussian mechanism for differential privacy, IEEE Trans. Knowl. Data Eng., 31, 747, 10.1109/TKDE.2018.2845388 B. Balle, Y.-X. Wang, Improving the Gaussian mechanism for differential privacy: Analytical calibration and optimal denoising, in: Proc. of ICML, 2018. B. Ding, J. Kulkarni, S. Yekhanin, Collecting telemetry data privately, in: Proc. of NIPS, 2017. X. Cao, J. Jia, N.Z. Gong, Data poisoning attacks to local differential privacy protocols, in: Proc. of USENIX Security Symposium, 2021. A. Cheu, A. Smith, J. Ullman, Manipulation attacks in local differential privacy, in: Proc. of IEEE S&P, 2021. J. Acharya, K. Bonawitz, P. Kairouz, D. Ramage, Z. Sun, Context aware local differential privacy, in: Proc. of ICML, 2020. F. McSherry, K. Talwar, Mechanism design via differential privacy, in: Proc. of IEEE Symposium on Foundations of Computer Science, FOCS, 2007. F.D. McSherry, Privacy integrated queries: An extensible platform for privacy-preserving data analysis, in: Proc. of ACM SIGMOD, 2009. B. Bichsel, T. Gehr, D. Drachsler-Cohen, P. Tsankov, M. Vechev, Dp-finder: Finding differential privacy violations by sampling and optimization, in: Proc. of ACM CCS, 2018. B. Bichsel, S. Steffen, I. Bogunovic, M. Vechev, DP-Sniper: Black-box discovery of differential privacy violations using classifiers, in: Proc. of IEEE S&P, 2021. C. Dwork, G.N. Rothblum, S. Vadhan, Boosting and differential privacy, in: Proc. of IEEE Annual Symposium on Foundations of Computer Science, 2010. K. Zhu, P. Van Hentenryck, F. Fioretto, Bias and variance of post-processing in differential privacy, in: Proc. of AAAI, 2021. R. Shokri, V. Shmatikov, Privacy-preserving deep learning, in: Proc. of ACM CCS, 2015. H.B. McMahan, D. Ramage, K. Talwar, L. Zhang, Learning differentially private recurrent language models, in: Proc. of ICLR, 2018. N. Phan, X. Wu, H. Hu, D. Dou, Adaptive laplace mechanism: Differential privacy preservation in deep learning, in: Proc. of ICDM, 2017. N. Papernot, A. Thakurta, S. Song, S. Chien, U. Erlingsson, Tempered sigmoid activations for deep learning with differential privacy, in: Proc. of AAAI, 2021. N. Agarwal, A.T. Suresh, F. Yu, S. Kumar, H.B. Mcmahan, cpSGD: Communication-efficient and differentially-private distributed SGD, in: Proc. of NIPS, 2018. McMahan, 2018 L. Xiang, J. Yang, B. Li, Differentially-private deep learning from an optimization perspective, in: Proc. of IEEE INFOCOM, 2019. L. Yu, L. Liu, C. Pu, M.E. Gursoy, S. Truex, Differentially private model publishing for deep learning, in: Proc. of IEEE S&P, 2019. Bu, 2020, Deep learning with Gaussian differential privacy, Harvard Data Sci. Rev., 2020 L. Jiang, X. Lou, R. Tan, J. Zhao, Differentially private collaborative learning for the IoT edge, in: Proc. of International Conference on Embedded Wireless Systems and Networks, EWSN, 2019. Wei, 2020, Federated learning with differential privacy: Algorithms and performance analysis, IEEE Trans. Inf. Forensics Secur., 15, 3454, 10.1109/TIFS.2020.2988575 S. Truex, L. Liu, K.-H. Chow, M.E. Gursoy, W. Wei, LDP-Fed: Federated learning with local differential privacy, in: Proc. of ACM International Workshop on Edge Systems, Analytics and Networking, 2020. Y. Wang, Y. Tong, D. Shi, Federated latent dirichlet allocation: A local differential privacy based framework, in: Proc. of AAAI, 2020. Xie, 2018 Zhang, 2018 Xu, 2019, GANobfuscator: Mitigating information leakage under GAN via differential privacy, IEEE Trans. Inf. Forensics Secur., 14, 2358, 10.1109/TIFS.2019.2897874 R. Torkzadehmahani, P. Kairouz, B. Paten, Dp-cgan: Differentially private synthetic data and label generation, in: Proc. of CVPR Workshops, 2019. Huang, 2020, DP-ADMM: ADMM-based distributed learning with differential privacy, IEEE Trans. Inf. Forensics Secur., 15, 1002, 10.1109/TIFS.2019.2931068 Phan, 2017, Preserving differential privacy in convolutional deep belief networks, Mach. Learn., 106, 1681, 10.1007/s10994-017-5656-2 J. Li, M. Khodak, S. Caldas, A. Talwalkar, Differentially private meta-learning, in: Proc. of ICLR, 2020. J. Jordon, J. Yoon, M. Van Der Schaar, PATE-GAN: Generating synthetic data with differential privacy guarantees, in: Proc. of ICLR, 2019. G. Damaskinos, C. Mendler-Dünner, R. Guerraoui, N. Papandreou, T. Parnell, Differentially private stochastic coordinate descent, in: Proc. of AAAI, 2021. Domingo-Ferrer, 2021, The limits of differential privacy (and its misuse in data release and machine learning), Commun. ACM, 64, 33, 10.1145/3433638 Z. Xu, S. Shi, A.X. Liu, J. Zhao, L. Chen, An adaptive and fast convergent approach to differentially private deep learning, in: Proc. of IEEE INFOCOM, 2020. Dong, 2019 D. Yu, H. Zhang, W. Chen, T.-Y. Liu, J. Yin, Gradient perturbation is underrated for differentially private convex optimization, in: Proc. of IJCAI, 2020. M. Jagielski, J. Ullman, A. Oprea, Auditing differentially private machine learning: How private is private SGD?, in: Proc. of NIPS, 2020. S. Truex, N. Baracaldo, A. Anwar, T. Steinke, H. Ludwig, R. Zhang, Y. Zhou, A hybrid approach to privacy-preserving federated learning, in: Proc. of ACM Workshop on Artificial Intelligence and Security, 2019. Goodfellow, 2014 B. Hitaj, G. Ateniese, F. Perez-Cruz, Deep models under the GAN: Information leakage from collaborative deep learning, in: Proc. of ACM CCS, 2017. Konečnỳ, 2016 Geyer, 2017 M. Hao, H. Li, G. Xu, S. Liu, H. Yang, Towards efficient and privacy-preserving federated deep learning, in: Proc. of IEEE International Conference on Communications, ICC, 2019. Rodríguez-Barroso, 2023, Survey on federated learning threats: Concepts, taxonomy on attacks and defences, experimental study and challenges, Inf. Fusion, 90, 148, 10.1016/j.inffus.2022.09.011 Rodríguez-Barroso, 2020, Federated learning and differential privacy: Software tools analysis, the Sherpa. AI FL framework and methodological guidelines for preserving data privacy, Inf. Fusion, 64, 270, 10.1016/j.inffus.2020.07.009 Naseri, 2020 Choudhury, 2019 L. Sun, L. Lyu, Federated model distillation with noise-free differential privacy, in: Proc. of IJCAI, 2021. Hu, 2020, Personalized federated learning with differential privacy, IEEE Internet Things J., 7, 9530, 10.1109/JIOT.2020.2991416 L. Sun, J. Qian, X. Chen, P.S. Yu, Ldp-fl: Practical private aggregation in federated learning with local differential privacy, in: Proc. of IJCAI, 2021. H. Phan, M.T. Thai, H. Hu, R. Jin, T. Sun, D. Dou, Scalable differential privacy with certified robustness in adversarial learning, in: Proc. of ICML, 2020. Farokhi, 2021, The cost of privacy in asynchronous differentially-private machine learning, IEEE Trans. Inf. Forensics Secur., 16, 2118, 10.1109/TIFS.2021.3050603 Hynes, 2018 Beaulieu-Jones, 2018 L. Fan, Image pixelization with differential privacy, in: Proc. of IFIP Annual Conference on Data and Applications Security and Privacy, 2018. F. Tramèr, D. Boneh, Differentially private learning needs better features (or much more data), in: Proc. of ICLR, 2021. Arachchige, 2020, Local differential privacy for deep learning, IEEE Internet Things J., 7, 5827, 10.1109/JIOT.2019.2952146 R. Cummings, V. Gupta, D. Kimpara, J. Morgenstern, On the compatibility of privacy and fairness, in: Proc. of Adjunct Publication of the 27th Conference on User Modeling, Adaptation and Personalization, 2019. J. Ding, X. Zhang, X. Li, J. Wang, R. Yu, M. Pan, Differentially private and fair classification via calibrated functional mechanism, in: Proc. of AAAI, 2020. Padala, 2021