Do gradient-based explanations tell anything about adversarial robustness to android malware?

Aafer Y, Du W, Yin H (2013) DroidAPIMiner: mining API-level features for robust malware detection in android. In: Proc. of international conference on security and privacy in communication networks (SecureComm). https://doi.org/10.1007/978-3-319-04283-1_6 Adadi A, Berrada M (2018) Peeking inside the black-box: a survey on explainable artificial intelligence (xai). IEEE Access 6:52138–52160 Allix K, Bissyandé TF, Klein J, Le Traon Y (2016) Androzoo: collecting millions of android apps for the research community. In: 2016 IEEE/ACM 13th working conference on mining software repositories (MSR), pp 468–471, IEEE Arp D, Spreitzenbarth M, Hübner M, Gascon H, Rieck K (2014) Drebin: efficient and explainable detection of android malware in your pocket. In: Proc. 21st annual network & distributed system security symposium (NDSS). The Internet Society Arzt S, Rasthofer S, Fritz C, Bodden E, Bartel A, Klein J, Le Traon Y, Octeau D, McDaniel P (2013) FlowDroid: precise context, flow, field, object-sensitive and lifecycle-aware taint analysis for Android apps. In: Proceedings of the 35th ACM SIGPLAN conference on programming language design and implementation—PLDI ’14, pp 259–269. ACM Press. https://doi.org/10.1145/2594291.2594299, http://dl.acm.org/citation.cfm?doid=2594291.2594299 Baehrens D, Schroeter T, Harmeling S, Kawanabe M, Hansen K, Müller KR (2010) How to explain individual classification decisions. J Mach Learn Res 11:1803–1831 Barreno M, Nelson B, Joseph A, Tygar J (2010) The security of machine learning. Mach Learn 81:121–148 Barreno M, Nelson B, Sears R, Joseph AD, Tygar JD (2006) Can machine learning be secure? In: Proc. ACM Symp. information, computer and comm. Sec., ASIACCS ’06, pp 16–25. ACM, New York Biggio B, Corona I, Maiorca D, Nelson B, Šrndić N, Laskov P, Giacinto G, Roli F (2013) Evasion attacks against machine learning at test time. In: Blockeel H, Kersting K, Nijssen S, Železný F (eds) Machine learning and knowledge discovery in databases (ECML PKDD), Part III, LNCS, vol 8190. Springer, Berlin, Heidelberg, pp 387–402 Biggio B, Fumera G, Roli F (2010) Multiple classifier systems for robust classifier design in adversarial environments. Int J Mach Learn Cybern 1(1):27–41 Biggio B, Fumera G, Roli F (2014) Security evaluation of pattern classifiers under attack. IEEE Trans Knowl Data Eng 26(4):984–996 Biggio B, Nelson B, Laskov P (2012) Poisoning attacks against support vector machines. In: Langford J, Pineau J (eds) 29th Int’l Conf. on Machine Learning, pp 1807–1814, Omnipress Biggio B, Roli F (2018) Wild patterns: ten years after the rise of adversarial machine learning. Pattern Recogn 84:317–331 Cai H, Meng N, Ryder B, Yao D (2018) Droidcat: effective android malware detection and categorization via app-level profiling. IEEE Trans Inf Forensics Secur 14(6):1455–1470 Calleja A, Martin A, Menendez HD, Tapiador J, Clark D (2018) Picking on the family: disrupting android malware triage by forcing misclassification. Expert Syst Appl 95:113–126 Cara F, Scalas M, Giacinto G, Maiorca D (2020) On the feasibility of adversarial sample creation using the android system api. Information 11(9):433 Chen J, Wang C, Zhao Z, Chen K, Du R, Ahn GJ (2018) Uncovering the Face of Android Ransomware: characterization and real-time detection. IEEE Trans Inf Forensics Secur 13(5):1286–1300. https://doi.org/10.1109/TIFS.2017.2787905, http://ieeexplore.ieee.org/document/8241433/ Chen J, Wu X, Rastogi V, Liang Y, Jha S (2019) Robust attribution regularization. Adv Neural Inf Process Syst 2019:14300–14310 Chen L, Hou S, Ye Y, Xu S (2018) Droideye: fortifying security of learning-based classifier against adversarial android malware attacks. In: Proceedings of the 2018 IEEE/ACM international conference on advances in social networks analysis and mining, ASONAM 2018, pp. 782–789. Institute of Electrical and Electronics Engineers Inc. https://doi.org/10.1109/ASONAM.2018.8508284 Chen S, Xue M, Tang Z, Xu L, Zhu H (2016) Stormdroid: a streaminglized machine learning-based system for detecting android malware. In: Proceedings of the 11th ACM on Asia conference on computer and communications security, pp 377–388 Chen YM, Yang CH, Chen GC (2021) Using generative adversarial networks for data augmentation in android malware detection. In: 2021 IEEE conference on dependable and secure computing (DSC), pp 1–8, IEEE. https://doi.org/10.1109/DSC49826.2021.9346277, https://ieeexplore.ieee.org/document/9346277/ Dalvi N, Domingos P, Mausam G, Sanghai S, Verma D (2004) Adversarial classification. In: Tenth ACM SIGKDD international conference on knowledge discovery and data mining (KDD), pp 99–108. Seattle Demontis A, Melis M, Biggio B, Maiorca D, Arp D, Rieck K, Corona I, Giacinto G, Roli F (2017) Yes, machine learning can be more secure! a case study on android malware detection. In: IEEE transactions on dependable and secure computing, pp 1–1. https://doi.org/10.1109/TDSC.2017.2700270 Demontis A, Melis, M., Pintor M, Jagielski M, Biggio B, Oprea A, Nita-Rotaru C, Roli F (2019) Why do adversarial attacks transfer? Explaining transferability of evasion and poisoning attacks. In: 28th USENIX Security Symposium (USENIX Security 19), pp 321–338. USENIX Association, Santa Clara Demontis A, Russu P, Biggio B, Fumera G, Roli F (2016) On security and sparsity of linear classifiers for adversarial settings. In: Robles-Kelly A, Loog M, Biggio B, Escolano F, Wilson R (eds) Joint IAPR Int’l workshop on structural, syntactic, and statistical pattern recognition, LNCS, vol 10029. Springer International Publishing, Cham, pp 322–332 Dombrowski AK, Alber M, Anders CJ, Ackermann M, Müller KR, Kessel P (2019) Explanations can be manipulated and geometry is to blame. arXiv:1906.07983 Feng Y, Anand S, Dillig I, Aiken A (2014) Apposcopy: semantics-based detection of Android malware through static analysis. In: Proceedings of the 22nd ACM SIGSOFT international symposium on foundations of software engineering—FSE 2014, pp 576–587. ACM Press. https://doi.org/10.1145/2635868.2635869, http://dl.acm.org/citation.cfm?doid=2635868.2635869 Fidel G, Bitton R, Shabtai A (2020) When explainability meets adversarial learning: Detecting adversarial examples using shap signatures. In: 2020 international joint conference on neural networks (IJCNN), pp 1–8, IEEE Goodfellow IJ, Shlens J, Szegedy C (2015) Explaining and harnessing adversarial examples. In: International conference on learning representations Goodman B, Flaxman S (2016) European Union regulations on algorithmic decision-making and a “right to explanation”. In: AI magazine, vol 38, pp 50–57 Grosse K, Papernot N, Manoharan P, Backes M, McDaniel PD (2017) Adversarial examples for malware detection. In: ESORICS (2), LNCS, vol 10493, pp 62–79. Springer Guo W, Mu D, Xu J, Su P, Wang G, Xing X (2018) Lemna: explaining deep learning based security applications. In: Proceedings of the 2018 ACM SIGSAC conference on computer and communications security, pp 364–379 Hijawi W, Alqatawna J, Al-Zoubi AM, Hassonah MA, Faris H (2021) Android botnet detection using machine learning models based on a comprehensive static analysis approach. J Inf Secur Appl 58:102735. https://doi.org/10.1016/j.jisa.2020.102735, https://linkinghub.elsevier.com/retrieve/pii/S2214212620308711 Kim B, Wattenberg M, Gilmer J, Cai C, Wexler J, Viegas F, Sayres R (2018) Interpretability beyond feature attribution: quantitative testing with concept activation vectors (TCAV). In: 35th international conference on machine learning (ICML 2018), vol 80, pp 2668–2677, Stockholm Koh PW, Liang P (2017) Understanding black-box predictions via influence functions. In: International conference on machine learning (ICML) Koh PW, Nguyen T, Tang YS, Mussmann S, Pierson E, Kim B, Liang P (2020) Concept bottleneck models. In: III HD, Singh A (eds) Proceedings of the 37th international conference on machine learning, Proceedings of Machine Learning Research, vol 119, pp 5338–5348, PMLR. http://proceedings.mlr.press/v119/koh20a.html Kolcz A, Teo CH (2009) Feature weighting for improved classifier robustness. In: Sixth conference on email and anti-spam (CEAS). Mountain View Li Q, Hu Q, Qi Y, Qi S, Liu X, Gao P. (2021)Semi-supervised two-phase familial analysis of Android malware with normalized graph embedding. Knowl Based Syst 218:106802. https://doi.org/10.1016/j.knosys.2021.106802, https://linkinghub.elsevier.com/retrieve/pii/S0950705121000654 Lindorfer M, Neugschwandtner M, Platzer C (2015) Marvin: efficient and comprehensive mobile app classification through static and dynamic analysis. In: Proceedings of the 39th annual international computers, software & applications conference (COMPSAC) Lindorfer M, Neugschwandtner M, Platzer C (2015) MARVIN: efficient and comprehensive mobile app classification through static and dynamic analysis. In: 2015 IEEE 39th annual computer software and applications conference, vol 2, pp 422–433 Lowd D, Meek C (2005) Adversarial learning. In: Proc. 11th ACM sigkdd international conference on knowledge discovery and data mining (KDD), pp 641–647. ACM Press, Chicago Lundberg SM, Erion G, Chen H, DeGrave A, Prutkin JM, Nair B, Katz R, Himmelfarb J, Bansal N, Lee SI (2020) From local explanations to global understanding with explainable AI for trees. Nature Mach Intell 2(1): 56–67. https://doi.org/10.1038/s42256-019-0138-9, http://www.nature.com/articles/s42256-019-0138-9 Lundberg SM, Lee SI (2017) A unified approach to interpreting model predictions. In: Advances in neural information processing systems, pp 4765–4774 Mahindru A, Sangal AL (2021) MLDroid-framework for Android malware detection using machine learning techniques. Neural Comput Appl 33(10):5183–5240. https://doi.org/10.1007/s00521-020-05309-4 Mahindru A, Sangal AL (2021) SemiDroid: a behavioral malware detector based on unsupervised machine learning techniques using feature selection approaches. Int J Mach Learn Cybern 12(5):1369–1411. https://doi.org/10.1007/s13042-020-01238-9 Maiorca D, Biggio B, Giacinto G (2019) Towards adversarial malware detection: lessons learned from pdf-based attacks. ACM Comput Surv (CSUR) 52(4):1–36 Maiorca D, Mercaldo F, Giacinto G, Visaggio CA, Martinelli F (2017) R-packdroid: Api package-based characterization and detection of mobile ransomware. In: Proceedings of the symposium on applied computing, SAC ’17, pp 1718–1723. ACM, New York. https://doi.org/10.1145/3019612.3019793 Mariconti E, Onwuzurike L, Andriotis P, Cristofaro ED, Ross GJ, Stringhini G (2017) Mamadroid: Detecting android malware by building markov chains of behavioral models. In: NDSS. The Internet Society Melis M, Demontis A, Biggio B, Brown G, Fumera G, Roli F (2017) Is deep learning safe for robot vision? Adversarial examples against the icub humanoid. In: ICCV workshop on vision in practice on autonomous robots (ViPAR) Melis M, Demontis A, Pintor M, Sotgiu A, Biggio B (2019) secml: a python library for secure and explainable machine learning. arXiv:1912.10013 Melis M, Maiorca D, Biggio B, Giacinto G, Roli F (2018) Explaining black-box android malware detection. In: 2018 26th european signal processing conference (EUSIPCO), pp 524–528, IEEE Pendlebury F, Pierazzi F, Jordaney R, Kinder J, Cavallaro L (2019) \(\{\)TESSERACT\(\}\): Eliminating experimental bias in malware classification across space and time. In: 28th \(\{\)USENIX\(\}\) Security Symposium (\(\{\)USENIX\(\}\) Security 19), pp 729–746 Peng H, Gates C, Sarma B, Li N, Qi Y, Potharaju R, Nita-Rotaru C, Molloy I (2012) Using probabilistic generative models for ranking risks of android apps. In: Proceedings of the 2012 ACM conference on computer and communications security Pierazzi F, Pendlebury F, Cortellazzi J, Cavallaro L (2020) Intriguing properties of adversarial ml attacks in the problem space. In: 2020 IEEE symposium on security and privacy (SP), pp 1332–1349, IEEE Ribeiro MT, Singh S, Guestrin C (2016) “why should i trust you?”: explaining the predictions of any classifier. In: 22nd ACM SIGKDD Int’l Conf. Knowl. Disc. Data Mining, KDD ’16, pp 1135–1144. ACM, New York Rosenberg I, Meir S, Berrebi J, Gordon I, Sicard G, David EO (2020) Generating end-to-end adversarial examples for malware classifiers using explainability. In: 2020 international joint conference on neural networks (IJCNN), pp 1–10, IEEE Scalas M, Maiorca D, Mercaldo F, Visaggio CA, Martinelli F, Giacinto G (2019) On the effectiveness of system api-related information for android ransomware detection. Comput Secur 86:168–182 Scalas M, Rieck K, Giacinto G (2021) Explanation-driven characterization of android ransomware. In: ICPR’2020 workshop on explainable deep learning—AI, pp 228–242. Springer, Cham. https://doi.org/10.1007/978-3-030-68796-0_17 Shrikumar A, Greenside P, Shcherbina A, Kundaje A (2016) Not just a black box: learning important features through propagating activation differences Sundararajan M, Taly A, Yan Q (2017) Axiomatic attribution for deep networks. In: Proceedings of the 34th international conference on machine learning-vol 70, pp 3319–3328. JMLR. org Szegedy C, Zaremba W, Sutskever I, Bruna J, Erhan D, Goodfellow I, Fergus R (2014) Intriguing properties of neural networks. In: International conference on learning representations. arxiv:1312.6199 Tam K, Khan SJ, Fattori A, Cavallaro L (2015) CopperDroid: automatic reconstruction of android malware behaviors. In: Proc. 22nd annual network & distributed system security symposium (NDSS). The Internet Society Tramer F, Carlini N, Brendel W, Madry A (2020) On adaptive attacks to adversarial example defenses. In: Larochelle H, Ranzato M, Hadsell R, Balcan MF, Lin H (eds) Advances in neural information processing systems, vol 33, pp 1633–1645. Curran Associates, Inc. https://proceedings.neurips.cc/paper/2020/file/11f38f8ecd71867b42433548d1078e38-Paper.pdf Šrndic N, Laskov P (2014) Practical evasion of a learning-based classifier: a case study. In: Proc. 2014 IEEE symp. security and privacy, SP ’14, pp 197–211. IEEE CS, Washington, DC Warnecke A, Arp D, Wressnegger C, Rieck K (2020) Evaluating explanation methods for deep learning in security. In: 2020 IEEE european symposium on security and privacy (EuroS&P), pp 158–174. IEEE, Genova. https://doi.org/10.1109/EuroSP48549.2020.00018 Yang W, Kong D, Xie T, Gunter CA (2017) Malware detection in adversarial settings: exploiting feature evolutions and confusions in android apps. In: ACSAC, pp 288–302. ACM Zhang X, Zhang Y, Zhong M, Ding D, Cao Y, Zhang Y, Zhang M, Yang M (2020) Enhancing State-of-the-art Classifiers with API Semantics to Detect Evolved Android Malware. In: Proceedings of the 2020 ACM SIGSAC conference on computer and communications security, pp 757–770. ACM, New York. https://doi.org/10.1145/3372297.3417291