Automated detection, categorisation and developers’ experience with the violations of honesty in mobile apps
Tóm tắt
Human values such as honesty, social responsibility, fairness, privacy, and the like are things considered important by individuals and society. Software systems, including mobile software applications (apps), may ignore or violate such values, leading to negative effects in various ways for individuals and society. While some works have investigated different aspects of human values in software engineering, this mixed-methods study focuses on honesty as a critical human value. In particular, we studied (i) how to detect honesty violations in mobile apps, (ii) the types of honesty violations in mobile apps, and (iii) the perspectives of app developers on these detected honesty violations. We first develop and evaluate 7 machine learning (ML) models to automatically detect violations of the value of honesty in app reviews from an end-user perspective. The most promising was a Deep Neural Network model with F1 score of 0.921. We then conducted a manual analysis of 401 reviews containing honesty violations and characterised honesty violations in mobile apps into 10 categories: unfair cancellation and refund policies; false advertisements; delusive subscriptions; cheating systems; inaccurate information; unfair fees; no service; deletion of reviews; impersonation; and fraudulent-looking apps. A developer survey and interview study with mobile developers then identified 7 key causes behind honesty violations in mobile apps and 8 strategies to avoid or fix such violations. The findings of our developer study also articulate the negative consequences that honesty violations might bring for businesses, developers, and users. Finally, the app developers’ feedback shows that our prototype ML-based models can have promising benefits in practice.
Tài liệu tham khảo
Cheating, Corruption, and Concealment (2016) pp 1–12. Cambridge University Press
Aggarwal CC, Zhai C (2012) A survey of text classification algorithms. In: Mining text data, pp 163–222. Springer
Agre PE (1997) Social science, technical systems and cooperative work: beyond the great divide, chap. Lessons learned in trying to reform AI. Erlbaum, toward a critical technical practice
Aldewereld H, Dignum V, Tan Yh (2015) Design for values information and communication technologies in software development, pp 831–845. Springer Netherlands, Dordrecht
AlOmar EA, Aljedaani W, Tamjeed M, Mkaouer MW, El-Glaly YN (2021) Finding the needle in a haystack: on the automatic identification of accessibility user reviews. In: Proceedings of the 2021 CHI conference on human factors in computing systems, CHI ’21. Association for computing machinery, New York, NY, USA. https://doi.org/10.1145/3411764.3445281
Barr K (2022) Pervasive ‘dark patterns’ are fooling people into signing up for services they don’t want. https://gizmodo.com/dark-patterns-ui-cancel-subscription-1849542166
Bowman E (2021) After data breach exposes 530 million, facebook says it will not notify users. https://www.npr.org/2021/04/09/986005820/after-data-breach-exposes-530-million-facebook-says-it-will-not-notify-users
Campbell JL, Quincy C, Osserman J, Pedersen OK (2013) Coding in-depth semistructured interviews: problems of unitization and intercoder reliability and agreement. Sociological Methods & Research 42(3):294–320
Carreño LVG, Winbladh K (2013) Analysis of user comments: an approach for software requirements evolution. In: 2013 35th International conference on software engineering (ICSE), pp 582–591. https://doi.org/10.1109/ICSE.2013.6606604
Chee FM, Taylor NT, de Castell S (2012) Re-mediating research ethics: end-user license agreements in online games. Bull Sci Technol Soc 32(6):497–506
Cheng AS, Fleischmann KR (2010) Developing a meta-inventory of human values. In: Proceedings of the 73rd ASIS &T annual meeting on navigating streams in an information ecosystem, vol 47. American society for information science
Ciurumelea A, Schaufelbühl A, Panichella S, Gall HC (2017) Analyzing reviews and code of mobile apps for better release planning. In: 2017 IEEE 24th international conference on software analysis, evolution and reengineering (SANER), pp 91–102. https://doi.org/10.1109/SANER.2017.7884612
Devlin J, Chang MW, Lee K, Toutanova K (2019) Bert: pre-training of deep bidirectional transformers for language understanding. In: Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technologies, vol 1 (long and short papers), pp 4171–4186
Di Sorbo A, Panichella S, Alexandru CV, Shimagaki J, Visaggio CA, Canfora G, Gall HC (2016) What would users change in my app? summarizing app reviews for recommending software changes. In: Proceedings of the 2016 24th ACM SIGSOFT international symposium on foundations of software engineering, FSE 2016, pp 499–510. Association for computing machinery, New York, NY, USA. https://doi.org/10.1145/2950290.2950299
Dictionary C (2021) Definition of ‘impersonate’. https://www.collinsdictionary.com/dictionary/english/impersonate
Dictionary C (2021) Definition of ‘honesty’. https://www.collinsdictionary.com/dictionary/english/honesty
Dong F, Wang H, Li L, Guo Y, Bissyandé TF, Liu T, Xu G, Klein J (2018) Frauddroid: automated ad fraud detection for android apps. In: Proceedings of the 2018 26th ACM joint meeting on European software engineering conference and symposium on the foundations of software engineering, pp 257–268
Eler MM, Orlandin L, Oliveira ADA (2019) Do android app users care about accessibility? An analysis of user reviews on the google play store. In: Proceedings of the 18th Brazilian symposium on human factors in computing systems, IHC ’19. Association for computing machinery, New York, NY, USA
Fochmann M, Fochmann N, Kocher MG, Müller N (2021) Dishonesty and risk-taking: Compliance decisions of individuals and groups. J Econ Behav Organ 185:250–286. https://doi.org/10.1016/j.jebo.2021.02.018, https://www.sciencedirect.com/science/article/pii/S0167268121%000822
Gao Y, Xu G, Li L, Luo X, Wang C, Sui Y (2022) Demystifying the underground ecosystem of account registration bots. In: ACM joint European software engineering conference and symposium on the foundations of software engineering (ESEC/FSE 2022)
Glaser BG, Strauss AL, Strutzel E (1968) The discovery of grounded theory; strategies for qualitative research. Nursing Research 17(4):364
Goodfellow I, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Courville A, Bengio Y (2014) Generative adversarial nets. In: Z. Ghahramani, M. Welling, C. Cortes, N. Lawrence, K. Weinberger (eds.) Advances in neural information processing systems, vol 27. Curran Associates, Inc. https://proceedings.neurips.cc/paper/2014/file/5ca3e9b122f61f%8f06494c97b1afccf3-Paper.pdf
Gotterbarn D, Bruckman A, Flick C, Miller K, Wolf MJ (2017) Acm code of ethics: a guide for positive action. Commun ACM 61(1):121–128. https://doi.org/10.1145/3173016
Guzman E, Maalej W (2014) How do users like this feature? A fine grained sentiment analysis of app reviews. In: 2014 IEEE 22nd international requirements engineering conference (RE), pp 153–162. https://doi.org/10.1109/RE.2014.6912257
van Haasteren A, Gille F, Fadda M, Vayena E (2019) Development of the mhealth app trustworthiness checklist. Digit Health 5:2055207619886463
Haggag O, Grundy J, Abdelrazek M, Haggag S (2022) A large scale analysis of mhealth app user reviews. Empir Softw Eng 27(7):1–53
Henderson JG (2022) FTC report shows rise in sophisticated dark patterns designed to trick and trap consumers. https://www.ftc.gov/news-events/news/press-releases/2022/09/ftc-report-shows-rise-sophisticated-dark-patterns-designed-trick-trap-consumers
Hu Y, Wang H, Zhou Y, Guo Y, Li L, Luo B, Xu F (2019) Dating with scambots: understanding the ecosystem of fraudulent dating applications. IEEE Transactions on Dependable and Secure Computing
Hussain W, Perera H, Whittle J, Nurwidyantoro A, Hoda R, Shams RA, Oliver G (2020) Human values in software engineering: contrasting case studies of practice. IEEE Transactions on Software Engineering. pp 1–15
Iacob C, Harrison R: Retrieving and analyzing mobile apps feature requests from online reviews. In: 2013 10th working conference on mining software repositories (MSR), pp 41–44 (2013). https://doi.org/10.1109/MSR.2013.6624001
Jacquemet N, James AG, Luchini S, Murphy JJ, Shogren JF (2021) Do truth-telling oaths improve honesty in crowd-working? PloS one 16(1):1–18
Joerges B (1999) Do politics have artefacts? Soc Stud Sci 29(3):411–431. https://doi.org/10.1177/030631299029003004
Keyes R (2004) The post-truth era: dishonesty and deception in contemporary life, 1st, ed. St. Martin’s Press, New York
Khalajzadeh H, Shahin M, Obie HO, Agrawal P, Grundy J (2022) Supporting developers in addressing human-centric issues in mobile apps. IEEE Transactions on software engineering (TSE), arXiv:2203.12212
King C (2017) Forcing players to walk the plank: why end user license agreements improperly control players’ rights regarding microtransactions in video games. William and Mary Law Review 58(4):1365
Kohavi R et al (1995) A study of cross-validation and bootstrap for accuracy estimation and model selection. Ijcai, vol 14. Montreal, Canada, pp 1137–1145
Lang J (2013) Cheating Lessons: Learning from Academic Dishonesty. Harvard University Press.https://books.google.fm/books?id=hTeImwEACAAJ
Lennox J (2020) 2084: artificial intelligence, the future of humanity, and the god question. Zondervan
Levin S, Yehudai A (2017) Boosting automatic commit classification into maintenance activities by utilizing source code changes. In: Proceedings of the 13th international conference on predictive models and data analytics in software engineering, PROMISE, pp 97–106. Association for Computing Machinery, New York, NY, USA
Levin S, Yehudai A (2019) Towards software analytics: modeling maintenance activities. CoRR abs/1903.04909. arXiv:1903.04909
Li C, Obie HO, Khalajzadeh H (2021) A first step towards detecting values-violating defects in android apis
Li H, Zhang L, Zhang L, Shen J (2010) A user satisfaction analysis approach for software evolution. In: 2010 IEEE international conference on progress in informatics and computing, vol 2, pp 1093–1097. https://doi.org/10.1109/PIC.2010.5687999
Li L, Allix K, Li D, Bartel A, Bissyandé TF, Klein J (2015) Potential component leaks in android apps: An investigation into a new feature set for malware detection. In: 2015 IEEE international conference on software quality, reliability and security, pp 195–200. IEEE
Li L, Li D, Bissyandé TF, Klein J, Cai H, Lo D, Le Traon Y (2017) Automatically locating malicious packages in piggybacked android apps. In: 2017 IEEE/ACM 4th international conference on mobile software engineering and systems (MOBILESoft), pp 170–174. IEEE
Li X, Zhang Z, Stefanidis K (2018) Mobile app evolution analysis based on user reviews. In: SoMeT
Maldonado EdS, Shihab E, Tsantalis N (2017) Using natural language processing to automatically detect self-admitted technical debt. IEEE Trans Softw Eng 43(11):1044–1062. https://doi.org/10.1109/TSE.2017.2654244
Mathews C, Ye K, Grozdanovski J, Marinelli M, Zhong K, Khalajzadeh H, Obie HO, Grundy J (2021) Ah-cid: a tool to automatically detect human-centric issues in app reviews. In: ICSOFT, pp 386–397
Mazar N, Ariely D (2006) Dishonesty in everyday life and its policy implications. J Pub Pol Market 25(1):117–126. http://www.jstor.org/stable/30000530
Miller CB (2021) Honesty: the philosophy and psychology of a neglected virtue. Oxford University Press USA - OSO, Oxford
Morrissey ER (1974) Sources of error in the coding of questionnaire data. Soc Methods Res 3(2):209–232
Mougouei D (2020) Engineering human values in software through value programming. Proceedings of the IEEE/ACM 42nd international conference on software engineering workshops. pp 133–136
Mougouei D, Perera H, Hussain W, Shams R, Whittle J (2018) Operationalizing human values in software: a research roadmap. ESEC/FSE 2018, pp 780–784. Association for Computing Machinery, New York, NY, USA. https://doi.org/10.1145/3236024.3264843
Nakamoto S (2009) Bitcoin: a peer-to-peer electronic cash system. http://www.bitcoin.org/bitcoin.pdf
Noble WS (2006) What is a support vector machine? Nat Biotechnol 24(12):1565–1567
Nurwidyantoro A, Shahin M, Chaudron M, Hussain W, Perera H, Shams RA, Whittle J (2021) Towards a human values dashboard for software development: an exploratory study. In: Proceedings of the 15th ACM / IEEE international symposium on empirical software engineering and measurement (ESEM), ESEM ’21. Association for Computing Machinery, New York, NY, USA
Obie HO, Hussain W, Xia X, Grundy J, Li L, Turhan B, Whittle J, Shahin M (2021) A first look at human values-violation in app reviews. In: 2021 IEEE/ACM 43rd international conference on software engineering: software engineering in society (ICSE-SEIS), pp 29–38
Obie HO, Ilekura I, Du H, Shahin M, Grundy J, Li L, Whittle J, Turhan B (2022) On the violation of honesty in mobile apps: automated detection and categories. In: 2022 IEEE/ACM 19th international conference on mining software repositories (MSR), pp 321–332. https://doi.org/10.1145/3524842.3527937
Obie HO, Ilekura I, Du H, Shahin M, Grundy J, Li L, Whittle J, Turhan B (2022) The replication repository of this manuscript. https://anonymous.4open.science/r/ml_app_reviews-3ED6/README. md
Obie HO, Shahin M, Grundy J, Turhan B, Li L, Hussain W, Whittle J (2021) Does domain change the opinion of individuals on human values? A preliminary investigation on ehealth apps end-users
O’Neill O (2002) Trust is the first casualty of the cult of transparency.https://www.telegraph.co.uk/comment/personal-view/3575750/Trust-is-the-first-casualty-of-the-cult-of-transparency.html
Palomba F, Linares-Vásquez M, Bavota G, Oliveto R, Di Penta M, Poshyvanyk D, De Lucia A (2015) User reviews matter! Tracking crowdsourced reviews to support evolution of successful apps. In: 2015 IEEE international conference on software maintenance and evolution (ICSME), pp 291–300. https://doi.org/10.1109/ICSM.2015.7332475
Panichella S, Di Sorbo A, Guzman E, Visaggio CA, Canfora G, Gall HC (2015) How can i improve my app? Classifying user reviews for software maintenance and evolution. In: 2015 IEEE international conference on software maintenance and evolution (ICSME), pp 281–290. https://doi.org/10.1109/ICSM.2015.7332474
Pelloni L, Grano G, Ciurumelea A, Panichella S, Palomba F, Gall HC (2018) Becloma: augmenting stack traces with user review information. In: 2018 IEEE 25th international conference on software analysis, evolution and reengineering (SANER), pp 522–526. https://doi.org/10.1109/SANER.2018.8330252
Perera H, Hussain W, Mougouei D, Shams RA, Nurwidyantoro A, Whittle J (2019) Towards integrating human values into software: mapping principles and rights of gdpr to values. In: 2019 IEEE 27th international requirements engineering conference (RE), pp 404–409
Perera H, Hussain W, Whittle J, Nurwidyantoro A, Mougouei D, Shams RA, Oliver G (2020) A study on the prevalence of human values in software engineering publications, 2015 – 2018. In: Proceedings of the ACM/IEEE 42nd international conference on software engineering, ICSE ’20, pp 409–420. Association for Computing Machinery, New York, NY, USA. https://doi.org/10.1145/3377811.3380393
Perez S (2019) Dating app maker match sued by ftc for fraud. https://techcrunch.com/2019/09/26/dating-app-maker-match-sued-by-ftc-for-fraud/
Peters F, Tun TT, Yu Y, Nuseibeh B (2017) Text filtering and ranking for security bug report prediction. IEEE Trans Softw Eng 45(6):615–631
Phong MV, Nguyen TT, Pham HV, Nguyen TT (2015) Mining user opinions in mobile app reviews: a keyword-based approach. In: 2015 30th IEEE/ACM international conference on automated software engineering (ASE), pp 749–759.https://doi.org/10.1109/ASE.2015.85
Rokeach M (1973) The Nature of Human Values. Free Press
Samhi J, Li L, Bissyandé TF, Klein J (2022) Difuzer: uncovering suspicious hidden sensitive operations in android apps. In: The 44th international conference on software engineering (ICSE 2022)
Schwartz S (1992) Universals in the content and structure of values: theoretical advances and empirical tests in 20 countries. Adv Exp Soc Psychol 25
Schwartz S (2012) An overview of the schwartz theory of basic values. Online Readings in Psychology and Culture 2
Seyff N, Graf F, Maiden N (2010) Using mobile re tools to give end-users their own voice. In: 2010 18th IEEE international requirements engineering Conference, pp 37–46. https://doi.org/10.1109/RE.2010.15
Shaffery P (2021) Cyber security: When the cover up is worse than the crime: uber & the consequences of hiding a data breach. https://www.pooleshaffery.com/news/2017/december/cyber-security-when-the-cover-up-is-worse-than-t/
Shams RA, Hussain W, Oliver G, Nurwidyantoro A, Perera H, Whittle J (2020) Society-oriented applications development: investigating users’ values from bangladeshi agriculture mobile applications. In: 2020 IEEE/ACM 42nd international conference on software engineering: software engineering in society (ICSE-SEIS), pp 53–62. IEEE
Shams RA, Shahin M, Oliver G, Hussain W, Perera H, Nurwidyantoro A, Whittle J (2021) Measuring bangladeshi female farmers’ values for agriculture mobile applications development. In: 54th Hawaii international conference on system sciences, HICSS’21, pp 1–10
Sullins J (2018) Information technology and moral values.https://plato.stanford.edu/entries/it-moral-values/
Whittle J (2019) Is your software valueless? IEEE Software 36(3):112–115. https://doi.org/10.1109/MS.2019.2897397
Whittle J, Ferrario MA, Simm W, Hussain W (2021) A case for human values in software engineering. IEEE Software 38(1):106–113. https://doi.org/10.1109/MS.2019.2956701
Winner L (1980) Do artifacts have politics? Daedalus 109(1):121–136. http://www.jstor.org/stable/20024652
Winter E, Forshaw S, Ferrario MA (2018) Measuring human values in software engineering. In: 2018 ACM/IEEE 12th international symposium on empirical software engineering and measurement, pp 1–4
Yiacoumi R (2021) Online educator shaw academy to refund students: ‘free trial’ charged students even when they cancelled. https://ia.acs.org.au/article/2021/online-educator-shaw-academy-to-refund-students.html
Zhu L, Xu X, Lu Q, Governatori G, Whittle J (2021) Ai and ethics – operationalising responsible ai
Zhu Y, Kiros R, Zemel R, Salakhutdinov R, Urtasun R, Torralba A, Fidler S (2015) Aligning books and movies: towards story-like visual explanations by watching movies and reading books. In: Proceedings of the IEEE international conference on computer vision, pp 19–27