First- and Second-Level Bias in Automated Decision-making
Tóm tắt
Recent advances in artificial intelligence offer many beneficial prospects. However, concerns have been raised about the opacity of decisions made by these systems, some of which have turned out to be biased in various ways. This article makes a contribution to a growing body of literature on how to make systems for automated decision-making more transparent, explainable, and fair by drawing attention to and further elaborating a distinction first made by Nozick (1993) between first-level bias in the application of standards and second-level bias in the choice of standards, as well as a second distinction between discrimination and arbitrariness. Applying the typology developed, a number of illuminating observations are made. First, it is observed that some reported bias in automated decision-making is first-level arbitrariness, which can be alleviated by explainability techniques. However, such techniques have only a limited potential to alleviate first-level discrimination. Second, it is argued that second-level arbitrariness is probably quite common in automated decision-making. In contrast to first-level arbitrariness, however, second-level arbitrariness is not straightforward to detect automatically. Third, the prospects for alleviating arbitrariness are discussed. It is argued that detecting and alleviating second-level arbitrariness is a profound problem because there are many contrasting and sometimes conflicting standards from which to choose, and even when we make intentional efforts to choose standards for good reasons, some second-level arbitrariness remains.
Tài liệu tham khảo
Altman, A. (2020). Discrimination. In E.N. Zalta (Ed.) The Stanford Encyclopedia of Philosophy, Winter 2020 edn, Metaphysics Research Lab, Stanford University. https://plato.stanford.edu/archives/win2020/entries/discrimination/.
Arrow, K.J. (1951). Social choice and individual values. Wiley, Cowles Commission Mongr, No. 12.
Berenguer, A., Goncalves, J., Hosio, S., Ferreira, D., Anagnostopoulos, T., & Kostakos, V. (2016). Are smartphones ubiquitous?: An in-depth survey of smartphone adoption by seniors. IEEE Consumer Electronics Magazine, 6(1), 104–110. https://doi.org/10.1109/MCE.2016.2614524.
Bickel, P.J., Hammel, E.A., & O’Connell, J.W. (1975). Sex bias in graduate admissions: Data from Berkeley. Science, 187(4175), 398–404. https://doi.org/10.1126/science.187.4175.398.
Binns, R. (2018a). Algorithmic accountability and public reason. Philosophy & Technology, 31(4), 543–556. https://doi.org/10.1007/s13347-017-0263-5.
Binns, R. (2018b). Fairness in machine learning: Lessons from political philosophy. In S.A. Friedler C. Wilson (Eds.) Proceedings of the 1st Conference on Fairness, Accountability and Transparency, PMLR, New York, NY, USA, Proceedings of Machine Learning Research, (Vol. 81 pp. 149–159).
Borges, J.L. (2007 [1942]). Funes the Memorious [Funes el memorioso]. In DA Yates JE Irby (Eds.) Labyrinths, New Directions, pp 59–66, translation by James E. Irby.
Carcary, M., Maccani, G., Doherty, E., & Conway, G. (2018). Exploring the determinants of IoT adoption: Findings from a systematic literature review. In International Conference on Business Informatics Research. https://doi.org/10.1007/978-3-319-99951-7_8 (pp. 113–125). Springer.
Castelvecchi, D. (2016). Can we open the black box of AI? Nature News, 538(7623), 20. https://doi.org/10.1038/538020a.
Cavazos, J.G., Phillips, P.J., Castillo, C.D., & O’Toole, A.J. (2020). Accuracy comparison across face recognition algorithms: Where are we on measuring race bias? IEEE Transactions on Biometrics, Behavior, and Identity Science, https://doi.org/10.1109/TBIOM.2020.3027269.
Chouldechova, A. (2017). Fair prediction with disparate impact: A study of bias in recidivism prediction instruments. Big data, 5(2), 153–163. https://doi.org/10.1089/big.2016.0047.
Chouldechova, A., & Roth, A. (2020). A snapshot of the frontiers of fairness in machine learning. Communications of the ACM, 63(5), 82–89. https://doi.org/10.1145/3376898.
Corbett-Davies, S., Pierson, E., Feller, A., Goel, S., & Huq, A. (2017). Algorithmic decision making and the cost of fairness. In Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. https://doi.org/10.1145/3097983.3098095 (pp. 797–806).
Cross, T. (2020). Artificial intelligence and its limits: Steeper than expected. The Economist Technology Quarterly, June 13.
Danziger, S., Levav, J., & Avnaim-Pesso, L. (2011). Extraneous factors in judicial decisions. Proceedings of the National Academy of Sciences, 108(17), 6889–6892. https://doi.org/10.1073/pnas.1018033108.
Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., & Fei-Fei, L. (2009). ImageNet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition. https://doi.org/10.1109/CVPR.2009.5206848(pp. 248–255). IEEE.
Dexe, J., Franke, U., Avatare Nöu, A., & Rad, A. (2020). Towards increased transparency with value sensitive design. In Artificial Intelligence in HCI. HCI International 2020. https://doi.org/10.1007/978-3-030-50334-5_1 (pp. 3–15). Springer.
Dressel, J., & Farid, H. (2018). The accuracy, fairness, and limits of predicting recidivism. Science Advances, 4(1), eaao5580. https://doi.org/10.1126/sciadv.aao5580.
Du, M., Liu, N., & Hu, X. (2019). Techniques for interpretable machine learning. Communications of the ACM, 63(1), 68–77. https://doi.org/10.1145/3359786.
Dworkin, R. (1978). Taking rights seriously. Harvard University Press, edition including the appendix “A Reply to Critics”.
Economist (2021). Design bias: Working in the dark. The Economist, 439(9240), 10. April 10.
Fast, N.J., Sivanathan, N., Mayer, N.D., & Galinsky, A.D. (2012). Power and overconfident decision-making. Organizational Behavior and Human Decision Processes, 117(2), 249–260. https://doi.org/10.1016/j.obhdp.2011.11.009
Fleischmann, K.R., & Wallace, W.A. (2005). A covenant with transparency: Opening the black box of models. Communications of the ACM, 48(5), 93–97. https://doi.org/10.1145/1060710.1060715.
Floridi, L., Cowls, J., Beltrametti, M., Chatila, R., Chazerand, P., Dignum, V., Luetge, C., Madelin, R., Pagallo, U., Rossi, F., & et al. (2018). AI4People—an ethical framework for a good AI society: Opportunities, risks, principles, and recommendations. Minds and Machines, 28(4), 689–707. https://doi.org/10.1007/s11023-018-9482-5.
Foushee, H.C. (1984). Dyads and triads at 35,000 feet: Factors affecting group process and aircrew performance. American Psychologist, 39(8), 885. https://doi.org/10.1037/0003-066X.39.8.885.
Franke, U. (2021). Rawls’s original position and algorithmic fairness. Philosophy & Technology, 34(4), 1803-1817. https://doi.org/10.1007/s13347-021-00488-x.
Friedman, B., Kahn, P.H., Borning, A., & Huldtgren, A. (2013). Value sensitive design and information systems. In N Doorn, D Schuurbiers, I van de Poel, & ME Gorman (Eds.) Early engagement and new technologies: Opening up the laboratory. https://doi.org/10.1007/978-94-007-7844-3_4 (pp. 55–95). Netherlands, Dordrecht: Springer.
Guidotti, R., Monreale, A., Ruggieri, S., Turini, F., Giannotti, F., & Pedreschi, D. (2018). A survey of methods for explaining black box models. ACM Computing Surveys (CSUR), 51(5), 1–42. https://doi.org/10.1145/3236009.
Hankerson, D., Marshall, A.R., Booker, J., El Mimouni, H., Walker, I., & Rode, J.A. (2016). Does technology have race?. In Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems. https://doi.org/10.1145/2851581.2892578 (pp. 473–486).
Heidari, H., Ferrari, C., Gummadi, K.P., & Krause, A. (2018). Fairness behind a veil of ignorance: A welfare analysis for automated decision making. In Proceedings of the 32nd International Conference on Neural Information Processing Systems (pp. 1273–1283).
Holstein, K., Wortman Vaughan, J., Daumé, IIIH., Dudik, M., & Wallach, H. (2019). Improving fairness in machine learning systems: What do industry practitioners need?. In Proceedings of the 2019 CHI conference on human factors in computing systems. https://doi.org/10.1145/3290605.3300830 (pp. 1–16).
Hutson, M. (2020). Eye-catching advances in some AI fields are not real. Science https://doi.org/10.1126/science.abd0313.
Ji, Y., Zhang, X., Ji, S., Luo, X., & Wang, T. (2018). Model-reuse attacks on deep learning systems. In Proceedings of the 2018 ACM SIGSAC Conference on Computer and Communications Security. https://doi.org/10.1145/3243734.3243757(pp. 349–363).
Jordan, M.I., & Mitchell, T.M. (2015). Machine learning: Trends, perspectives, and prospects. Science, 349(6245), 255–260. https://doi.org/10.1126/science.aaa8415.
Kant, I. (1948 [1785]). The Moral Law: Groundwork for the Metaphysics of Morals. Routledge, translated and analyzed by H.J. Paton. The page number, as is customary, refers to the pagination of the standard Royal Prussian Academy edition.
Kleinberg, J., Mullainathan, S., & Raghavan, M. (2017). Inherent trade-offs in the fair determination of risk scores. In 8th innovations in theoretical computer science conference (ITCS 2017), Schloss Dagstuhl-Leibniz-Zentrum für Informatik. https://doi.org/10.4230/LIPIcs.ITCS.2017.43, (Vol. 67 p. 43).
Koenecke, A., Nam, A., Lake, E., Nudell, J., Quartey, M., Mengesha, Z., Toups, C., Rickford, J.R., Jurafsky, D., & Goel, S. (2020). Racial disparities in automated speech recognition. Proceedings of the National Academy of Sciences, 117(14), 7684–7689. https://doi.org/10.1073/pnas.1915768117.
Kuhlman, C., Jackson, L., & Chunara, R. (2020). No computation without representation: Avoiding data and algorithm biases through diversity. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. https://doi.org/10.1145/3394486.3411074 (pp. 3593–3593).
de Laat, P.B. (2018). Algorithmic decision-making based on machine learning from Big data: Can transparency restore accountability? Philosophy & Technology, 31(4), 525–541. https://doi.org/10.1007/s13347-017-0293-z.
Livermore, D.A. (2016). Driven by difference: How great companies fuel innovation through diversity, 1st edn. AMACOM.
Mackie, J. (1977). Ethics: Inventing right and wrong. Penguin.
Nagel, T. (1986). The view from nowhere. Oxford University Press.
Narveson, J. (2002). Respecting persons in theory and practice: Essays on moral and political philosophy. Rowman & Littlefield.
Nature (2016). More accountability for big-data algorithms. Nature, 537(7621), 449. https://doi.org/10.1038/537449a.
Nickerson, R.S. (1998). Confirmation bias: A ubiquitous phenomenon in many guises. Review of General Psychology, 2(2), 175–220. https://doi.org/10.1037/1089-2680.2.2.175.
Nozick, R. (1974). Anarchy, state, and utopia. Basic Books.
Nozick, R. (1989). The examined life: Philosophical meditations. Simon and Schuster.
Nozick, R. (1993). The nature of rationality. Princeton University Press.
Obermeyer, Z., Powers, B., Vogeli, C., & Mullainathan, S. (2019). Dissecting racial bias in an algorithm used to manage the health of populations. Science, 366 (6464), 447–453. https://doi.org/10.1126/science.aax2342.
O’Neil, C. (2016). Weapons of math destruction: How big data increases inequality and threatens democracy. Crown.
Rawls, J. (1999). A theory of justice/Revised edition. Oxford University Press.
Rubin, E. (2018). Ireneo funes: Superman or failure? a Husserlian analysis. In A.J. García-Osuna (Ed.) Borges, Language and Reality, Palgrave Macmillan. https://doi.org/10.1007/978-3-319-95912-2_4 (pp. 51–61).
Rycroft, R.W., & Kash, D.E. (2002). Path dependence in the innovation of complex technologies. Technology Analysis & Strategic Management, 14 (1), 21–35. https://doi.org/10.1080/09537320220125865.
Sjoding, M.W., Dickson, R.P., Iwashyna, T.J., Gay, S.E., & Valley, T.S. (2020). Racial bias in pulse oximetry measurement. New England Journal of Medicine, 383 (25), 2477–2478. https://doi.org/10.1056/NEJMc2029240.
Sturm, B.L. (2013). Classification accuracy is not enough. Journal of Intelligent Information Systems, 41(3), 371–406. https://doi.org/10.1007/s10844-013-0250-y.
Timmons, S., & Byrne, R.M. (2019). Moral fatigue: The effects of cognitive fatigue on moral reasoning. Quarterly Journal of Experimental Psychology, 72 (4), 943–954. https://doi.org/10.1177/1747021818772045.
Wong, P.H. (2019). Democratizing algorithmic fairness. Philosophy & Technology, 33(2), 225–244. https://doi.org/10.1007/s13347-019-00355-w.
World Bank. (2021). World development report 2021: Data for better lives. The World Bank, https://doi.org/10.1596/978-1-4648-1600-0.
Yang, K., Qinami, K., Fei-Fei, L., Deng, J., & Russakovsky, O. (2020). Towards fairer datasets: Filtering and balancing the distribution of the people subtree in the imagenet hierarchy. In Proceedings of the 2020 conference on fairness, accountability, and transparency. https://doi.org/10.1145/3351095.3375709 (pp. 547–558).
Zerilli, J., Knott, A., Maclaurin, J., & Gavaghan, C. (2019). Transparency in algorithmic and human decision-making: Is there a double standard?. Philosophy & Technology, 32 (4), 661–683. https://doi.org/10.1007/s13347-018-0330-6.