Navigating fairness measures and trade-offs
AI and Ethics - Trang 1-12 - 2023
Tóm tắt
To monitor and prevent bias in AI systems, we can use a wide range of (statistical) fairness measures. However, it is mathematically impossible to optimize all of these measures at the same time. In addition, optimizing a fairness measure often greatly reduces the accuracy of the system (Kozodoi et al., Eur J Oper Res 297:1083–1094, 2022). As a result, we need a substantive theory that informs us how to make these decisions and for what reasons. I show that by using Rawls’ notion of justice as fairness, we can create a basis for navigating fairness measures and the accuracy trade-off. In particular, this leads to a principled choice focusing on both the most vulnerable groups and the type of fairness measure that has the biggest impact on that group. This also helps to close part of the gap between philosophical accounts of distributive justice and the fairness literature that has been observed by (Kuppler et al. Distributive justice and fairness metrics in automated decision-making: How much overlap is there? arXiv preprint
arXiv:2105.01441
, 2021), and to operationalise the value of fairness.
Tài liệu tham khảo
Arneson, R. Equality of opportunity. In: Zalta, E.N. (ed.) The Stanford Encyclopedia of Philosophy. Summer 2015 ed., Metaphysics Research Lab, Stanford University (2015)
Barsotti, F., Koçer, R.G.: MinMax fairness: from Rawlsian Theory of Justice to solution for algorithmic bias. AI Society, 1–14 (2022)
Carey, A.N., Wu, X.: The statistical fairness field guide: perspectives from social and formal sciences. AI Ethics, 1–23 (2022)
Corbett-Davies, S. Pierson, E., Feller, A., Goel, S., Huq, A.: Algorithmic decision making and the cost of fairness. In: Proceedings of the 23rd acm sigkdd international conference on knowledge discovery and data mining. 797–806 (2017)
Deng, W.H., Nagireddy, M., Lee, M.S.A., Singh, J., Wu, Z.S., Holstein, K., Zhu, H.: Exploring how machine learning practitioners (try to) use fairness toolkits. In: 2022 ACM Conference on Fairness, Accountability, and Transparency. 473–484 (2022)
Diener, E., Wirtz, D., Biswas-Diener, R., Tov, William, Kim-Prieto, C., Choi, D.-W., Oishi, S.: New measures of well-being. Assessing well-being: The collected works of Diener, E. (ed.) 247–266 (2009)
Fawkes, J., Evans, R., Sejdinovic, D.: Selection, ignorability and challenges with causal fairness. In: Conference on Causal Learning and Reasoning. PMLR, 275–289 (2022)
Foulds, J.R., Pan, S.: Are parity-based notions of \(\{\)AI\(\}\) fairness desirable? A Quarterly bulletin of the Computer Society of the IEEE Technical Committee on Data Engineering 43(4), (2020)
Kleinberg, J., Mullainathan, S., Raghavan, M.: Inherent trade-offs in the fair determination of risk scores. arXiv preprint arXiv:1609.05807 (2016)
Kozodoi, N., Jacob, J., Lessmann, S.: Fairness in credit scoring: Assessment, implementation and profit implications. Eur. J. Oper. Res. 297(3), 1083–1094 (2022)
Kuppler, M., Kern, C., Bach, R.L., Kreuter, F.: Distributive justice and fairness metrics in automated decision-making: How much overlap is there? arXiv preprint arXiv:2105.01441 (2021)
Kusner, M.J., Loftus, J., Russell, C., Silva, R.: 2017. Counterfactual fairness. Adv. Neural Inform. Process. Syst. 30, (2017)
Lee, M.S.A., Floridi, L., Singh, J.: Formalising trade-offs beyond algorithmic fairness: lessons from ethical philosophy and welfare economics. AI Ethics 1(4), 529–544 (2021)
Little, C.O., Weylandt, M., Allen, G.I.: To the Fairness Frontier and Beyond: Identifying, Quantifying, and Optimizing the Fairness-Accuracy Pareto Frontier. arXiv preprint arXiv:2206.00074 (2022)
Liu, L.T., Dean, S., Rolf, E., Simchowitz, M., Hardt, M.: Delayed impact of fair machine learning. In: International Conference on Machine Learning. PMLR, 3150–3158 (2018)
Loi, M., Christen, M.: Choosing how to discriminate: Navigating ethical trade-offs in fair algorithmic design for the insurance sector. Philos. Technol. 34, 967–992 (2021)
Makhlouf, K., Zhioua, S., Palamidessi, C.: Survey on causal-based machine learning fairness notions. arXiv preprint arXiv:2010.09553 (2020)
Mehrabi, N., Morstatter, F., Saxena, N., Lerman, K., Galstyan, A.: A survey on bias and fairness in machine learning. ACM Comput. Surv. 54(6), 1–35 (2021)
Mitchell, M., Wu, S., Zaldivar, A., Barnes, P., Vasserman, L., Hutchinson, B., Spitzer, E., Raji, I.D., Gebru, T.: Model cards for model reporting. In: Proceedings of the conference on fairness, accountability, and transparency 220–229 (2019)
Osberg, L., Sharpe, A.: How should we measure the economic aspects of well-being? Rev. Income Wealth 51(2), 311–336 (2005)
Pessach, D., Shmueli, E.: Algorithmic fairness. arXiv preprint arXiv:2001.09784 (2020)
Quiggin, J.: Generalized expected utility theory: The rank-dependent model. Springer Science & Business Media, New York (2012)
Rawls, J.: Justice as fairness: A restatement. Harvard University Press, Cambridge (2001)
Rosenblatt, L., Teal Witter, R.: Counterfactual Fairness Is Basically Demographic Parity. arXiv preprint arXiv:2208.03843 (2022)
Ruf, B., Detyniecki, M.: Towards the right kind of fairness in AI. arXiv preprint arXiv:2102.08453 (2021)
Topp, C.W., Østergaard, S.D., Søndergaard, S., Bech, P.: The WHO-5 Well-Being Index: a systematic review of the literature. Psychotherapy Psychosom. 84(3), 167–176 (2015)
Wang, Y., Wang, X., Beutel, A., Prost, F., Chen, J., Chi, E.H.: Understanding and improving fairness-accuracy trade-offs in multi-task learning. In: Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining. 1748–1757 (2021)
Wei, S., Niethammer, M.: The fairness-accuracy Pareto front. Stat. Anal. Data Mining 15(3), 287–302 (2022)
Whitehead, S.J., Ali, S.: Health outcomes in economic evaluation: the QALY and utilities. Br. Med. Bull. 96(1), 5–21 (2010)
Zou, L., Khern-am-nuai, W.: AI and housing discrimination: the case of mortgage applications. AI Ethics 1–11 (2022)