How the machine ‘thinks’: Understanding opacity in machine learning algorithms

Big Data and Society - Tập 3 Số 1 - 2016
Jenna Burrell1
1School of Information, UC-Berkeley, Berkeley, CA, USA

Tóm tắt

This article considers the issue of opacity as a problem for socially consequential mechanisms of classification and ranking, such as spam filters, credit card fraud detection, search engines, news trends, market segmentation and advertising, insurance or loan qualification, and credit scoring. These mechanisms of classification all frequently rely on computational algorithms, and in many cases on machine learning algorithms to do this work. In this article, I draw a distinction between three forms of opacity: (1) opacity as intentional corporate or state secrecy, (2) opacity as technical illiteracy, and (3) an opacity that arises from the characteristics of machine learning algorithms and the scale required to apply them usefully. The analysis in this article gets inside the algorithms themselves. I cite existing literatures in computer science, known industry practices (as they are publicly presented), and do some testing and manipulation of code as a form of lightweight code audit. I argue that recognizing the distinct forms of opacity that may be coming into play in a given application is a key to determining which of a variety of technical and non-technical solutions could help to prevent harm.

Từ khóa


Tài liệu tham khảo

Barocas S (2014a) Data mining and the discourse on discrimination. In: Proceedings of the Data Ethics Workshop, Conference on Knowledge Discovery and Data Mining (KDD), 24-27 August, New York City.

Barocas S (2014b) Panic Inducing: Data Mining, Fairness, and Privacy, PhD Thesis, New York University, USA.

Barocas S, California Law Review

Both G (2014) What drives research in self-driving cars? (Part 2: Surprisingly not machine learning). Available at: http://blog.castac.org/2014/04/what-drives-research-in-self-driving-cars-part-2-surprisingly-not-machine-learning/.

10.7551/mitpress/6352.001.0001

10.7551/mitpress/9384.001.0001

Burrell J (2012) Invisible Users: Youth in the Internet Cafes of Urban Ghana. Cambridge, MA: The MIT Press.

Datta A, Tschantz MC and Datta A (2015) Automated experiments on ad privacy settings: A tale of opacity, choice, and discrimination. In: Proceedings on Privacy Enhancing Technologies, 30 June-2 July, Philadelphia, PA.

Diakopoulos N (2013) Algorithmic Accountability Reporting: On the Investigation of Black Boxes. Report, Tow Center for Digital Journalism, Columbia University.

10.1145/2347736.2347755

Dwork C, Hardt M, Pitassi T, et al. (2012) Fairness through awareness. In: Proceedings of the 3rd Innovations in Theoretical Computer Science Conference, 8-10 January, Cambridge, MA, pp. 214–226.

10.1017/S0020859003001305

Eubanks V, 2012, Digital Dead End: Fighting for Social Justice in the Information Age

10.1016/j.aos.2013.11.002

10.1007/s10676-009-9198-6

Gillespie T, 2012, Media Technologies: Essays on Communication, Materiality, and Society

10.1145/1151314.1151346

Hargittai E, 2008, Social Stratification, 936

Khosla (2012) Will we need teachers or algorithms? In: TechCrunch. Available at: http://techcrunch.com/2012/01/15/teachers-or-algorithms/ (accessed 11 December 2015).

10.1145/1929887.1929902

Lipton Z (2015) The myth of model interpretability. Available at: http://www.kdnuggets.com/2015/04/model-interpretability-neural-networks-deep-learning.html (accessed 11 December 2015).

Mateas M and Montfort N (2005) A box, darkly: Obfuscation, weird languages, and code aesthetics. In: Proceedings of the 6th Annual Digital Arts and Culture Conference, 1-3 December, Copenhagen, Denmark.

10.1177/030631296026003005

10.4159/harvard.9780674736061

Sandvig C, 2014, Journal of the New Media Caucus, 10, 1

Sandvig C, Hamilton K, Karahalios K, et al. (2014) Auditing algorithms: Research methods for detecting discrimination on internet platforms. In: Annual Meeting of the International Communication Association, Seattle, WA, pp. 1–23.

Seaver N (2014) Knowing algorithms. Presented at Media in Transition 8, Cambridge, MA.

Straka JW, 2000, Journal of Housing Research, 11, 207

Tufekci Z (2014) The year we get creeped out by the algorithms. Available at: http://www.niemanlab.org/2014/12/the-year-we-get-creeped-out-by-algorithms/ (accessed 17 June 2015).

10.1145/1118178.1118215

10.1016/j.artint.2006.10.011