Design for responsibility: safeguarding moral perception via a partnership architecture
Tóm tắt
Advanced warfare technologies (AWT) create unprecedented capabilities to control the delivery of military force up to the point, some argue, that we are loosing humanity. But dependence on them generates difficult moral challenges impacting the decision-making process, which are only beginning to be addressed. In order to arrive at an informed opinion about the impact of AWT on decision-making, we need to know more about what AWTs are and how they operate. We provide a short overview of the different types of AWTs and discuss the key principles that underlie Humanitarian Law. We also discuss the impact of physical distance and increased levels of autonomy on AWT and discuss the challenges posed to moral perception. Before such systems can be deployed, we need to rest assured that their usage enhances, rather than undermines, human decision-making capacities. There are important choices to be made, and sound design is ‘design for responsibility’. As a solution, we therefore propose the partnership architecture that embeds concurrent views of the world and working agreements, ensuring that operators use appropriate information in the decision-making process.
Tài liệu tham khảo
Abé N (2012) Dreams in infrared: the woes of an American drone operator (Part 2: An Unpopular Job). in Der Spiegel, issue 50
Altman A, Wellman CH (2009) A liberal theory of international justice. Annals of Physics (vol 2, pp 1–42). Oxford University Press. doi:10.1093/acprof
Arciszewski HFR, de Greef TE, van Delft JH (2009) Adaptive automation in a naval combat management system. IEEE Trans Syst Man Cybern A Syst Hum 39(6):1188–1199
Arkin R (2009) Governing lethal behaviour in autonomous robots. Taylor & Francis, Baco Raton
BEASystems (2011) Taranis—looking to the future. BEA Syst. Retrieved from 23 Oct 2013. http://www.baesystems.com/cs/groups/public/documents/document/mdaw/mdy2/~edisp/baes_057655.pdf
Blanson Henkemans OA (2009) ePartner for self-care/how to enhance ehealth with personal computer assistants. Fac EWI Dep ManMachine Interaction Group. Delft University of Technology, Delft
Bradshaw JM, Feltovich P, Jung H, Kulkarni S, Taysom W, Uszok A (2004) Dimensions of adjustable autonomy and mixed-initiative interaction. In: Carbonell JG, Siekmann J (eds) Agents and computational autonomy: potential, risks, and solutions. Lecture notes in artificial intelligence, vol 2969. Springer, pp 17–39
Clark H (1996) Using language. Cambridge University Press, Cambridge
Cummings ML (2003) Automation and accountability in decision support system interface design. J Technol Stud 32:23–31
De Greef T (2012) ePartners for dynamic task allocation and coordination. Ph.D. dissertation: Delft
De Greef TE, Arciszewski HFR, Neerincx MA (2010) Adaptive automation based on an object-oriented task model: implementation and evaluation in a realistic C2 environment. J Cogn Eng Decis Mak 31:152–182
De Greef T, Mohabir A, van der Poel I, Neerincx MA (2013) sCEthics: embedding ethical values in cognitive engineering. In proceedings of the 31st European conference on cognitive ergonomics
Endsley M, Kiris E (1995) The out-of-the-loop performance problem and level of control in automation. Hum Factors 37:381–394
Friedman B, Kahn PH, Borning A (2003) Value sensitive design and information systems. In: Zhang P, Galletta D (eds) Technology 3(6):1–27
Heyns C (2013a) Report of the special rapporteur on extrajudicial, summary or arbitrary executions (A/HRC/23/47)
Heyns C (2013b) report of the special rapporteur on extrajudicial, summary or arbitrary executions (A/68/382)
Hollnagel E, Woods DD (2005) Joint cognitive systems: foundations of cognitive systems engineering. Taylor & Francis, Boca Raton
Human Rights Watch (2012) Losing humanity: the case against killer robots. Human Rights Watch, New York
Ignatieff M (2000) Virtual wars: Kosovo and beyond. Vintage, London
International Human Rights and Conflict Resolution Clinic (2012) Living under drones: death, injury, and trauma to civilians from US drone practices in Pakistan. Stanford Law School and NYU School of Law
Klein G, Woods DD, Bradshaw JM, Hoffman RR, Feltovich PJ (2004) Ten challenges for marking automation a “Team Player” in joint human-agent activity. IEEE Intell Syst 19(6):91–95
Matthias A (2004) The responsibility gap. Ascribing responsibility for the actions of learning automata. Ethics Inf Technol 6:175–183
May, L. (2005). Crimes against humanity: a normative account. Eur J Int Law vol 18, p 310. Cambridge University Press
Miller CA, Parasuraman R (2007) Designing for flexible interaction between humans and automation: delegation interfaces for supervisory control. Hum Factors 49(1):57–75
Neerincx MA, Grant T (2010) Evolution of electronics partners: human-automation operations and ePartners during planetary missions. J Cosmol 12:3825–3833
O’Connell ME (2010) Unlawful killing with combat drones: a case study of Pakistan, 2004–2009. New America A Review, pp 2004–2009
Parasuraman R, Miller CA (2004) Trust and etiquette in high-criticality automated systems. Commun ACM 47(4):51–55
Parasuraman R, Sheridan TB, Wickens CD (2000) A model for types and levels of human interaction with automation. IEEE Trans Syst Man Cybern A Syst Hum 30(3):286–297
Rasmussen J (1986) Information processing and human-machine interaction: an approach to cognitive engineering. North-Holland, Amsterdam
Royakkers L, van Est R (2010) The cubicle warrior: the marionette of digitalized warfare. Ethics Inf Technol 12(3):289–296
Salas E, Cooke N, Rosen M (2008) On teams, teamwork, and team performance: discoveries and developments. Hum Factors 50(3):540–547
Sharkey N (2010) Saying “No!” to lethal autonomous targeting. J Mil Ethics 9(4):369–383. doi:10.1080/15027570.2010.537903
Shaw M (2005) The new western way of war: risk-transfer war and its crisis in Iraq. Sierra pp VII, 183. Polity Press. Retrieved from. http://eprints.sussex.ac.uk/1506/
Sheridan TB, Verplank WL (1978) Human and computer control of undersea teleoperators. DTIC Res
Sparrow R (2007) Killer robots. J Appl Philos 24(1):62–77
Strawser BJ (2010) Moral predators: the duty to employ uninhabited aerial vehicles. J Mil Ethics 9(4):342–368
Strawser BJ (2013) Killing by remote: ethics of an unmanned military. Oxford University Press, Oxford
Vicente KJ, Rasmussen J (1992) Ecological interface design: theoretical foundations. IEEE Trans Syst Man Cybern 22(4):589–606
Walzer M (2006) Just and unjust wars: a moral argument with historical illustrations, vol 32. Basic Books, New York, p 361
Woods DD, Tittle J, Feil M, Roesler A (2004) Envisioning human-robot coordination in future operations. IEEE Trans Syst Man Cybern C Appl Rev 34(2):210–218