Security-by-Experiment: Lessons from Responsible Deployment in Cyberspace

Bilge, L., & Dumitras, T. (2012). Before we knew it: An empirical study of zero-day attacks in the real world. In Proceedings of the 2012 ACM conference on computer and communications security (pp. 833–844). New York, NY, USA: ACM. doi:10.1145/2382196.2382284. Böhme, R. (2006). A comparison of market approaches to software vulnerability disclosure. In G. Müller (Ed.), Emerging trends in information and communication security (Vol. 3995, pp. 298–311). Berlin: Springer. doi:10.1007/11766155_21. Bullée, J.-W. H., Montoya, L., Pieters, W., Junger, M., & Hartel, P. H. (2015). The persuasion and security awareness experiment: Reducing the success of social engineering attacks. Journal of Experimental Criminology. doi:10.1007/s11292-014-9222-7. Cavusoglu, H., Cavusoglu, H., & Raghunathan, S. (2005). Emerging issues in responsible vulnerability disclosure. In Proceedings of the workshop on the economics of information security (WEIS). Commissie onderzoek elektronisch stemmen in het stemlokaal. (2013). Elke stem telt: Elektronisch stemmen en tellen. http://tinyurl.com/nkg5m2s. Ministerie van Binnenlandse Zaken en Koninkrijksrelaties. Cuijpers, C., & Koops, B.-J. (2013). Smart metering and privacy in europe: Lessons from the dutch case. In S. Gutwirth, R. Leenes, P. de Hert, & Y. Poullet (Eds.), European data protection: Coming of age (pp. 269–293). Netherlands: Springer. doi:10.1007/978-94-007-5170-5_12. Dechesne, F. (2013). (Cyber)security in smart grid pilots. http://tinyurl.com/pm4a43o. TU Delft. Dechesne, F., Hadžiosmanović, D., & Pieters, W. (2014). Experimenting with incentives: Security in pilots for future grids. IEEE Security & Privacy, 12(6), 59–66. Dimkov, T., Pieters, W., & Hartel, P. (2010). Effectiveness of physical, social and digital mechanisms against laptop theft in open organizations. In Green computing and communications (GreenCom), 2010 IEEE/ACM Int’l conference on Int’l conference on cyber, physical and social computing (CPSCom) (pp. 727–732). 2010, doi:10.1109/GreenCom-CPSCom.165. Dimkov, T., Pieters, W., & Hartel, P. (2011). Training students to steal: A practical assignment in computer security education. In Proceedings of the 42nd ACM technical symposium on computer science education (pp. 21–26). New York, NY, USA: ACM. doi:10.1145/1953163.1953175. Dimkov, T., van Cleeff, A., Pieters, W., & Hartel, P. (2010). Two methodologies for physical penetration testing using social engineering. In Proceedings of the 26th annual computer security applications conference (pp. 399–408). New York, NY, USA: ACM. doi:10.1145/1920261.1920319. Drupsteen, L., & Guldenmund, F. W. (2014). What is learning? a review of the safety literature to define learning from incidents, accidents and disasters. Journal of Contingencies and Crisis Management, 22(2), 81–96. doi:10.1111/1468-5973.12039. Election Process Advisory Commission. (2007). Voting with confidence. http://www.kiesraad.nl/nl/Overige_Content/Bestanden/pdf_thema/Voting_with_confidence. Finn, P., & Jakobsson, M. (2007). Designing ethical phishing experiments. Technology and Society Magazine, IEEE, 26(1), 46–58. doi:10.1109/MTAS.2007.335565. Floridi, L. (2005). The ontological interpretation of informational privacy. Ethics and Information Technology, 7, 185–200. Garcia, F. D., de Koning Gans, G., Muijrers, R., van Rossum, P., Verdult, R., Wichers Schreur, R., et al. (2008). Dismantling mifare classic. In S. Jajodia & J. Lopez (Eds.), Computer security—ESORICS 2008 (Vol. 5283, pp. 97–114). Berlin: Springer. doi:10.1007/978-3-540-88313-5_7. Gonggrijp, R., & Hengeveld, W.-J. (2007). Studying the Nedap/Groenendaal ES3B voting computer: A computer security perspective. In Proceedings of the USENIX workshop on accurate electronic voting technology (pp. 1–1). Berkeley, CA, USA: USENIX Association. http://dl.acm.org/citation.cfm?id=1323111.1323112. Gradon, K. (2013). Crime science and the internet battlefield: Securing the analog world from digital crime. Security & Privacy, IEEE, 11(5), 93–95. doi:10.1109/MSP.2013.112. Grayman, W., Ostfeld, A., & Salomons, E. (2006). Locating monitors in water distribution systems: Red team–blue team exercise. Journal of Water Resources Planning and Management, 132(4), 300–304. doi:10.1061/(ASCE)0733-9496(2006)132:4(300). Hermans, L., & van Twist, M. (2007). Stemmachines: een verweesd dossier. Rapport van de Commissie Besluitvorming Stemmachines. Ministerie van Binnenlandse Zaken en Koninkrijksrelaties. (Available online: http://www.minbzk.nl/contents/pages/86914/rapportstemmachineseenverweesddossier, consulted April 19, 2007). Hoenkamp, R., Huitema, G. B., & de Moor-van Vugt, A. J. C. (2011). The neglected consumer: The case of the smart meter rollout in the Netherlands. Renewable Energy Law and Policy Review, 4, 269–282. Hoepman, J.-H., & Jacobs, B. (2007). Increased security through open source. Communications of the ACM, 50(1), 79–83. doi:10.1145/1188913.1188921. Jacobs, B., & Pieters, W. (2009). Electronic voting in the Netherlands: From early adoption to early abolishment. In A. Aldini, G. Barthe, & R. Gorrieri (Eds.), Foundations of security analysis and design V (Vol. 5705, pp. 121–144). Berlin: Springer. doi:10.1007/978-3-642-03829-7_4. Jacobs, J. F., Van de Poel, I., & Osseweijer, P. (2010). Sunscreens with titanium dioxide (TiO\(_{2}\)) nano-particles: A societal experiment. NanoEthics, 4(2), 103–113. doi:10.1007/s11569-010-0090-y. Just, S., Premraj, R., & Zimmermann, T. (2008). Towards the next generation of bug tracking systems. In Visual languages and Human-Centric computing, 2008. VL/HCC 2008. IEEE symposium on (pp. 82–85). doi:10.1109/VLHCC.2008.4639063. Kreibich, C., & Crowcroft, J. (2004). Honeycomb: Creating intrusion detection signatures using honeypots. SIGCOMM Computer Communication Review, 34(1), 51–56. doi:10.1145/972374.972384. Krohn, W., & Weingart, P. (1987). Commentary: Nuclear power as a social experiment-European political “fall out” from the Chernobyl meltdown. Science, Technology, & Human Values, 12(2), pp. 52–58. http://www.jstor.org/stable/689655. Lastdrager, E., Montoya, L., Hartel, P., & Junger, M. (2013). Applying the lost-letter technique to assess it risk behaviour [conference proceedings]. In Socio-technical aspects in security and trust (STAST), 2013 third workshop on (pp. 2–9). doi:10.1109/STAST.2013.15. Lehtveer, M., & Hedenus, F. (2015). Nuclear power as a climate mitigation strategy—technology and proliferation risk. Journal of Risk Research, 18(3), 273–290. doi:10.1080/13669877.2014.889194. Milgram, S. (1974). Obedience to authority: An experimental view. London: Tavistock Publications. Mirkovic, J., Reiher, P., Papadopoulos, C., Hussain, A., Shepard, M., Berg, M., et al. (2008). Testing a collaborative DDoS defense in a red team/blue team exercise. Computers, IEEE Transactions on, 57(8), 1098–1112. doi:10.1109/TC.2008.42. Nohl, K., Evans, D., Starbug, & Plötz, H. (2008). Reverse-engineering a cryptographic RFID tag. In Usenix security symposium (Vol. 28, pp. 185–193). Pavlovic, D. (2011). Gaming security by obscurity. In Proceedings of the 2011 new security paradigms workshop (pp. 125–140). New York, NY, USA: ACM. doi:10.1145/2073276.2073289. Payne, C. (2002). On the security of open source software. Information Systems Journal, 12(1), 61–78. doi:10.1046/j.1365-2575.2002.00118.x. Pieters, W. (2008). La volonté machinale: understanding the electronic voting controversy. Unpublished doctoral dissertation, Radboud University Nijmegen. http://eprints.eemcs.utwente.nl/13896/. Pieters, W. (2009). Combatting electoral traces: the Dutch tempest discussion and beyond. In P. Ryan & B. Schoenmakers (Eds.), E-Voting and identity: Second international conference, VOTE-ID 2009 (Vol. 5767). Springer. Pieters, W. (2011). The (social) construction of information security. The Information Society, 27(5), 326–335. doi:10.1080/01972243.2011.607038. Pieters, W. (2013). On thinging things and serving services: Technological mediation and inseparable goods. Ethics and Information Technology, 15(3), 195–208. doi:10.1007/s10676-013-9317-2. Pieters, W., Hadžiosmanović, D., & Dechesne, F. (2014a). Cyber security as social experiment. In Proceedings of the 2014 new security paradigms workshop. ACM. Pieters, W., Probst, C. W., Lukszo, S., & Montoya Morales, A. L. (2014b). Cost-effectiveness of security measures: A model-based framework. In T. Tsiakis, T. Kargidis, & P. Katsaros (Eds.), Approaches and processes for managing the economics of information systems (pp. 139–156). Hershey, PA, USA: IGI Global. doi:10.4018/978-1-4666-4983-5.ch009. Pieters, W., & Van Cleeff, A. (2009). The precautionary principle in a world of digital dependencies. IEEE Computer, 42(6), 50–56. Pieters, W., & Van Haren, R. (2007). Temptations of turnout and modernisation: E-voting discourses in the UK and The Netherlands. Journal of Information, Communication and Ethics in Society, 5(4), 276–292. Prakken, H., Ionita, D., & Wieringa, R. (2013). Risk assessment as an argumentation game. In J. Leite, T. Son, P. Torroni, L. van der Torre, & S. Woltran (Eds.), Computational logic in multi-agent systems (Vol. 8143, pp. 357–373). Berlin Heidelberg: Springer. doi:10.1007/978-3-642-40624-9_22. Rennoch, A., Schieferdecker, I., & Großmann, J. (2014). Security testing approaches for research, industry and standardization. In Y. Yuan, X. Wu, & Y. Lu (Eds.), Trustworthy computing and services (Vol. 426, pp. 397–406). Berlin Heidelberg: Springer. doi:10.1007/978-3-662-43908-1_49. Rios Insua, D., Rios, J., & Banks, D. (2009). Adversarial risk analysis. Journal of the American Statistical Association, 104(486), 841–854. doi:10.1198/jasa.2009.0155. Robaey, Z. (2013). Who owns hazard? the role of ownership in the GM social experiment. In H. Rcklinsberg & P. Sandin (Eds.), The ethics of consumption (pp. 51–53). Wageningen: Wageningen Academic Publishers. doi:10.3920/978-90-8686-784-4_7. Standaert, F.-X., Malkin, T. G., & Yung, M. (2009). A unified framework for the analysis of side-channel key recovery attacks. In A. Joux (Ed.), Advances in cryptology-EUROCRYPT 2009 (Vol. 5479, pp. 443–461). Berlin: Springer. doi:10.1007/978-3-642-01001-9_26. Stuttard, D. (2005). Security & obscurity. Network Security, 2005(7), 10–12. http://www.sciencedirect.com/science/article/pii/S1353485805702592 doi:10.1016/S1353-4858(05)70259-2. Taebi, B., Roeser, S., & van de Poel, I. (2012). The ethics of nuclear power: Social experiments, intergenerational justice, and emotions. Energy Policy, 51(0), 202–206. http://www.sciencedirect.com/science/article/pii/S0301421512007628 (Renewable Energy in China) doi:10.1016/j.enpol.2012.09.004 Tetri, P., & Vuorinen, J. (2013). Dissecting social engineering. Behaviour & Information Technology, 32(10), 1014–1023. doi:10.1080/0144929X.2013.763860. Tsipenyuk, K., Chess, B., & McGraw, G. (2005). Seven pernicious kingdoms: A taxonomy of software security errors. Security & Privacy, IEEE, 3(6), 81–84. doi:10.1109/MSP.2005.159. Van de Poel, I. (2009). The introduction of nanotechnology as a societal experiment. In S. Arnaldi, A. Lorenzet, & F. Russo (Eds.), Technoscience in progress. Managing the uncertainty of nanotechnology (pp. 129–142). Amsterdam: IOS Press. doi:10.3233/978-1-60750-022-3-129. Van de Poel, I. (2011). Nuclear energy as a social experiment. Ethics, Policy & Environment, 14(3), 285–290. Vuorinen, J., & Tetri, P. (2012). The order machine—the ontology of information security. Journal of the Association for Information Systems, 13(9), 695–713.