Cyber ranges and security testbeds: Scenarios, functions, tools and architecture

Al-Ayyoub, 2015, Sdsecurity: a software defined security experimental framework, 1871 Alfieri, 2005, The infn-grid testbed, Future Gener. Comput. Syst., 21, 249, 10.1016/j.future.2003.10.002 Almalawi, 2013, Scadavt-a framework for scada security testbed based on virtualization technology, 639 Alvarenga, 2016, Rio: a denial of service experimentation platform in a future internet testbed, 1 Alves, 2016, Virtualization of industrial control system testbeds for cybersecurity, 10 Antonioli, 2017, Gamifying ics security training and research: design, implementation, and results of s3, 93 Ashok, 2016, Powercyber: a remotely accessible testbed for cyber physical security of the smart grid, 1 Balenson, 2015, Cybersecurity experimentation of the future (cef): catalyzing a new generation of experimental cybersecurity research Barcellos, 2012, Beyond network simulators: fostering novel distributed applications and protocols through extendible design, J. Netw. Comput. Appl., 35, 328, 10.1016/j.jnca.2011.07.015 Benzel, 2009, Current developments in deter cybersecurity testbed technology, 57 Benzel, 2006, Experience with deter: a testbed for security research Bergin, 2015, Cyber-attack and defense simulation framework, J. Defense Model. Simul., 12, 383, 10.1177/1548512915593528 Beuran, 2018, Integrated framework for hands-on cybersecurity training: cytrone, Comput. Secur., 10.1016/j.cose.2018.06.001 Border, 2007, The development and deployment of a multi-user, remote access virtualization system for networking, security, and system administration classes, ACM SIGCSE Bull., 39, 576, 10.1145/1227504.1227501 Braidley, 2016 Caliskan, 2017, Capability detection and evaluation metrics for cyber security lab exercises, 407 Čeleda, 2015, Kypo–a platform for cyber defence exercises Chadha, 2016, Cybervan: A cyber security virtual assured network testbed, 1125 Chandra, 2019, Design of cyber warfare testbed, 249 Chiang, 2013, Cyber testing tools and methodologies Childers, 2010, Organizing large scale hacking competitions, 132 Chitu, 2010, A guide to conducting a systematic literature review of information systems research, Sprouts: Working Pap. Inf. Syst., 26 Chow, 2010, An intelligent network for federated testing of netcentric systems, 44 Cintuglu, 2017, A survey on smart grid cyber-physical system testbeds., IEEE Commun. Surv. Tutor., 19, 446, 10.1109/COMST.2016.2627399 cybersecuritydegrees, A comprehensive list of cyber security competitions. Damodaran, 2015, CRIS Cyber Range Lexicon, Version 1.0 Davis, 2013, A survey of cyber ranges and testbeds Domínguez, 2017, Cybersecurity training in control systems using real equipment, IFAC-PapersOnLine, 50, 12179, 10.1016/j.ifacol.2017.08.2151 Doupé, 2011, Hit’em where it hurts: a live security exercise on cyber situational awareness, 51 Edgar, 2011, Towards an experimental testbed facility for cyber-physical security research, 53 Edgar, 2017 Edgar, 2017, Experiment as a service, 1 Ernits, 2015, i-tee: a fully automated cyber defense competition for students, 45, 113 Farooqui, 2014, Cyber security backdrop: a scada testbed, 98 Ferguson, 2014, National cyber range overview, 123 Flauzac, 2016, Developing a distributed software defined networking testbed for iot, Procedia Comput. Sci., 83, 680, 10.1016/j.procs.2016.04.151 Fovino, 2010, An experimental platform for assessing scada vulnerabilities and countermeasures in power plants, 679 Furfaro, 2017, Using virtual environments for the assessment of cybersecurity issues in iot scenarios, Simul. Model. Pract. Theory, 73, 43, 10.1016/j.simpat.2016.09.007 Furnell, 2017, Can’T get the staff? the growing need for cyber-security skills, Comput. Fraud Secur., 2017, 5, 10.1016/S1361-3723(17)30013-1 Gao, 2013, The design of ics testbed based on emulation, physical, and simulation (eps-ics testbed), 420 Gao, 2015, Cyber-physical systems testbed based on cloud computing and software defined network, 337 Gavras, 2007, Future internet research and experimentation: the fire initiative, ACM SIGCOMM Comput. Commun. Rev., 37, 89, 10.1145/1273445.1273460 Genge, 2012, Amici: an assessment platform for multi-domain security experimentation on critical infrastructures, 228 Gephart, 2010, Design of a virtual computer lab environment for hands-on information security exercises, J. Comput. Sci. Colleges, 26, 32 Glumich, 2011, DefEX: Hands-On Cyber Defense Exercise for Undergraduate Students Gunathilaka, 2016, Softgrid: a software-based smart grid testbed for evaluating substation cybersecurity solutions, 113 Gurnani, 2014, A scalable model for implementing cyber security exercises, 680 Hahn, 2013, Cyber-physical security testbeds: architecture, application, and evaluation for smart grid, IEEE Trans. Smart Grid, 4, 847, 10.1109/TSG.2012.2226919 Herold, 2017, Achieving reproducible network environments with insalata, 30 Hoffman, 2005, Exploring a national cybersecurity exercise for universities, IEEE Secur. Privacy, 3, 27, 10.1109/MSP.2005.120 Holm, 2015, A survey of industrial control system testbeds, 11 Holm, 2016, Sved: scanning, vulnerabilities, exploits and detection, 976 Hu, J., Cordel, D., Meinel, C., 2006. A virtual machine architecture for creating it-security laboratories. Jirsik, 2014, Cloud-based security research testbed: a ddos use case, 1 Jung, 2008, Design on scada test-bed and security device, Int. J. Multim. Ubiq.Eng., 3, 75 Kick, 2014, Cyber exercise playbook Kouril, 2014, Cloud-based testbed for simulation of cyber attacks, 1 Koutsandria, 2015, A real-time testbed environment for cyber-physical security on the power grid, 67 Kuhl, 2007, Cyber attack modeling and simulation for network security analysis, 1180 Labuschagne, 2017, Developing a capability to classify technical skill levels within a cyber range, 224 Langner, 2011, Stuxnet: dissecting a cyberwarfare weapon, IEEE Secur. Privacy, 9, 49, 10.1109/MSP.2011.67 Leblanc, 2011, An overview of cyber attack and computer network operations simulation, 92 Leblanc, 2011, An overview of cyber attack and computer network operations simulation, 92 Lee, 2017, Design and implementation of cybersecurity testbed for industrial iot systems, J.Supercomput., 1 Li, 2009, Real-time security exercises on a realistic interdomain routing experiment platform, 54 Liljenstam, 2005, Rinse: the real-time immersive network simulation environment for network security exercises, 119 Line, 2015, Understanding collaborative challenges in it security preparedness exercises, 311 Louthan, 2010, The blunderdome: an offensive exercise for building network, systems, and web security awareness. Maennel, 2017, Improving and measuring learning effectiveness at cyber defense exercises, 123 Mallouhi, 2011, A testbed for analyzing security of scada control systems (tasscs), 1 Marshall, 2009, The cyber scenario modeling and reporting tool (cybersmart), 305 Miciolino, 2015, Communications network analysis in a scada system testbed under cyber-attacks, 341 Mirkovic, 2010, The deter project: advancing the science of cyber security experimentation and test, 1 Moraes, 2014, Fits: a flexible virtual network testbed architecture, Comput. Netw., 63, 221, 10.1016/j.bjp.2014.01.002 Morris, 2011, A control system testbed to validate critical infrastructure protection concepts, Int. J. Crit. Infrastruct.Protect., 4, 88, 10.1016/j.ijcip.2011.06.005 Murphy, 2014, Building a virtual cybersecurity collaborative learning laboratory (vccll), 1 Ošlejšek, 2017, Towards a unified data storage and generic visualizations in cyber ranges, 298 Palleschi, 2010, Pentagon fought proposal: congress adopts provision to halt funding for national cyber range, Inside the Air Force, 21 Patriciu, 2009, Guide for designing cyber security exercises, 172 Pfrang, 2016, Design and architecture of an industrial it security lab, 114 Pham, 2016, Cyris: A cyber range instantiation system for facilitating security training, 251 Qassim, 2017, A survey of scada testbed implementation approaches, Indian J. Sci. Technol., 10, 10.17485/ijst/2017/v10i26/116775 Rahman, 2009, Network modelling and simulation tools, Simul. Model. Pract. Theory, 17, 1011, 10.1016/j.simpat.2009.02.005 Reed, 2013, Instrumenting competition-based exercises to evaluate cyber defender situation awareness, 80 Richmond, 2005, Vise: a virtual security testbed Rossey, 2002, Lariat: Lincoln adaptable real-time information assurance testbed, 6 Rubio-Hernan, 2016, Security of cyber-physical systems, 3 Rursch, 2013, This is child’s play creating a “playground”(computer network testbed) for high school students to learn, practice, and compete in cyber defense competitions, 1776 Rursch, 2013, When a testbed does more than testing: the internet-scale event attack and generation environment (iseage)-providing learning and synthesizing experiences for cyber security students., 1267 Schepens, 2002, The cyber defense exercise: an evaluation of the effectiveness of information assurance education, J. Inf. Secur., 1, 1 Schreuders, 2017, Security scenario generator (secgen): a framework for generating randomly vulnerable rich-scenario vms for learning computer security and hosting ctf events Shumba, 2006, Teaching hands-on linux host computer security, J. Educ. Resour. Comput.(JERIC), 6, 5, 10.1145/1243481.1243486 Siaterlis, 2014, Cyber-physical testbeds, Commun. ACM, 57, 64, 10.1145/2602575 Siaterlis, 2009, A review of available software for the creation of testbeds for internet security research, 79 Siaterlis, 2010, A survey of software tools for the creation of networked testbeds, Int. J. Adv. Secur., 3, 1 Siaterlis, 2011, Using an emulation testbed for operational cyber security exercises, 185 Siboni, 2016, Advanced security testbed framework for wearable iot devices, ACM Trans. Internet Technol. (TOIT), 16, 26, 10.1145/2981546 Silva, 2014, Factors impacting performance in competitive cyber exercises Snyder, 2006, Ethical hacking and password cracking: a pattern for individualized security exercises, 13 Sommestad, 2015 Sommestad, 2012, Cyber security exercises and competitions as a platform for cyber security experiments, 47 Soupionis, 2015, Cyber-physical testbedthe impact of cyber attacks and the human factor, 326 Staff, 2012, Joint training manual for the armed forces of the united states (cjcsm 3500.03 d), Washington, DC: Joint Chiefs of Staff StepForward, Carnegie mellon university - software engineering institute. Stites, 2013, Smart grid security educational training with thundercloud: a virtual security test bed, 105 Subaşu, 2017, Modeling and simulation architecture for training in cyber defence education, 1 Sun, 2018, Cyber security of a power grid: state-of-the-art, Int. J. Electr. Power Energy Syst., 99, 45, 10.1016/j.ijepes.2017.12.020 Tsai, 2017, Control frameworks in network emulation testbeds: a survey, J. Comput. Sci., 22, 148, 10.1016/j.jocs.2017.03.003 Tsai, 2018, Testbed@ twisc: a network security experiment platform, Int. J. Commun. Syst., 31, e3446, 10.1002/dac.3446 Urias, 2012, Supervisory command and data acquisition (scada) system cyber security analysis using a live, virtual, and constructive (lvc) testbed, 1 Vigna, 2014, Ten years of ictf: the good, the bad, and the ugly. Volynkin, 2007, Large-scale reconfigurable virtual testbed for information security experiments, 1 Vykopal, J., Ošlejšek, R., Čeleda, P., Vizvary, M., Tovarňák, D., 2017a. Kypo cyber range: design and use cases. Vykopal, 2017, Lessons learned from complex hands-on defence exercises in a cyber range, 1 White, 2002, An integrated experimental environment for distributed systems and networks, ACM SIGOPS Oper. Syst. Rev., 36, 255, 10.1145/844128.844152 White, 2005, The collegiate cyber defense competition Willems, 2011, Practical network security teaching in an online virtual laboratory, 1 Willems, 2012, Online assessment for hands-on cyber security training in a virtual lab, 1 Wood, 2000, Red teaming of advanced information assurance concepts, 2, 112 Xypolytou, 2017, The fuse testbed: establishing a microgrid for smart grid security experiments, e & i Elektrotechnik und Informationstechnik, 134, 30, 10.1007/s00502-017-0483-5 Yasuda, 2016, Alfons: a mimetic network environment construction system, 59