How are Autistic People Involved in the Design of Extended Reality Technologies? A Systematic Literature Review

Adams, L., Simonoff, E., Tierney, K., Hollocks, M. J., Brewster, A., Watson, J., & Valmaggia, L. (2022). Developing a user-informed intervention study of a virtual reality therapy for social anxiety in autistic adolescents. Design for Health, 6(1), 114–133. https://doi.org/10.1080/24735132.2022.2062151. Adiani, D., Itzkovitz, A., Bian, D., Katz, H., Breen, M., Hunt, S., & Sarkar, N. (2022). Career interview readiness in virtual reality (CIRVR): a platform for simulated interview training for autistic individuals and their employers. ACM Transactions on Accessible Computing (TACCESS), 15(1), 1–28. https://doi.org/10.1145/3505560. Amara, K., Boudjemila, C., Zenati, N., Djekoune, O., Aklil, D., & Kenoui, M. (2022). AR computer-assisted learning for children with ASD based on hand gesture and voice interaction. IETE Journal of Research. https://doi.org/10.1080/03772063.2022.2101554 Bauer, V., Bouchara, T., & Bourdot, P. (2022). Extended reality guidelines for supporting autism interventions based on stakeholders’ needs. Journal of Autism and Developmental Disorders. https://doi.org/10.1007/s10803-022-05447-9 Benton, L. (2014). Participatory design and autism: supporting the participation, contribution and collaboration of children with ASD during the technology design process (Doctoral dissertation, University of Bath, 2014). Benton, L., & Johnson, H. (2014). Structured approaches to participatory design for children: can targeting the needs of children with autism provide benefits for a broader child population? Instructional Science, 42, 47–65. https://doi.org/10.1007/s11251-013-9297-y. Benton, L., Vasalou, A., Khaled, R., Johnson, H., & Gooch, D. (2014). Diversity for design: a framework for involving neurodiverse children in the technology design process. In Proceedings of the SIGCHI conference on Human Factors in Computing Systems (pp. 3747–3756). https://doi.org/10.1145/2556288.2557244. Berenguer, C., Baixauli, I., Gómez, S., Andrés, M. D. E. P., & De Stasio, S. (2020). Exploring the impact of augmented reality in children and adolescents with autism spectrum disorder: A systematic review. International Journal of Environmental Research and Public Health, 17(17), 6143. https://doi.org/10.3390/ijerph17176143. Bossavit, B., & Parsons, S. (2018). Outcomes for design and learning when teenagers with autism codesign a serious game: A pilot study. Journal of Computer Assisted Learning, 34(3), 293–305. https://doi.org/10.1111/jcal.12242. Bottema-Beutel, K., Kapp, S. K., Lester, J. N., Sasson, N. J., & Hand, B. N. (2021). Avoiding ableist language: Suggestions for autism researchers. Autism in adulthood, 3(1), 18–29. Boucher, J., Bigham, S., Mayes, A., & Muskett, T. (2008). Recognition and language in low functioning autism. Journal of Autism and Developmental Disorders, 38, 1259–1269. Cobb, S., Beardon, L., Eastgate, R., Glover, T., Kerr, S., Neale, H., & Wilson, J. (2002). Applied virtual environments to support learning of social interaction skills in users with Asperger's Syndrome. Digital Creativity, 13(1), 11–22. https://doi.org/10.1076/digc.13.1.11.3208. Cohen, J. (1960). A coefficient of agreement for nominal scales. Educational and psychological measurement, 20(1), 37–46. https://doi.org/10.1177/001316446002000104. Den Houting, J., & Pellicano, E. (2019). A portfolio analysis of autism research funding in Australia, 2008–2017. Journal of Autism and Developmental Disorders, 49, 4400–4408. https://doi.org/10.1007/s10803-019-04155-1. Engelbertink, M. M., Kelders, S. M., Woudt-Mittendorff, K. M., & Westerhof, G. J. (2020). Participatory design of persuasive technology in a blended learning course: A qualitative study. Education and Information Technologies, 25, 4115–4138. https://doi.org/10.1007/s10639-020-10147-x. Escobedo, L., Tentori, M., Quintana, E., Favela, J., & Garcia-Rosas, D. (2014). Using augmented reality to help children with autism stay focused. IEEE Pervasive Computing, 13(1), 38–46. https://doi.org/10.1109/MPRV.2014.19. Fletcher-Watson, S., Adams, J., Brook, K., Charman, T., Crane, L., Cusack, J., & Pellicano, E. (2019). Making the future together: Shaping autism research through meaningful participation. Autism, 23(4), 943–953. https://doi.org/10.1177/136236131878672. Frazier, T. W., Dawson, G., Murray, D., Shih, A., Sachs, J. S., & Geiger, A. (2018). Brief report: A survey of autism research priorities across a diverse community of stakeholders. Journal of autism and developmental disorders, 48, 3965–3971. https://doi.org/10.1007/s10803-018-3642-6. Ghanouni, P., Jarus, T., Zwicker, J. G., Lucyshyn, J., Mow, K., & Ledingham, A. (2019). Social stories for children with autism spectrum disorder: Validating the content of a virtual reality program. Journal of autism and developmental disorders, 49, 660–668. https://doi.org/10.1007/s10803-018-3737-0. Giaconi, C., Ascenzi, A., Del Bianco, N., D’Angelo, I., & Capellini, S. A. (2021). Virtual and Augmented Reality for the Cultural Accessibility of People with Autism Spectrum Disorders: A Pilot Study. International Journal of the Inclusive Museum, 14(1), 95–106. https://doi.org/10.18848/1835-2014/CGP/v14i01/95-106. Gillespie-Lynch, K., Bisson, J. B., Saade, S., Obeid, R., Kofner, B., Harrison, A. J., Daou, N., Tricarico, N., Delos Santos, J., Pinkava, W., & Jordan, A. (2021). If you want to develop an effective autism training, ask autistic students to help you. Autism. https://doi.org/10.1177/13623613211041006 Glaser, N., & Schmidt, M. (2022). Systematic literature review of virtual reality intervention design patterns for individuals with autism spectrum disorders. International Journal of Human–Computer Interaction, 38(8), 753–788. https://doi.org/10.1080/10447318.2021.1970433. Halabi, O., El-Seoud, S. A., Alja’am, J. M., Alpona, H., Al-Hemadi, M., & Al-Hassan, D. (2017). Design of immersive virtual reality system to improve communication skills in individuals with autism. International Journal of Emerging Technologies in Learning. https://doi.org/10.3991/ijim.v11i2.6555 Hale, K. S., & Stanney, K. M. (2014). Handbook of virtual environments: Design, implementation, and applications (2nd ed.). CRC Press. Happé, F., & Frith, U. (2020). Annual Research Review: Looking back to look forward–changes in the concept of autism and implications for future research. Journal of Child Psychology and Psychiatry, 61(3), 218–232. Hinkle, D. E., Wiersma, W., & Jurs, S. G. (2003). Applied statistics for the behavioral sciences (5th ed.). Houghton Mifflin. Howard, M. C., & Gutworth, M. B. (2020). A meta-analysis of virtual reality training programs for social skill development. Computers & Education, 144, 103707. Frith, U., & Happé, F. (1994). Language and communication in autistic disorders. Philosophical Transactions of the Royal Society of London Series B: Biological Sciences, 346(1315), 97–104. Harris, L., Gilmore, D., Longo, A., & Hand, B. N. (2021). Patterns of US federal autism research funding during 2017–2019. Autism, 25(7), 2135–2139. https://doi.org/10.1177/13623613211003430. Ip, H. H., Wong, S. W., Chan, D. F., Li, C., Kon, L. L., Ma, P. K., & Byrne, J. (2022). Enhance affective expression and social reciprocity for children with autism spectrum disorder: using virtual reality headsets at schools. Interactive Learning Environments. https://doi.org/10.1080/10494820.2022.2107681 Karami, B., Koushki, R., Arabgol, F., Rahmani, M., & Vahabie, A. (2020). Effectiveness of virtual reality-based therapeutic interventions on individuals with autism spectrum disorder: A comprehensive meta-analysis. https://doi.org/10.31234/osf.io/s2jvy. Keating, C. T., Hickman, L., Leung, J., Monk, R., Montgomery, A., Heath, H., & Sowden, S. (2023). Autism-related language preferences of English‐speaking individuals across the globe: A mixed methods investigation. Autism Research, 16(2), 406–428. https://doi.org/10.1002/aur.2864. Kenny, L., Hattersley, C., Molins, B., Buckley, C., Povey, C., & Pellicano, E. (2016). Which terms should be used to describe autism? Perspectives from the UK autism community. Autism, 20(4), 442–462. Kerr, S. J., Neale, H. R., & Cobb, S. V. (2002). Virtual environments for social skills training: the importance of scaffolding in practice. In Proceedings of the fifth international ACM conference on Assistive technologies (pp. 104–110). https://doi.org/10.1145/638249.638269. Lee, I. J. (2019). Augmented reality coloring book: An interactive strategy for teaching children with autism to focus on specific nonverbal social cues to promote their social skills. Interaction Studies, 20(2), 256–274. https://doi.org/10.1075/is.18004.lee. Lian, X., & Sunar, M. S. (2021). Mobile augmented reality technologies for autism spectrum disorder interventions: A systematic literature review. Applied Sciences, 11(10), 4550. Lion-Bailey, C., Lubinsky, J., & Shippee, M. (2023). The XR ABC framework: Fostering immersive learning through augmented and virtual realities. Immersive education: Designing for learning (pp. 123–134). Springer International Publishing. https://doi.org/10.1007/978-3-031-18138-2_8 Lorenzo, G., Lledó, A., Arráez-Vera, G., & Lorenzo-Lledó, A. (2019). The application of immersive virtual reality for students with ASD: A review between 1990–2017. Education and Information Technologies, 24, 127–151. https://doi.org/10.1007/s10639-018-9766-7. McHugh, M. L. (2012). Interrater reliability: The kappa statistic. Biochemia Medica, 22(3), 276–282. Mesa-Gresa, P., Gil-Gómez, H., Lozano-Quilis, J. A., & Gil-Gómez, J. A. (2018). Effectiveness of virtual reality for children and adolescents with autism spectrum disorder: An evidence-based systematic review. Sensors. https://doi.org/10.3390/s18082486 Millen, L., Cobb, S., & Patel, H. (2011). Participatory design approach with children with autism. https://doi.org/10.1515/IJDHD.2011.048 Miningrum, T., Tolle, H., & Bachtiar, F. A. (2021). Augmented reality adapted book (AREmotion) design as emotional expression recognition media for children with autistic spectrum disorders (ASD). International Journal of Advanced Computer Science and Applications, 12(6), 632. Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & Group*, P. R. I. S. M. A., T (2009). Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. Annals of internal medicine, 151(4), 264–269. https://doi.org/10.7326/0003-4819-151-4-200908180-00135. Monahan, J., Freedman, B., Pini, K., & Lloyd, R. (2023). Autistic input in social skills interventions for young adults: a systematic review of the literature. Review Journal of Autism and Developmental Disorders, 10(1), 1–21. https://doi.org/10.1007/s40489-021-00280-9. Newbutt, N. A. (2013). Exploring communication and representation of the self in a virtual world by young people with autism. University College Dublin. Newbutt, N., & Bradley, R. (2022). Autism, virtual reality head-mounted displays and schools: An exploration of ethical and participatory issues. Journal of Enabling Technologies Special Issue (Virtual Reality and Autism), 16(2), 124–140. https://doi.org/10.1108/JET-01-2022-0010 Newbutt, N., Lemaignan, R. L., Daly, S., & Charisi, J., V (2023). Co-designing a social robot in a special educational needs school: Listening to the ambitions of autistic pupils and their teachers. Interaction Studies, 23(2), 204–242. https://doi.org/10.1075/is.21031.new. Nicolaidis, C., Raymaker, D., Kapp, S. K., Baggs, A., Ashkenazy, E., McDonald, K., & Joyce, A. (2019). The AASPIRE practice-based guidelines for the inclusion of autistic adults in research as co-researchers and study participants. Autism, 23(8), 2007–2019. Page, M. J., McKenzie, J. E., Bossuyt, P. M., Boutron, I., Hoffmann, T. C., Mulrow, C. D., & Moher, D. (2021). The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. International journal of surgery, 88, 105906. https://doi.org/10.1016/j.ijsu.2021.105906. Parsons, S., Beardon, L., Neale, H. R., Reynard, G., Eastgate, R., Wilson, J. R., Hopkins, E. (2000). Development of social skills amongst adults with Asperger’s Syndrome using virtual environments: the ‘AS Interactive’project. In Proc. The 3rd International Conference on Disability, Virtual Reality and Associated Technologies (pp. 23–25). ICDVRAT. Parsons, S., & Mitchell, P. (2002). The potential of virtual reality in social skills training for people with autistic spectrum disorders. Journal of intellectual disability research, 46(5), 430–443. Parsons, S., & Cobb, S. (2013). Who chooses what I need? Child voice and user-involvement in the development of learning technologies for children with autism. EPSRC Observatory for Responsible Innovation in ICT. Parsons, S. (2016). Authenticity in virtual reality for assessment and intervention in autism: A conceptual review. Educational Research Review, 19, 138–157. https://doi.org/10.1016/j.edurev.2016.08.001 Parsons, T. D., Riva, G., Parsons, S., Mantovani, F., Newbutt, N., Lin, L., & Hall, T. (2017). Virtual reality in pediatric psychology. Pediatrics, 140(Supplement_2), S86–S91. https://doi.org/10.1542/peds.2016-1758I. Pellicano, E., Dinsmore, A., & Charman, T. (2014). What should autism research focus upon? Community views and priorities from the United Kingdom. Autism, 18(7), 756–770. https://doi.org/10.1177/1362361314529627. Politis, Y., Robb, N., Yakkundi, A., Dillenburger, K., Herbertson, N., Charlesworth, B., & Goodman, L. (2017). People with disabilities leading the design of serious games and virtual worlds. International Journal of Serious Games, 4(2), 63–73. Politis, Y., Olivia, L., & Olivia, T. (2019). Empowering autistic adults through their involvement in the development of a virtual world. Advances in Autism. https://doi.org/10.1108/AIA-01-2019-0001. Politis, Y., Sung, C., Goodman, L., & Leahy, M. (2020). Conversation skills training for people with autism through virtual reality: Using responsible research and innovation approach. Advances in Autism, 6(1), 3–16. https://doi.org/10.1108/AIA-05-2018-0017 Poulsen, R., Brownlow, C., Lawson, W., & Pellicano, E. (2022). Meaningful research for autistic people? Ask autistics! Autism, 26(1), 3–5. https://doi.org/10.1177/13623613211064421. Ramachandiran, C. R., Jomhari, N., Thiyagaraja, S., & Maria, M. (2015). Virtual reality based behavioral learning for autistic children. Electronic Journal of E-Learning, 13(5), 357–365. Roche, L., Adams, D., & Clark, M. (2021). Research priorities of the autism community: A systematic review of key stakeholder perspectives. Autism, 25(2), 336–348. https://doi.org/10.1177/1362361320967790. Roschelle, J., Penuel, W., & Shechtman, N. (2006). Co-design of innovations with teachers: Definition and dynamics. In Proceedings of the international conference of the learning sciences (pp. 606–612). International Society of the Learning Sciences Rutten, A., Cobb, S., Neale, H., Kerr, S., Leonard, A., Parsons, S., & Mitchell, P. (2003). The AS interactive project: single-user and collaborative virtual environments for people with high‐functioning autistic spectrum disorders. The Journal of Visualization and Computer Animation, 14(5), 233–241. https://doi.org/10.1002/vis.320. Savickaite, S., Husselman, T. A., Taylor, R., Millington, E., Hayashibara, E., & Arthur, T. (2022). Applications of virtual reality (VR) in autism research: current trends and taxonomy of definitions. Journal of Enabling Technologies. (ahead-of-print). Schmidt, M. M. (2014). Designing for learning in a three-dimensional virtual learning environment: A design-based research approach. Journal of Special Education Technology, 29(4), 59–71. Schmidt, M., Glaser, N., Schmidt, C., Kaplan, R., Palmer, H., & Cobb, S. (2023). Programming for generalization: Confronting known challenges in the design of virtual reality interventions for autistic users. Computers & Education: X Reality. https://doi.org/10.1016/j.cexr.2023.100013 Schmidt, M. M., & Glaser, N. (2021a). Piloting an adaptive skills virtual reality intervention for adults with autism: findings from user-centered formative design and evaluation. Journal of Enabling Technologies, 15(3), 137–158. https://doi.org/10.1108/JET-09-2020-0037. Schmidt, M., & Glaser, N. (2021b). Investigating the usability and learner experience of a virtual reality adaptive skills intervention for adults with autism spectrum disorder. Educational Technology Research and Development, 69(3), 1665–1699. https://doi.org/10.1007/s11423-021-10005-8. Schmidt, M., Newbutt, N., Schmidt, C., & Glaser, N. (2021). A process-model for minimizing adverse effects when using head mounted display-based virtual reality for individuals with autism. Frontiers in Virtual Reality, 2, 611740. Schober, P., Boer, C., & Schwarte, L. A. (2018). Correlation coefficients: Appropriate use and interpretation. Anesthesia & Analgesia, 126(5), 1763–1768. https://doi.org/10.1213/ANE.0000000000002864. Smith, M. J., Pinto, R. M., Dawalt, L., Smith, J. D., Sherwood, K., Miles, R., & Steacy, C. (2020). Using community-engaged methods to adapt virtual reality job-interview training for transition-age youth on the autism spectrum. Research in Autism Spectrum Disorders, 71, 101498. https://doi.org/10.1016/j.rasd.2019.101498. Speicher, M., Hall, B. D., & Nebeling, M. (2019). What is mixed reality?. In Proceedings of the 2019 CHI conference on human factors in computing systems (pp. 1–15). Stern, C., Jordan, Z., & McArthur, A. (2014). Developing the review question and inclusion criteria. AJN The American Journal of Nursing, 114(4), 53–56. https://doi.org/10.1097/01.NAJ.0000445689.67800.86. Tsikinas, S., & Xinogalos, S. (2019). Studying the effects of computer serious games on people with intellectual disabilities or autism spectrum disorder: A systematic literature review. Journal of Computer Assisted Learning, 35(1), 61–73. Wang, C. P., Tsai, C. H., & Lee, Y. L. (2022). Requesting help module interface design on key partial video with action and augmented reality for children with autism spectrum disorder. Applied Sciences, 12(17), 8527. https://doi.org/10.3390/app12178527 Wohlin, C. (2014). Guidelines for snowballing in systematic literature studies and a replication in software engineering. In Proceedings of the 18th international conference on evaluation and assessment in software engineering (pp. 1–10) https://doi.org/10.1145/2601248.2601268. Zhu, R., Hardy, D., & Myers, T. (2019). Co-designing with adolescents with autism spectrum disorder: From ideation to implementation. In Proceedings of the 31st Australian Conference on Human-Computer-Interaction (pp. 106–116). https://doi.org/10.1145/3369457.3370914.