Putting undergraduate medical students in AI-CDSS designers’ shoes: An innovative teaching method to develop digital health critical thinking

World Health Organization. Global strategy on digital health 2020-2025 [Internet]. Geneva: World Health Organization; 2021 [cited 2022 Feb 9]. 44 p. Available from: https://apps.who.int/iris/handle/10665/344249. Tsopra, 2021, A framework for validating AI in precision medicine: considerations from the European ITFoC consortium, BMC Med Inform Decis Mak., 21, 274, 10.1186/s12911-021-01634-3 Fridsma, 2018, Health informatics: a required skill for 21st century clinicians, BMJ., 12 Fernando, 2018, Lessons learned from piloting mHealth informatics practice curriculum into a medical elective, J Am Med Inform Assoc., 25, 380, 10.1093/jamia/ocx076 Wartman, 2018, Medical Education Must Move From the Information Age to the Age of Artificial Intelligence, Academic Medicine., 93, 1107, 10.1097/ACM.0000000000002044 Foadi N, Koop C, Mikuteit M, Paulmann V, Steffens S, Behrends M. Defining Learning Outcomes as a Prerequisite of Implementing a Longitudinal and Transdisciplinary Curriculum with Regard to Digital Competencies at Hannover Medical School. J Med Educ Curric Dev. 2021 Dec;8:23821205211028348. MC E. European Deans meeting “Training Future-Proof Doctors for the Digital Society” [Internet]. 2019 [cited 2022 Feb 22]. Available from: https://www.erasmusmc.nl/en/articles/deans-meeting-april-2019. Foadi, 2022, Digital competence - A Key Competence for Todays and Future Physicians, J Eur CME., 11, 2015200, 10.1080/21614083.2021.2015200 Aulenkamp J, Mikuteit M, Löffler T, Schmidt J. Overview of digital health teaching courses in medical education in Germany in 2020. GMS J Med Educ. 2021 Apr 15;38(4):Doc80. Informatics Academic Programs [Internet]. AMIA - American Medical Informatics Association. [cited 2022 Feb 22]. Available from: https://amia.org/careers-certifications/informatics-academic-programs. Wang, 2022, Technology Literacy in Undergraduate Medical Education: Review and Survey of the US Medical School Innovation and Technology Programs, JMIR Med Educ., 8, e32183, 10.2196/32183 Machleid F, Kaczmarczyk R, Johann D, Balčiūnas J, Atienza-Carbonell B, von Maltzahn F, et al. Perceptions of Digital Health Education Among European Medical Students: Mixed Methods Survey. J Med Internet Res. 2020 Aug 14;22(8):e19827. Tudor Car, 2021, Digital Health Training Programs for Medical Students: Scoping Review, JMIR Med Educ., 7, e28275, 10.2196/28275 Echelard, 2020, Medical Student Training in eHealth: Scoping Review, JMIR Med Educ., 6, e20027, 10.2196/20027 Khurana, 2022, Digital health competencies in medical school education: a scoping review and Delphi method study, BMC Med Educ., 26, 129, 10.1186/s12909-022-03163-7 Hsu Tien, 2016, A multi-year study of teaching an online computer literacy course in a medical university: A lesson learnt, Educ Res Rev., 11, 1424, 10.5897/ERR2016.2853 Gibson, 2000, A two-year experience teaching computer literacy to first-year medical students using skill-based cohorts, Bull Med Libr Assoc., 88, 157 Burgun, 2006, Problem-based learning in medical informatics for undergraduate medical students: an experiment in two medical schools, Int J Med Inform., 75, 396, 10.1016/j.ijmedinf.2005.07.014 Lee, 2017, Incorporating the human touch: piloting a curriculum for patient-centered electronic health record use, Med Educ Online., 22, 1396171, 10.1080/10872981.2017.1396171 Pereira, 2018, Minnesota Epic101 Collaborative. Collaborating for Competency-A Model for Single Electronic Health Record Onboarding for Medical Students Rotating among Separate Health Systems. Appl Clin, Inform., 9, 199 Milano, 2014, Simulated electronic health record (Sim-EHR) curriculum: teaching EHR skills and use of the EHR for disease management and prevention, Acad Med., 89, 399, 10.1097/ACM.0000000000000149 Masters, 2014, Health professionals as mobile content creators: teaching medical students to develop mHealth applications, Med Teach., 36, 883, 10.3109/0142159X.2014.916783 Law, 2019, Computing for Medicine: Can We Prepare Medical Students for the Future?, Acad Med., 94, 353, 10.1097/ACM.0000000000002521 Poncette, 2020, Undergraduate Medical Competencies in Digital Health and Curricular Module Development: Mixed Methods Study, Journal of Medical Internet Research., 22, e22161, 10.2196/22161 Aungst TD, Patel R. Integrating Digital Health into the Curriculum—Considerations on the Current Landscape and Future Developments. J Med Educ Curric Dev. 2020 Jan 20;7:2382120519901275. Thomas, 2015 Sweet, 2015, Application of Kern’s Six-step approach to curriculum development by global health residents, Educ Health (Abingdon)., 28, 138, 10.4103/1357-6283.170124 Légifrance - Publications officielles - Journal officiel - JORF n° 0221 du 10/09/2020 [Internet]. [cited 2022 Feb 11]. Available from: https://www.legifrance.gouv.fr/download/pdf?id=eH691Pmr3LnxkaMkxCddJfxc8Q4iGnu6jiCIeEKo3-A. Berner ES. Clinical Decision Support Systems: Theory and Practice. 3rd ed. 2016 édition. New York, NY: Springer International Publishing AG; 2016. 313 p. Sutton, 2020, An overview of clinical decision support systems: benefits, risks, and strategies for success, npj Digital Medicine., 3, 1, 10.1038/s41746-020-0221-y Long, 2001, Medical informatics: reasoning methods, Artif Intell Med., 23, 71, 10.1016/S0933-3657(01)00076-8 Kwan, 2020, Computerised clinical decision support systems and absolute improvements in care: meta-analysis of controlled clinical trials, BMJ., 17 Campbell, 2006, Types of unintended consequences related to computerized provider order entry, J Am Med Inform Assoc., 13, 547, 10.1197/jamia.M2042 Tsopra R, Jais JP, Venot A, Duclos C. Comparison of two kinds of interface, based on guided navigation or usability principles, for improving the adoption of computerized decision support systems: application to the prescription of antibiotics. J Am Med Inform Assoc. 2014 Feb;21(e1):e107-116. Tsopra, 2018, Using preference learning for detecting inconsistencies in clinical practice guidelines: Methods and application to antibiotherapy, Artif Intell Med., 89, 24, 10.1016/j.artmed.2018.04.013 Tsopra, 2018, The impact of three discharge coding methods on the accuracy of diagnostic coding and hospital reimbursement for inpatient medical care, Int J Med Inform., 115, 35, 10.1016/j.ijmedinf.2018.03.015 Tsopra, 2019, Helping GPs to extrapolate guideline recommendations to patients for whom there are no explicit recommendations, through the visualization of drug properties. The example of AntibioHelp® in bacterial diseases, J Am Med Inform Assoc., 26, 1010, 10.1093/jamia/ocz057 Madar, 2021, A Web Interface for Antibiotic Prescription Recommendations in Primary Care: User-Centered Design Approach, J Med Internet Res., 23, e25741, 10.2196/25741 Agher, 2022, Encouraging Behavior Changes and Preventing Cardiovascular Diseases Using the Prevent Connect Mobile Health App: Conception and Evaluation of App Quality, J Med Internet Res., 24, e25384, 10.2196/25384 Vrel, 2021, A COVID-19 Decision Support System for Phone Call Triage, Designed by and for Medical Students, Stud Health Technol Inform., 27, 525 Durchon, 2022, An Interactive Interface for Displaying Recommendations on Emergency Phone Triage in Pediatrics, Stud Health Technol Inform., 25, 430 Michot, 2022, Towards a Clinical Decision Support System for Helping Medical Students in Emergency Call Centers, Stud Health Technol Inform., 25, 425 Tsopra, 2020, AntibioGame®: a serious game for teaching medical students about antibiotic use, International Journal of Medical Informatics., 3 Feldman, 2008, Evaluating the AMIA-OHSU 10x10 Program to Train Healthcare Professionals in Medical Informatics, AMIA Annu Symp Proc., 2008, 182 McCoy, 2020, What do medical students actually need to know about artificial intelligence? npj Digit Med., 3, 1 Amini, 2016, Using standardized patients to evaluate medical students’ evidence-based medicine skills, J Evid Based Med., 9, 38, 10.1111/jebm.12183 Charow, 2021, Artificial Intelligence Education Programs for Health Care Professionals: Scoping Review, JMIR Med Educ., 7, e31043, 10.2196/31043 Singer, 2017, UCLA Health Physician Informaticist Committee. The UCLA Health Resident Informaticist Program - A Novel Clinical Informatics Training Program, J Am Med Inform Assoc., 24, 832, 10.1093/jamia/ocw174 Jimenez, 2020, Digital health competencies for primary healthcare professionals: A scoping review, International Journal of Medical Informatics., 1 Rajaram, 2020, Training medical students and residents in the use of electronic health records: a systematic review of the literature, Journal of the American Medical Informatics Association., 27, 175, 10.1093/jamia/ocz178 Health C for D and R. Artificial Intelligence and Machine Learning in Software as a Medical Device. FDA [Internet]. 2021 Sep 22 [cited 2022 Feb 23]; Available from: https://www.fda.gov/medical-devices/software-medical-device-samd/artificial-intelligence-and-machine-learning-software-medical-device. Topol, 2019, High-performance medicine: the convergence of human and artificial intelligence, Nat Med., 25, 44, 10.1038/s41591-018-0300-7 Li, 2019, Why We Needn’t Fear the Machines: Opportunities for Medicine in a Machine Learning World, Acad Med., 94, 623, 10.1097/ACM.0000000000002661 Poncette, 2019, Clinical Requirements of Future Patient Monitoring in the Intensive Care Unit: Qualitative Study, JMIR Med Inform., 7, e13064, 10.2196/13064 Berner, 2010, Use of diagnostic decision support systems in medical education, Methods Inf Med., 49, 412, 10.3414/ME9309 Ash, 2012, Recommended practices for computerized clinical decision support and knowledge management in community settings: a qualitative study, BMC Med Inform Decis Mak., 14, 6, 10.1186/1472-6947-12-6 Lee, 2021, Artificial Intelligence in Undergraduate Medical Education: A Scoping Review, Acad Med., 96, S62, 10.1097/ACM.0000000000004291 Walpole S, Taylor P, Banerjee A. Health informatics in UK Medical Education: an online survey of current practice. JRSM Open. 2016 Jan;8(1):2054270416682674. Behrends, 2021, Interdisciplinary Teaching of Digital Competencies for Undergraduate Medical Students - Experiences of a Teaching Project by Medical Informatics and Medicine, Stud Health Technol Inform., 27, 891 Vitiello, 2020, Building for the future: the creation of a residency training track to foster innovation through clinical informatics in psychiatry, J Am Med Inform Assoc., 27, 1747, 10.1093/jamia/ocaa160 Booth, 2021, How the nursing profession should adapt for a digital future, BMJ., 14 Buchanan, 2021, Predicted Influences of Artificial Intelligence on Nursing Education: Scoping Review, JMIR Nursing., 4, e23933, 10.2196/23933 Sapci, 2017, The Effectiveness of Hands-on Health Informatics Skills Exercises in the Multidisciplinary Smart Home Healthcare and Health Informatics Training Laboratories, Appl Clin Inform., 8, 1184, 10.4338/ACI-2017-08-RA-0136 Gulshan, 2016, Development and Validation of a Deep Learning Algorithm for Detection of Diabetic Retinopathy in Retinal Fundus Photographs, JAMA., 13;316(22):2402–10