A step-by-step researcher's guide to the use of an AI-based transformer in epidemiology: an exploratory analysis of ChatGPT using the STROBE checklist for observational studies

Journal of Public Health - Trang 1-36 - 2023
Francesco Sanmarchi1, Andrea Bucci2, Andrea Giovanni Nuzzolese3, Gherardo Carullo4, Fabrizio Toscano5, Nicola Nante6, Davide Golinelli1,6
1Department of Biomedical and Neuromotor Sciences, Alma Mater Studiorum – University of Bologna, Bologna, Italy
2Department of Economics and Law, University of Macerata, Macerata, Italy
3STLab, Institute for Cognitive Sciences and Technologies (ISTC)-CNR, Roma, Italy
4Department of Italian and Supranational Public Law, University of Milan, Milan, Italy
5Montefiore Medical Center, Bronx, USA
6Department of Molecular and Developmental Medicine, University of Siena, Siena, Italy

Tóm tắt

This study aims at investigating how AI-based transformers can support researchers in designing and conducting an epidemiological study. To accomplish this, we used ChatGPT to reformulate the STROBE recommendations into a list of questions to be answered by the transformer itself. We then qualitatively evaluated the coherence and relevance of the transformer’s outputs. Descriptive study. We first chose a study to be used as a basis for the simulation. We then used ChatGPT to transform each STROBE checklist’s item into specific prompts. Each answer to the respective prompt was evaluated by independent researchers in terms of coherence and relevance. The mean scores assigned to each prompt were heterogeneous. On average, for the coherence domain, the overall mean score was 3.6 out of 5.0, and for relevance it was 3.3 out of 5.0. The lowest scores were assigned to items belonging to the Methods section of the checklist. ChatGPT can be considered as a valuable support for researchers in conducting an epidemiological study, following internationally recognized guidelines and standards. It is crucial for the users to have knowledge on the subject and a critical mindset when evaluating the outputs. The potential benefits of AI in scientific research and publishing are undeniable, but it is crucial to address the risks, and the ethical and legal consequences associated with its use.

Tài liệu tham khảo

Abdel-Aty H, Gould IR (2022) Large-Scale Distributed Training of Transformers for Chemical Fingerprinting. J Chem Inf Model 62(20):4852–4862. https://doi.org/10.1021/acs.jcim.2c00715

Adami HO, Berry SCL, Breckenridge CB, Smith LL, Swenberg JA, Trichopoulos D, Weiss NS, Pastoor TP (2011) Toxicology and Epidemiology: Improving the Science with a Framework for Combining Toxicological and Epidemiological Evidence to Establish Causal Inference. Toxicol Sci 122(2):223–234. https://doi.org/10.1093/toxsci/kfr113

Alba S, Verdonck K, Lenglet A, Rumisha SF, Wienia M, Teunissen I, Straetemans M, Mendoza W, Jeannetot D, Weibel D, Mayanja-Kizza H, Juvekar S (2020) Bridging research integrity and global health epidemiology (BRIDGE) statement: Guidelines for good epidemiological practice. BMJ Glob Health 5(10):e003236. https://doi.org/10.1136/bmjgh-2020-003236

Arroyave WD, Mehta SS, Guha N, Schwingl P, Taylor KW, Glenn B, Radke EG, Vilahur N, Carreón T, Nachman RM, Lunn RM (2021) Challenges and recommendations on the conduct of systematic reviews of observational epidemiologic studies in environmental and occupational health. J Exposure Sci Environ Epidemiol 31(1):21–30. https://doi.org/10.1038/s41370-020-0228-0

Brown T, Mann B, Ryder N et al (2020) Language Models are Few-Shot Learners. Adv Neural Inf Proces Syst 33:1877–1901 https://papers.nips.cc/paper/2020/hash/1457c0d6bfcb4967418bfb8ac142f64a-Abstract.html

Cahan P, Treutlein B (2023) A conversation with ChatGPT on the role of computational systems biology in stem cell research. Stem Cell Reports 18(1):1–2. https://doi.org/10.1016/j.stemcr.2022.12.009

Castelvecchi D (2022) Are ChatGPT and AlphaCode going to replace programmers? Nature. https://doi.org/10.1038/d41586-022-04383-z

Dai Y, Gao Y, Liu F (2021) TransMed: Transformers Advance Multi-Modal Medical Image Classification. Diagnostics 11(8):8. https://doi.org/10.3390/diagnostics11081384

Dwivedi YK, Hughes L, Ismagilova E et al (2021) Artificial Intelligence (AI): Multidisciplinary perspectives on emerging challenges, opportunities, and agenda for research, practice and policy. Int J Inf Manag 57:101994. https://doi.org/10.1016/j.ijinfomgt.2019.08.002

Else H (2023) Abstracts written by ChatGPT fool scientists. Nature 613(7944):423–423. https://doi.org/10.1038/d41586-023-00056-7

Gauch HG Jr (2002) Scientific Method in Practice. Cambridge University Press. https://doi.org/10.1017/CBO9780511815034

Gordijn B, Have H (2023) ChatGPT: Evolution or revolution? Med Health Care Philos. https://doi.org/10.1007/s11019-023-10136-0

Graham F (2022) Daily briefing: Will ChatGPT kill the essay assignment? Nature. https://doi.org/10.1038/d41586-022-04437-2

Graham F (2023) Daily briefing: ChatGPT listed as author on research papers. Nature. https://doi.org/10.1038/d41586-023-00188-w

Huh S (2023) Are ChatGPT’s knowledge and interpretation ability comparable to those of medical students in Korea for taking a parasitology examination?: A descriptive study. J Educ Eval Health Professions 20:1. https://doi.org/10.3352/jeehp.2023.20.01

King MR, chatGPT. (2023) A Conversation on Artificial Intelligence, Chatbots, and Plagiarism in Higher Education. Cell Mol Bioeng 16(1):1–2. https://doi.org/10.1007/s12195-022-00754-8

O’Connor S, ChatGPT (2023) Open artificial intelligence platforms in nursing education: Tools for academic progress or abuse? Nurse Educ Pract 66:103537. https://doi.org/10.1016/j.nepr.2022.103537

Radford A, Wu J, Child R, Luan D, Amodei D, Sutskever I (2019) Language Models are Unsupervised Multitask Learners. https://www.semanticscholar.org/paper/Language-Models-are-Unsupervised-Multitask-Learners-Radford-Wu/9405cc0d6169988371b2755e573cc28650d14dfe

Raffel C, Shazeer N, Roberts A, Lee K, Narang S, Matena M, Zhou Y, Li W, Liu PJ (2020) Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer. J Mach Learn Res 21(140):1–67

Stokel-Walker C (2022) AI bot ChatGPT writes smart essays—Should professors worry? Nature. https://doi.org/10.1038/d41586-022-04397-7

Subramanian SV, Kumar A (2021) Increases in COVID-19 are unrelated to levels of vaccination across 68 countries and 2947 counties in the United States. Eur J Epidemiol 36(12):1237–1240. https://doi.org/10.1007/s10654-021-00808-7

Thiébaut R, Thiessard F, Section Editors for the IMIA Yearbook Section on Public Health and Epidemiology Informatics (2018) Artificial Intelligence in Public Health and Epidemiology. Yearbook Med Inform 27(01):207–210. https://doi.org/10.1055/s-0038-1667082

Topol EJ (2019) High-performance medicine: The convergence of human and artificial intelligence. Nat Med 25(1):1. https://doi.org/10.1038/s41591-018-0300-7

Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser Ł, Polosukhin I (2017) Attention is All you Need. Adv Neural Inf Proces Syst 30 https://papers.nips.cc/paper/2017/hash/3f5ee243547dee91fbd053c1c4a845aa-Abstract.html

von Elm E, Altman DG, Egger M, Pocock SJ, Gøtzsche PC, Vandenbroucke JP (2008) The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: Guidelines for reporting observational studies. J Clin Epidemiol 61(4):344–349. https://doi.org/10.1016/j.jclinepi.2007.11.008

Thông tin
Thông tin xuất bản

Journal of Public Health

1-36

Thông tin tác giả