Assessing GPT-4’s role as a co-collaborator in scientific research: a case study analyzing Einstein’s special theory of relativity
Tóm tắt
This paper investigates GPT-4’s role as a research partner, particularly its ability to scrutinize complex theories like Einstein’s Special Relativity Theory (SRT). GPT-4’s advanced capabilities prove invaluable in complex research scenarios where human expertise might be limited. Despite initial biases, an inclination to uphold Einstein’s theory, and certain mathematical limitations, GPT-4 validated an inconsistency within the SRT equations, leading to a questioning of the theory's overall validity. GPT-4 contributed significantly to honing the analytical approach and expanding constraints. This paper explores the strengths and challenges associated with the use of GPT-4 in scientific research, with a strong emphasis on the need for vigilance concerning potential biases and limitations in large language models. The paper further introduces a categorization framework for AI collaborations, and specific guidelines for optimal interaction with advanced models like GPT-4. Future research endeavors should focus on augmenting these models’ precision, trustworthiness, and impartiality, particularly within complex or contentious research domains.
Tài liệu tham khảo
Bubeck S, Chandrasekaran V, Eldan R, Gehrke J, Horvitz E, Kamar E, Lee P, Lee YT, Li Y, Lundberg S. Sparks of artificial general intelligence: early experiments with gpt-4, arXiv preprint arXiv:2303.12712. 2023.
OpenAI, GPT-4 Technical Report, arXiv preprint arXiv:2303.08774. 2023.
Lewkowycz A, Andreassen A, Dohan D, Dyer E, Michalewski H, Ramasesh V, Slone A, Anil C, Schlag I, Gutman-Solo T. Solving quantitative reasoning problems with language models, arXiv preprint arXiv:2206.14858. 2022.
Singhal K, Azizi S, Tu T, Mahdavi SS, Wei J, Chung HW, Scales N, Tanwani A, Cole-Lewis H, Pfohl S. Large language models encode clinical knowledge, arXiv preprint arXiv:2212.13138. 2022.
Bommarito II M, Katz DM. GPT takes the bar exam, arXiv preprint arXiv:2212.14402. 2022.
Ali R, Tang OY, Connolly ID, Zadnik Sullivan PL, Shin JH, Fridley JS, Asaad WF, Cielo D, Oyelese AA, Doberstein CE. Performance of ChatGPT and GPT-4 on Neurosurgery Written Board Examinations, medRxiv, 2023; 2023.2003. 2025.23287743.
Einstein A. Zur Elektrodynamik bewegter Körper. Ann Phys. 1905;322(10):891.
Kahan DM. Climate-science communication and the measurement problem. Polit Psychol. 2015;36:1–43.
Lord CG, Ross L, Lepper MR. Biased assimilation and attitude polarization: the effects of prior theories on subsequently considered evidence. J Pers Soc Psychol. 1979;37(11):2098.
Kuhn D. The skills of argument. Cambridge: Cambridge University Press; 1991.
Nickerson RS. Confirmation bias: a ubiquitous phenomenon in many guises. Rev Gen Psychol. 1998;2(2):175–220.
Bryant S. The failure of the Einstein-Lorentz spherical wave proof. In: Proceedings Conference Proceedings, NPA Conference, California State University, Long Beach, California, 2010.
Bryant S. Debugging relativity: analyzing special relativity theory’s zero day defect, viXra Preprint vixra: 2208.0142, (2022).
Bryant S. Disruptive: rewriting the rules of physics. El Cerrito: Infinite Circle Publishing; 2016.
Dingle H. The case against special relativity. Nature. 1967;216(5111):119–22.
McCrea WH. Why the special theory of relativity is correct. Nature. 1967;216(5111):122–4.
Lalli R. Anti-relativity in action: the scientific activity of Herbert E. Ives between 1937 and 1953. Hist Stud Nat Sci. 2012;43(1):41–104.
Liu Y, Han T, Ma S, Zhang J, Yang Y, Tian J, He H, Li A, He M, Liu Z. Summary of ChatGPT/GPT-4 research and perspective towards the future of large language models, arXiv preprint arXiv:2304.01852. 2023.
Okun LB. The concept of mass. Phys Today. 1989;42(6):31–6.
Popper K. The logic of scientific discovery. Milton Park: Routledge; 2005.
Ellis GF, Issues in the philosophy of cosmology, arXiv preprint astro-ph/0602280, (2006).
Gomori M, Szabó LE, Is the relativity principle consistent with classical electrodynamics? Towards a logico-empiricist reconstruction of a physical theory, arXiv preprint arXiv:0912.4388. 2009.
Ives HE. Derivation of the mass-energy relation. JOSA. 1952;42(8):540–3.
White J, Fu Q, Hays S, Sandborn M, Olea C, Gilbert H, Elnashar A, Spencer-Smith J, Schmidt DC. A prompt pattern catalog to enhance prompt engineering with chatgpt, arXiv preprint arXiv:2302.11382. 2023.
Zhang B, Haddow B, Birch A. Prompting large language model for machine translation: a case study, arXiv preprint arXiv:2301.07069. 2023.
Sorensen T, Robinson J, Rytting CM, Shaw AG, Rogers KJ, Delorey AP, Khalil M, Fulda N, Wingate D. An information-theoretic approach to prompt engineering without ground truth labels, arXiv preprint arXiv:2203.11364. 2022.
Arora S, Narayan A, Chen MF, Orr LJ, Guha N, Bhatia K, Chami I, Sala F, Ré C. Ask me anything: a simple strategy for prompting language models, arXiv preprint arXiv:2210.02441. 2022.
Taveekitworachai P, Abdullah F, Dewantoro MF, Thawonmas R, Togelius J, Renz J. ChatGPT4PCG competition: character-like level generation for science birds, arXiv preprint arXiv:2303.15662. 2023.
Yang X, Peynetti E, Meerman V, Tanner C, What gpt knows about who is who, arXiv preprint arXiv:2205.07407. 2022.
Osmanovic-Thunström A, Steingrimsson S. Does GPT-3 qualify as a co-author of a scientific paper publishable in peer-review journals according to the ICMJE criteria? A case study. Discover Artif Intell. 2023;3(1):12.
Gao J, Guo Y, Lim G, Zhan T, Zhang Z, Li TJ-J, Perrault ST. CollabCoder: a GPT-powered workflow for collaborative qualitative analysis, arXiv preprint arXiv:2304.07366. 2023.
Li H, Wang Y, Liao QV, Qu H. Why is AI not a panacea for data workers? An interview study on human-AI collaboration in data storytelling, arXiv preprint arXiv:2304.08366. 2023.
Guanzhi WangYX, Jiang Y, Mandlekar A, Xiao C, Zhu Y, Fan L, Anandkumar A. Voyager: an open-ended embodied agent with large language models, arXiv preprint arXiv:2305.16291. 2023.
Bryant S. Examining modern mechanics as a three-system classical mechanics-based theory of moving systems. Phys Sci Biophys J. 2022;6(1):1–9.
Holzinger A, The next frontier: AI we can really trust. In: Machine Learning and Principles and Practice of Knowledge Discovery in Databases: International Workshops of ECML PKDD 2021, Virtual Event, September 13–17, 2021, Proceedings, Part I, Springer, 2022, pp. 427–440.
de Bruijn H, Warnier M, Janssen M. The perils and pitfalls of explainable AI: strategies for explaining algorithmic decision-making. Gov Inf Q. 2022;39(2): 101666.