Potential and Limitations of ChatGPT 3.5 and 4.0 as a Source of COVID-19 Information: Comprehensive Comparative Analysis of Generative and Authoritative Information.

Authors:
Wang G; Gao K; Liu Q; Wu Y; Zhang K and 2 more

Journal:
J Med Internet Res

Publication Year: 2023

DOI:
10.2196/49771

PMCID:
PMC10755661

PMID:
38096014

Journal Information

Full Title: J Med Internet Res

Abbreviation: J Med Internet Res

Country: Unknown

Publisher: Unknown

Language: N/A

Publication Details

Subject Category: Medical Informatics

Available in Europe PMC: Yes

Available in PMC: Yes

PDF Available: No

Transparency Score
3/6
50.0% Transparent
Transparency Indicators
Click on green indicators to view evidence text
Core Indicators
Data Sharing
Code Sharing
Evidence found in paper:

"Conflicts of Interest: None declared."

Evidence found in paper:

"The authors are deeply indebted to the advancements in machine learning and artificial intelligence for bolstering the methodological framework of this study. Specifically, the authors used the ChatGPT 3.5 and 4.0 language models to autonomously generate the questions that served as the cornerstone of our evaluation metrics. The generated text and prompt words from these models can be found in Multimedia Appendices 2 and 5, respectively. Concurrently, we used bidirectional encoder representations from transformers (BERT) algorithms for the quantitative evaluation of text quality. Detailed metrics, including BERT scores, are available in Multimedia Appendix 5. This computational approach underwent rigorous statistical scrutiny, which was instrumental in enhancing both the analytical rigor and methodological precision of our research. The “nonhuman assistance” provided by these advanced algorithms was indispensable in elevating the academic quality of our study. The study received funding from several sources. The National Natural Science Foundation of China (grants 30973440 and 30770950) supported the data collection, analysis, and interpretation. The Ministry of Key Laboratory of Child Development and Disorders provided funding through the Youth Basic Research Project (grant YBRP-2021XX). Additionally, the preparation of the paper was funded by key projects of the Chongqing Natural Science Foundation, specifically grants cstc2020jcyj-msxmX0326 and CSTB2022NSCQ-MSX0819. The funding agency paid for the scholarships of the students involved in the research."

Protocol Registration
Open Access
Paper is freely available to read
Additional Indicators
Replication
Novelty Statement
Assessment Info

Tool: rtransparent

OST Version: N/A

Last Updated: Aug 05, 2025