A Comparison of ChatGPT and Fine-Tuned Open Pre-Trained Transformers (OPT) Against Widely Used Sentiment Analysis Tools: Sentiment Analysis of COVID-19 Survey Data.

Publication Year: 2024

DOI:
10.2196/50150

PMCID:
PMC10813836

PMID:
38271138

Journal Information

Full Title: JMIR Ment Health

Abbreviation: JMIR Ment Health

Country: Unknown

Publisher: Unknown

Language: N/A

Publication Details

Subject Category: Health Services

Available in Europe PMC: Yes

Available in PMC: Yes

PDF Available: No

Transparency Score
5/6
83.3% Transparent
Transparency Indicators
Click on green indicators to view evidence text
Core Indicators
Evidence found in paper:

"abbreviations bert bidirectional encoder representations from transformers dnn deep neural network elmo embeddings from language models gpt generative pre-trained transformers liwc linguistic inquiry and word count liwc2015 linguistic inquiry and word count 2015 llm large language model nih national institutes of health nlp natural language processing opt open pre-trained transformers palm pathways language model phi protected health information roberta robustly optimized bidirectional encoder representations from transformers approach vader valence aware dictionary and sentiment reasoner data availability the test data sets generated and analyzed during this study are deidentified and freely available in the figshare repository [ 89 ]. data availability the test data sets generated and analyzed during this study are deidentified and freely available in the figshare"

Evidence found in paper:

"the source code for fine-tuning the opt models and using chatgpt in the experiments conducted in this study is publicly accessible on github [ 110 ]."

Evidence found in paper:

"Conflicts of Interest: None declared."

Evidence found in paper:

"The authors would like to thank Kelsey Yeh and Nathaniel Menon, who worked as research assistants in the Social Media Lab at Stanford University, for their assistance in annotating the Stanford test data set. The authors would also like to express gratitude to the many respondents who shared their experiences in both studies. The research reported in this publication was supported in part by the Intramural Research Programs of the National Institute of Mental Health (ZIAMH002922: Principal Investigator JC and ZIC-MH002968: Principal Investigator FP), National Center for Complementary and Integrative Health (ZIA-AT000035: Principal Investigator LA), and National Institute of Arthritis and Musculoskeletal and Skin Diseases (K24AR075060: Principal Investigator EL). The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health."

Protocol Registration
Open Access
Paper is freely available to read
Additional Indicators
Replication
Novelty Statement
Assessment Info

Tool: rtransparent

OST Version: N/A

Last Updated: Aug 05, 2025