A New Approach: Generative Artificial Intelligence in Physiatry Resident Education

Year 2025, Volume: 7 Issue: 1, 120 - 128, 15.01.2025

Selkin Yılmaz Muluk , Vedat Altuntaş , Zehra Duman Şahin

https://doi.org/10.37990/medr.1581104

Abstract

Aim: This study assessed the effectiveness of ChatGPT-4o, an artificial intelligence (AI) platform, in creating a therapeutic exercises presentation for physiatry residents’ education. The aim was to compare the quality of content created by ChatGPT-4o with that of an expert, exploring the potential of AI in healthcare education.
Material and Method: Both an expert and AI created 24 PowerPoint slides across six topics, using same reputable sources. Two other experts assessed these slides according to CLEAR criteria: completeness, lack of false information, appropriateness, and relevance and scored as excellent, 5; very good=4, good=3, satisfactory/fair=2, or poor, 1.
Results: Interrater reliability was confirmed. Average scores (calculated from the two raters’ scores) for each topic were significantly lower for AI than for the expert, although whole presentation scores did not differ between the two. Overall scores (calculated from the average scores of all items) for each topic were good to excellent for AI, excellent for the expert. The overall score for whole presentation was good for AI, excellent for the expert. Highest ranks for individual criteria was relevance for AI, lack of false information for the expert. Some AI-generated elements were later integrated into the expert work, enhancing the content.
Conclusion: ChatGPT-4o can generate effective educational content, though expert outperforms it, highlighting the need for professional oversight. Collaboration between humans and AI may further enhance educational outcomes.

Keywords

Healthcare education, physiatry residency, therapeutic exercises, ChatGPT, generative AI

Ethical Statement

Since the study did not involve human participation, ethical approval was not necessary.

Supporting Institution

None.

Thanks

-

References

• World Federation for Medical Education. WFME standards for postgraduate medical education, the 2023 revision: https://wfme.org/standards/pgme/ access date 01.09.2024.
• Sierocinski E, Mathias L, Freyer Martins Pereira J, Chenot JF. Postgraduate medical training in Germany: a narrative review. GMS J Med Educ. 2022;39:Doc49.
• Kayhan Z. Teaching our students, our residents, and ourselves. Turk J Anaesthesiol Reanim. 2014;42:1-5.
• Yüksek Öğretim Kurulu. Tıpta Uzmanlık Tüzüğü (Mülga). https://www.yok.gov.tr/Sayfalar/Kurumsal/mevzuat/tipta_uzmanlik_tuzugu_mulga.aspx access date 01.09.2024.
• Bielecki JE, Tadi P. Therapeutic exercise. In: StatPearls [Internet]. Treasure Island (FL): StatPearls Publishing; 2023.
• Katwaroo AR, Adesh VS, Lowtan A, Umakanthan S. The diagnostic, therapeutic, and ethical impact of artificial intelligence in modern medicine. Postgrad Med J. 2024;100:289-96.
• OpenAI. ChatGPT. https://chat.openai.com. access date 01.09.2024.
• Liu J, Wang C, Liu S. Utility of ChatGPT in clinical practice. J Med Internet Res. 2023;25:e48568.
• Ying L, Liu Z, Fang H, et al. Text summarization with ChatGPT for drug labeling documents. Drug Discov Today. 2024;29:104018.
• Yilmaz Muluk S. Enhancing musculoskeletal injection safety: evaluating checklists generated by artificial intelligence and revising the preformed checklist. Cureus. 2024;16:e59708.
• Sallam M, Barakat M, Sallam M. A preliminary checklist (METRICS) to standardize the design and reporting of studies on generative artificial intelligence-based models in health care education and practice: Development study involving a literature review. Interact J Med Res. 2024;13:e54704.
• Özdemir H, Demirbağ Kabayel D. Range of Motion Exercises, Stretching Exercises and Peripheral Joint Mobilization/Manipulation. In: Durmaz B, ed. Tedavi Edici Egzersizler. 1st edition. Ankara: Türkiye Klinikleri; 2019:8-14.
• Eker Büyükşireci D, Meray J. Muscle perfomance exercises. In: Durmaz B, ed. Tedavi Edici Egzersizler. 1st edition. Ankara: Türkiye Klinikleri. 2019:15-20.
• Kurtaiş Aytür Y. Aerobic exercises. In: Durmaz B, ed. Tedavi Edici Egzersizler. 1st edition. Ankara: Türkiye Klinikleri; 2019:21-5.
• Alp A. Aquatic exercises. In: Durmaz B, ed. Tedavi Edici Egzersizler. 1st edition. Ankara: Türkiye Klinikleri; 2019:26-32.
• Üzümcügil Karapolat H, Akgöl I. Posture and sportive performance. In: Durmaz B, ed. Tedavi Edici Egzersizler. 1st edition. Ankara: Türkiye Klinikleri; 2019:33-39.
• Demirsoy N. Relaxation exercises. In: Durmaz B, ed. Tedavi Edici Egzersizler. 1st ed. Ankara: Türkiye Klinikleri; 2019:40-6.
• Sallam M, Barakat M, Sallam M. Pilot testing of a tool to standardize the assessment of the quality of health information generated by artificial intelligence-based models. Cureus. 2023;15:e49373.
• Yilmaz Muluk S, Altuntas V, Duman Sahin Z. Preparing educational presentations about therapeutic exercises for resident physiatrists (Version 2) [Data set]. Zenodo. Published October 28, 2024. doi:10.5281/zenodo.14003512.
• Meskó B. Prompt engineering as an important emerging skill for medical professionals: tutorial. J Med Internet Res. 2023;25:e50638.
• Hollander M, Wolfe DA, Chicken E. Nonparametric statistical methods. 2nd ed. New York: Wiley; 1999;101-3.
• Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1977;33:159-74.
• Kaneda Y, Takita M, Hamaki T, et al. ChatGPT's potential in enhancing physician efficiency: a Japanese case study. Cureus. 2023;15:e48235.
• Diane A, Gencarelli P Jr, Lee JM Jr, Mittal R. Utilizing ChatGPT to Streamline the Generation of Prior Authorization Letters and Enhance Clerical Workflow in Orthopedic Surgery Practice: A Case Report. Cureus. 2023;15:e49680.
• Sallam M. ChatGPT utility in healthcare education, research, and practice: A systematic review on the promising perspectives and valid concerns. Healthcare (Basel). 2023;11:887.
• Munaf U, Ul-Haque I, Arif TB. ChatGPT: A helpful tool for resident physicians?. Acad Med. 2023;98:868-9.
• Boscardin CK, Gin B, Golde PB, Hauer KE. ChatGPT and generative artificial intelligence for medical education: Potential impact and opportunity. Acad Med. 2024;99:22-7.
• Sun L, Yin C, Xu Q, Zhao W. Artificial intelligence for healthcare and medical education: a systematic review. Am J Transl Res. 2023;15:4820-8.
• Li J, Dada A, Puladi B, et al. ChatGPT in healthcare: a taxonomy and systematic review. Comput Methods Programs Biomed. 2024;245:108013.
• Bajwa J, Munir U, Nori A, Williams B. Artificial intelligence in healthcare: transforming the practice of medicine. Future Healthc J. 2021;8:e188-e194.
• Kleesiek J, Wu Y, Stiglic G, et al. An opinion on ChatGPT in healthcare: Written by humans only. J Nucl Med. 2023;64:701-3.