AI CHEATING AND THE USE OF LARGE LANGUAGE MODELS (LLM) IN ONLINE EXAMINATIONS

Yıl 2024, Sayı: 37, 339 - 352, 15.10.2024

Mustafa Demir

https://doi.org/10.54600/igdirsosbilder.1498843

Öz

In 2020, the global pandemic and its consequences radically changed traditional lifestyles, and educational processes. During this period, institutions turned to distance learning in particular; attempts were made to manage the measurement and assessment processes together with the educational process through distance learning platforms. Distance learning and online assessment and evaluation processes continued after the end of the pandemic. Although this situation provides some convenience for instructors and students, it raises questions about the validity of online exams. Data on the use of artificial intelligence algorithms in online exams has led to questioning the validity and reliability of this exam model. Among the applications of artificial intelligence, it has been discussed that large language models, which are most likely to be associated with the educational process, can be used to answer online exam questions. In this context, this study investigates the performance of large language models in online exams, which are mostly designed as multiple-choice tests, and the measures that can be taken against AI cheating, which has become an important problem today. To this end, a set of questions previously used in online exams was created; the questions were posed to large language models; answers were sought to the questions about the potential of these models to get the questions right/wrong and what types of questions are more likely to be answered correctly. The study also discusses the precautions that can be taken against applications that copy artificial intelligence.

Anahtar Kelimeler

Artificial Intelligence, Distance Education, AI Cheating, Large Language Models

YAPAY ZEKÂ KOPYASI (AI CHEATING) VE BÜYÜK DİL MODELLERİNİN ÇEVRİMİÇİ SINAVLARDA KULLANIMI

Öz

2020 yılında ortaya çıkan küresel salgın ve sonrasında klasik yaşam tarzı, çalışma hayatı ve eğitim süreçleri kökten değişmeye başlamıştır. Bu dönemde eğitim kurumları çoğunlukla uzaktan eğitime yönelmiş; eğitim süreci ile birlikte ölçme ve değerlendirme işlemleri uzaktan eğitim platformları yoluyla yapılmaya çalışılmıştır. Uzaktan eğitim ve online ölçme değerlendirme uygulamaları, pandeminin bitişi ile de devam etmiştir. Bu durum her ne kadar eğitmen ve öğrenciler açısından bazı kolaylıklar sağlasa da online sınavların geçerliliği ile ilgili soru işaretlerini beraberinde getirmiştir. Online sınavlarda yapay zekâ algoritmalarının kullanımına dair veriler, bu sınav modelinin geçerliliği ve güvenilirliğinin sorgulanmasına yol açmıştır. Yapay zekâ uygulamaları arasında, eğitim süreci ile en fazla ilinti kurulabilecek büyük dil modellerinin, çevrimiçi sınav sorularının cevaplanmasında kullanılabileceği tartışılmaya başlanmıştır. Çalışma bu bağlamda büyük dil modellerinin çoğunlukla çoktan seçmeli (test) olarak tasarlanan çevrimiçi sınavlardaki performansını ve günümüzde önemli bir sorun haline gelen yapay zekâ kopyasına (ai cheating) karşı alınabilecek önlemleri, araştırmaktadır. Bu amaçla daha önce çevrimiçi sınavlarda kullanılmış sorulardan bir set oluşturulmuş; sorular, büyük dil modellerine yöneltilmiş; bu modellerin yöneltilen soruları doğru/yanlış çözme potansiyeli, hangi tip soruların daha doğru yanıtlandığı sorularının cevabı aranmıştır. Ayrıca çalışmada yapay zekâ kopyası olarak adlandırılan uygulamalara karşı alınabilecek önlemler tartışılmaktadır.

Anahtar Kelimeler

Yapay Zekâ, Uzaktan Eğitim, Yapay Zekâ Kopyası, Büyük Dil Modelleri

Kaynakça

• Abdelaal, E., Mills, J., & Walpita Gamage, S. (2019). Artificial Intelligence Is a Tool for Cheating Academic Integrity.
• Akbari, N. (2024, Şubat 28). The AI Cheating Crisis: Education Needs Its Anti-Doping Movement. Education Week. https://www.edweek.org/technology/opinion-the-ai-cheating-crisis-education-needs-its-anti-doping-movement/2024/02
• Andrade, I. M. D., & Tumelero, C. (2022). Increasing customer service efficiency through artificial intelligence chatbot. Revista de Gestão, 29(3), 238-251. https://doi.org/10.1108/REGE-07-2021-0120
• Asaro, P. M. (2012). A body to kick, but still no soul to damn: Legal perspectives on robotics. P. Lin, K. Abney, G. A. Bekey (Ed), Robot ethics: The ethical and social implication of robotics (169-186 ss.). Massachusetts: MIT Press.
• Aytekin, Ç., & Karabina, T. B. (2024). Chatgpt’nin Farklı Büyük Dil Modelleri Performanslarının Türkçedeki Eş Adlı Kelimeler Üzerinden İncelenmesi. İstanbul Aydın Üniversitesi Sosyal Bilimler Dergisi, 16(3), Article 3.
• Beer, D. (2017). The social power of algorithms. Information, Communication & Society, 20(1), 1–13.
• Birhane, A., Kasirzadeh, A., Leslie, D. & Watcher, S. (2023). Science in the age of large language models. Nature Reviews Physics 5, 277–280
• Campa, R. (2020). Fourth Industrial Revolution and Emotional Intelligence: A Conceptual and Scientometric Analysis, Changing Societies & Personalities, 4(1), 8–30. DOI: 10.15826/csp.2020.4.1.087
• Cellan-Jones, R. (2014). Stephen Hawking Warns Artificial İntelligence Could End Mankind. BBC News. https://www.bbc.com/news/technology-30290540
• Chomsky, N., Roberts, I., & Watumull, J. (2023). The false promise of ChatGPT. New York Times. https://www.nytimes.com/2023/03/08/opinion/noam-chomsky-chatgpt-ai.html
• Devlin J., Chang M.W., Lee K., Toutanova K. (2019) BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), Minneapolis, Minnesota. Association for Computational Linguistics.
• Doğan, A. (2002). Yapay Zekâ. Ankara. Kariyer
• Floridi L., Chiriatti M. (2020). Gpt-3: Its nature, scope, limits, and consequences. Minds and Machines, 30:681–694.
• Goodfellow I.J, Shlens J., Szegedy C. (2014). Explaining and harnessing adversarial examples. arXiv preprint arXiv:1412.6572.
• Hayawi, K., Shahriar, S., & Mathew, S. S. (2023). The imitation game: Detecting human and aı-generated texts in the era of large language models. arXiv preprint arXiv:2307.12166.
• Johnson, P. (2017). 99 Facts On The Future Of Business in The Digital Economy, SAP: https://www.slideshare.net/sap/99-facts-on-the-future-of-business-in-the-digital-economy- [01.07.2020].
• Kabak, T., & Kirbaş, İ. (2023). Chatgpt With Risks And Opportunities. Mehmet Akif Ersoy Üniversitesi Fen Bilimleri Enstitüsü Dergisi, 14(2), 365-376. https://doi.org/10.29048/makufebed.1271477
• Kirmani, A.R. (2022). Artificial intelligence-enabled science poetry, ACS Energy Letters, Vol. 8, pp. 574-576. https://doi.org/10.1021/acsenergylett.2c02758
• Kozma, R., Alippi, C., Choe, Y. ve Morabito, F. C. (Eds.). (2018). Artificial ıntelligence in the age of neural networks and brain computing. United States: Academic Press.
• Kurian, N., Cherian, J. M., Sudharson, N. A., Varghese, K. G., & Wadhwa, S. (2023). AI is now everywhere. British Dental Journal, 234(2), 72-72. https://doi.org/10.1038/s41415-023-5461-1
• Liu, X., Zheng, Y., Du, Z., Ding, M., Qian, Y., Yang, Z., & Tang, J. (2021). GPT understands, too. arXiv preprint arXiv:2103.10385.
• Lee, E. (2023). Is ChatGPT a false promise?. Berkeley Blog. https://blogs.berkeley.edu/2023/03/19/is-chatgpt-a-false-promise/
• Levy, D. (2012). The ethics of robot prostitutes. P. Lin, K. Abney, G. A. Bekey (Ed), Robot ethics: The ethical and social implication of robotics (223-232 ss.). Massachusetts: MIT Press.
• Liu, Z., Yao, Z., Li, F., & Luo, B. (2023). Check me ıf you can: Detecting ChatGPT-generated academic writing using CheckGPT. arXiv preprint arXiv:2306.05524.
• Lokhorst, G.A. & van den Hoven, J. (2012). Responsibility for military robots. P. Lin, K. Abney, G. A. Bekey (Ed), Robot ethics: The ethical and social implication of robotics (145-156 ss.). Massachusetts: MIT Press.
• Mark Massaro, O. C. (2023, Ağustos 23). AI cheating is hopelessly, irreparably corrupting US higher education [Text]. The Hill. https://thehill.com/opinion/education/4162766-ai-cheating-has-hopelessly-irreparably-corrupted-us-higher-education/
• McCarthy, J., Minsky, M., Rochester, N., & Shannon, C. (1955). A proposal for Dartmouth Summer Research Project on Artificial Intelligence. http://wwwformal.stanford.edu/jmc/history/dartmouth.pdf. Accessed 13 January 2023.
• Miyato T., Dai A.M., Goodfellow I. (2016). Adversarial training methods for semi-supervised text classification. arXiv preprint arXiv:1605.07725.
• Nithuna, S. & Laseena, C. (2020). Review on implementation techniques of chatbot. In Proceedings of the 2020 International Conference on Communication and Signal Processing (ICCSP), Chennai, India, 28–30 July 2020; pp. 157–161.
• OpenAI. (2015). OpenAI. https://openai.com/about/ (Erişim Tarihi: 17.04.2024).
• Oravec, J. A. (2023). Artificial Intelligence Implications for Academic Cheating: Expanding the Dimensions of Responsible Human-AI Collaboration with ChatGPT and Bard. Jl. of Interactive Learning Research, 2(34), 213-237.
• Pasquale, F. (2015). The black box society: The secret algorithms that control money and information. Massachusetts: Harvard University Press.
• Pool, C. R. (1997). Brain-based learning and students. The Education Digest, 63(3), 10.
• Rudolph, J., Tan, S. & Tan, S. (2023), Chatgpt: Bullshit Spewer Or The End Of Traditional Assessments In Higher Education? Journal Of Applied Learning & Teaching, 6(1), 1-22.
• Russell, S., & Norvig, P. (2021). Artificial intelligence: a modern approach (3rd ed). Prentice Hall.
• See, A., Pappu, A., Saxena, R., Yerukola, A., & Manning, C. D. (2019). Do massively pretrained language models make better storytellers?. arXiv preprint arXiv:1909.10705.
• Shah, C. (2022), The Rise Of Aı Chat Agents And The Discourse With Dilettantes. Information Matters, 2 (12). Https://İnformationmatters.Org/2022/12/The-Rise-Of-Aichat-Agents-And-The-Discourse-WithDilettantes/
• Sharkey, N. (2012). Killing made easy. P. Lin, K. Abney, G. A. Bekey (Ed). Robot ethics: The ethical and social implication of robotics (111-128 ss.). Massachusetts: MIT Press.
• Sheikh, S. (2020). Understanding the Role of Artificial Intelligence and Its Future Social Impact, IGI Global
• Smith, A. (2024). AI Cheating: Tips to Avoid and Detect. https://screenapp.io/blog/how-to-avoid-and-detect-ai-cheating-with-exam-assignments-and-essays
• Susnjak, T. (2022). ChatGPT: The end of online exam integrity? Preprint. ArXiv:2212.09292v1. https://doi.org/10.48550/arXiv.2212.09292
• Taecharungroj, V. (2023). What can ChatGPT do? Analyzing early reactions to the ınnovative AI chatbot on Twitter. Big Data and Cognitive Computing, 7(1),35. https://doi.org/10.3390/bdcc7010035
• Thorat, S.A. & Jadhav, V. (2020). A review on implementation issues of rule-based chatbot systems. In Proceedings of the International Conference on Innovative Computing & Communications (ICICC); SSRN: Rochester, NY, USA, 2020. http://dx.doi.org/10.2139/ssrn.3567047
• Xu Y., Chen L,, Wei J., Ananiadou S., Fan Y., Qian Y, Chang E.I., Tsujii J. (2015). Bilingual term alignment from comparable corpora in english discharge summary and chinese discharge summary. BMC Bioinform., 16:149:1–149:10.