Measuring cognitive loads and attitudes of pre-service teachers in computer based testing environment

Year 2024, , 84 - 101, 31.01.2024

Mehmet Mevlüt Odacı , Erman Uzun

https://doi.org/10.31681/jetol.1366359

Abstract

This study aimed to reveal the attitudes of the students who were subjected to measurement and evaluation in an online testing environment towards the computer based testing (CBT) platform and the factors that affect their attitudes. It also examined the students’ cognitive loads in the exam designed with multimedia elements from message design principles by using the facilities of this platform. The study was conducted with a case study design. The participants consisted of 32 students. Participants took a previously used and redesigned English achievement test based on the multimedia design principle. Then, cognitive load scale and attitude towards computer scale were implemented. In addition, seven volunteer participants were interviewed. The results of the study showed that factors such as students’ equipment advantage, time, success, flexibility, individuality and less mental effort affected the participants’ attitudes towards the CBT environment. A significant difference was found between the scores of the participants in the cognitive load scale and the duration of the exam. There was a low and positive correlation between the cognitive load score of the participants for the listening test and the time to complete these questions, and a low and negative correlation between the correct answer scores of the participants in the listening test.

Keywords

Cognitive load theory, principles of message design, online testing, computer based testing (CBT), case study.

Ethical Statement

This research is based on a M.A. thesis submitted to the Institute of Educational Sciences, Mersin University (Odacı, 2019) which was supported by Mersin University, Scientific Research Project Office, under the project number 2019-2-TP2-3524.

Supporting Institution

Mersin University, Scientific Research Project Office

Project Number

2019-2-TP2-3524

References

• Akdemir, Ö. & Oğuz, A. (2008). Computer-based testing: an alternative for the assessment of Turkish undergraduate students, Computers and Education, 1, 1198-1204. https://doi.org/10.1016/j.compedu.2007.11.007
• Aybek, E. C. (2012). Kâğıt-kalem formu ve bilgisayar ortamında uygulanan genel yetenek testinin psikometrik özelliklerinin karşılaştırılması [Master’s thesis, Ankara University]. YÖK Açıkbilim. https://acikbilim.yok.gov.tr/handle/20.500.12812/85211
• Brünken, R., Seufert, T., & Paas, F. (2010). Measuring Cognitive Load. In J. L. Plass, R. Moreno, & R. Brünken (Eds.), Cognitive Load Theory (pp. 181–202). Cambridge University Press.
• Bugbee, A.C. (1996). The Equivalence of Paper-and-Pencil and Computer-Based Testing. Journal of Research on Computing in Education, 28(3), 282. Retrieved September 19, 2023 from https://www.learntechlib.org/p/80961/.
• Büyüköztürk, Ş., Kılıç Çakmak, E., Akgün, Ö. E., Karadeniz Oran, Ş., & Demirel, F., (2019). Bilimsel Araştırma Yöntemleri. Pegem Akademi.
• Çaka, C. & Dursun, Ö.Ö. (2022). Evaluation of the effectiveness of different infographic designs. Journal of Educational Technology & Online Learning, 5(3), 519-534. https://doi.org/10.31681/jetol.1075352
• Çetin, İ. (2016). Nitel içerik analizi. In Özden, M. Y. & Durdu, L. (Eds.), Eğitimde Üretim Tabanlı Çalışmalar İçin Nitel Araştırma Yöntemleri (pp. 125-148). Anı Yayıncılık.
• Dinar, M., Kabakçı Yurdakul, I. & Dönmez, F. (2015). Measuring Cognitive Load in Test Items: Static Graphics versus Animated Graphics. Journal of Computer Assisted Learning, 31, 148-161. https://doi.org/10.1111/jcal.12086
• Fraenkel, J. R., Wallen, N. E., ve Hyun, H. H. (2012). How to design and evaluate research in education (8th ed.). McGraw-Hill.
• Hamad, W. (2023). Students’ experiences of learning in virtual classrooms facilitated by Google Classroom. Journal of Educational Technology and Online Learning, 6 (2), 362-383. http://doi.org/10.31681/jetol.1250095
• Kirschner, P.A. (2003). Cognitive Load Theory: Implications of Cognitive Load Theory on the Design of Learning, Learning and Instruction, 12, 1-10. https://doi.org/10.1016/S0959-4752(01)00014-7
• Kılıç, E. (2009). The effects of cognitive load in learning from goal based scenario designed multimedia learning environment for learners having different working memory capacities (Publication No. 255259) [Doctoral Dissertation, METU]. YÖK Tez, https://tez.yok.gov.tr/UlusalTezMerkezi/TezGoster?key=NtBAevXNhYaNqJFoAcdBdoczOCd_zbRfD8kVAKPsDubMuN-FCZVksYHFE-wH0v9e
• Kılıç, E. ve Karadeniz, Ş. (2004). Hiper ortamlarda öğrencilerin bilişsel yüklenme ve kaybolma düzeylerinin belirlenmesi. Kuram ve Uygulamada Eğitim Yönetimi, 40, 56—579. Retrieved from https://dergipark.org.tr/tr/pub/kuey/issue/10357/126808
• Lemmo, A., Mariotti, M-A. (2017, 1-5 February). From paper and pencil to computer-based assessment: some issues raised in the comparison, In. Dooley, T. & Gueudet, G. (Eds.), Proceedings of the Tenth Congress of the European Society for Research in Mathematics Education, DCU Institute of Education & ERME, 3540 – 3547.
• Lindner, M. A., Eitel, A., Barenthien, J., & Köller, O. (2018). An integrative study on learning and testing with multimedia: Effects on students’ performance and metacognition. Learning and Instruction, 1(2), 1-9. https://doi.org/10.1016/j.learninstruc.2018.01.002
• Mayer, R.E. (2003). The Promise of Multimedia Learning: Using The Same İnstructional Design Methods Across Different Media, Learning and Instruction, 13, 125-139. https://doi.org/10.1016/S0959-4752(02)00016-6
• Özturan, T. (2016). The impact of computer assisted assessment on the exam success and attitudes of prospective English teachers. (Publication No. 435226) [ Master’s thesis, Hacettepe University]. YÖK Tez, https://tez.yok.gov.tr/UlusalTezMerkezi/TezGoster?key=cbOXH84ZayrLjc0tI-QXKtTWLLFqln0MiUoLfO-MxUyK83GNRlTkrzR5Bs7ix3I1
• Paas, F. G., & Van Merriënboer, J. J. (1994). Instructional control of cognitive load in the training of complex cognitive tasks. Educational Psychology Review, 6, 351-371. https://doi.org/10.1007/BF02213420
• Paas, F., Renkl, A. & Sweller, J. (2003). Cognitive load theory and instructional design: recent developments, Educational Psychologist, 38(1), 1-4. https://doi.org/10.1207/S15326985EP3801_1
• Prisacari, A.A. & Danielson, J. (2017). Computer-based versus paper-based testing: Investigating testing mode with cognitive load and scratch paper use, Computers in Human Behavior, 77, 1-10. https://doi.org/10.1016/j.chb.2017.07.044
• Şenel, S. (2018). Programlama Konusu “Kazanımları” ile “Sınıf İçi Ölçme Süreçlerinin” Bilişsel Düzeylerinin Karşılaştırılması. International Journal of Computers in Education, 1 (2), 1-20. Retrieved from https://dergipark.org.tr/en/pub/ijce/issue/42323/486158
• Şenel, S., Pekdağ B., & Günaydın, S. (2018). Kimya öğretmenlerinin ölçme ve değerlendirme sürecinde yaşadıkları problemler ve yetersizlikler. Necatibey Eğitim Fakültesi Elektronik Fen ve Matematik Eğitimi Dergisi (EFMED). 12(1). 419-441. https://doi.org/10.17522/balikesirnef.437824
• Sweller, J., van Merrienboer, J.J.G. & Paas FGWC. (1998). Cognitive Architecture and Instructional Design, Educational Psychology Review, 10(3), 251-296. https://doi.org/10.1023/A:1022193728205
• Takır, A. (2011). The effect of an instruction designed by cognitive load theory principles on 7th grade students' achievement in algebra topics and cognitive load. (Publication No. 255259) [ Doctoral Dissertation, METU]. YÖK Tez, https://tez.yok.gov.tr/UlusalTezMerkezi/TezGoster?key=EEdeQgIdFRxX5NbvVau-AhjV-QzHr495E11DeqZt2L6gI88s6pwJxJAMh4_IhC9C
• Woodside, A. G. (2010). Case Study Research: Theory, Methods, Practice. Emerald Group Pub. Ltd., ebrary Inc.
• Yıldırım, A. & Şimşek, H. (2018). Sosyal Bilimlerde Nitel Araştırma Yöntemleri. Seçkin Yayıncılık.
• Yıldırım, Z. (2016). Öğretim teknolojileri ve ileti tasarımı. In Çağıltay, K. & Göktaş, Y. (Eds), Öğretim Teknolojilerinin Temelleri: Teoriler, Araştırmalar, Eğilimler (279-296). Pegem Akademi.