The Opinions of Field Experts on Online Test Applications and Test Security During the COVID-19 Pandemic

Year 2021, Volume: 8 Issue: 4, 975 - 990, 04.12.2021

Hakan Kılınç , Muhammet Recep Okur , İlker Usta

https://doi.org/10.21449/ijate.875293

Cited By: 1

Abstract

Within the scope of this study, it was aimed to determine the factors that should be considered regarding the usability and security of online test applications used as an assessment and evaluation tool during the COVID-19 pandemic. In this context, the case study method was used to obtain the opinions of field experts. Furthermore, in this study, using the focus group interview technique as a data collection technique, the criterion sampling method, one of the purposeful sampling methods, was used to determine participants. At this point, it was taken into consideration that the participants were experts in the field of open and distance learning. In this regard, a total of 15 field experts who have experienced online test during the emergency distance education period at Anadolu University, Turkey contributed to the focus group interviews consisting of three groups. The results obtained at the end of the study offer solutions for the usability of online test applications, the use of which has increased with the pandemic process and ensuring the security of these applications.

Keywords

Covid-19, Online test applications, Online test security, Online learning environments, Focus group interview, Case study.

References

• Abduh, M. Y. M. (2021). Full-time online assessment during covid-19 lockdown: EFL teachers’ perceptions. Asian EFL Journal, 28(1), 1-21.
• Albayrak, E. (2014). Elektronik Ortamlardaki Sınavlarda Tasarım Etmenlerinin Öğrencilerin Başarıları ve Elektronik Sınav Kaygılarına Etkisi [The Effects of Design Factors on Students' Success and Test Anxiety in Electronic Tests]. International Online Journal of Educational Sciences, 6(2), 460-474.
• Anderson, H. M., Cain, J., & Bird, E. (2005). Online course evaluations: Review of literature and a pilot study. American Journal of Pharmaceutical Education, 69(1), 34-43.
• Angus, S.D. & Watson J. (2009) Does regular online testing enhance student learning in the numerical sciences? Robust evidence from a large data set. British Journal of Educational Technology, 40, 255–272. https://doi.org/10.1111/j.1467-8535.2008.00916.x
• Bloor, M., Frankland, J., Thomas, M., & Robson, K. (2001). Focus Groups in Social Research. SAGE.
• Borich, G. D. (2013). Effective teaching methods (8 ed.). England: Person Education.
• Bowling, A. (2002). Research Methods in Health: Investigating Health and Health Services. McGraw-Hill House.
• Bozkurt, A. (2020). Koronavirüs (Covid-19) pandemi süreci ve pandemi sonrası dünyada eğitime yönelik değerlendirmeler: Yeni normal ve yeni eğitim paradigması [Coronavirus (Covid-19) pandemic process and educational evaluations in the post-pandemic world: New normal and new education paradigm]. AUAd, 6(3), 112-142.
• Bozkurt, A., & Sharma, R. C. (2020). Emergency remote teaching in a time of global crisis due to CoronaVirus pandemic. Asian Journal of Distance Education, 15(1), i-vi. https://doi.org/10.5281/zenodo.3778083
• Bozkurt, A., & Uçar, H. (2018). E-Öğrenme ve e-sınavlar: Çevrimiçi ölçme değerlendirme süreçlerinde kimlik doğrulama yöntemlerine ilişkin öğrenen görüşlerinin incelenmesi [E-Learning and e-exams: Examination of learner views on identity verification methods in online assessment and evaluation processes]. Mersin Üniversitesi Eğitim Fakültesi Dergisi, 14(2), 745-755. https://doi.org/10.17860/mersinefd.357339
• Can, E. (2020). Coronavirüs (Covid-19) pandemisi ve pedagojik yansımaları: Türkiye’de açık ve uzaktan eğitim uygulamaları [Coronavirus (Covid-19) pandemic and pedagogical implications: open and distance education applications in Turkey]. AUAd, 6(2), 11-53.
• Clough, S. J. (2008). Computerized versus paper and pencil assessment of socially desirable responding: score congruence, completion time, and respondent perefences [Unpublished doctoral dissertation]. The University of Iowa, USA.
• Creswell, J. W. (2007). Qualitative inquiry and research method: Choosing among five approaches. Academic Press. Sage.
• d’Orville, H. (2020). COVID-19 causes unprecedented educational disruption: Is there a road towards a new normal? Prospects, 2020(49), 11-15. https://doi.org/10.1007/s11125-020-09475-0
• Doghonadze, N., Aliyev, A., Halawachy, H., Knodel, L., & Adedoyin, A. S. (2020). The Degree of Readiness to Total Distance Learning in the Face of COVID-19-Teachers’ View (Case of Azerbaijan, Georgia, Iraq, Nigeria, UK and Ukraine). Journal of Education in Black Sea Region, 5(2), 2-41. https://doi.org/10.31578/jebs.v5i2.197
• Ersoy, A. (2013). Türk öğretmen adaylarının kültürlerarası deneyimlerinde karşılaştıkları sorunlar: Erasmus değişim programı örneği [Problems faced by Turkish teacher candidates in their intercultural experiences: Erasmus exchange program example]. Eğitim ve Bilim, 38(168),154-166.
• Fatmasari, R. (2020). Student satisfaction on distance education academic services. In International Conference on Education, Science and Technology (pp. 31-37). Redwhite Press.
• Golden, J., & Kohlbeck, M. (2020). Addressing cheating when using test bank questions in online Classes. Journal of Accounting Education, 52(2020), 1 14. https://doi.org/10.1016/j.jaccedu.2020.100671
• Gülbahar, Y. (2013). E-değerlendirme.[ E-evaluation], K. Çağıltay, Y. Göktaş. (Eds.). Öğretim Teknolojilerinin Temelleri: Teoriler, Araştırmalar, Eğilimler [Foundations of Instructional Technology: Theories, Research, Trends], pp. 651-663. Pegem Akademi Yayıncılık
• Harnkajornsuk, S., Chinda, B., Witayasakpan, S., Wongboonnak, S., & Bunto, P. A. S. (2019). Development of a Web-based Online Examination for Screening Gifted Students. In Proceedings of the 2019 8th International Conference on Educational and Information Technology (pp. 56-60).
• Hewson, C. (2012). Can Online Course-Based Assessment Methods Be Fair and Equitable? Relationships between Students’ Preferences and Performance Within Online and Offline Assessments. Journal of Computer Assisted Learning, 28(5), 488–498. https://doi.org/10.1111/j.1365-2729.2011.00473.x
• Hoque, M. J., Ahmed, M. R., Uddin, M. J., & Faisal, M. M. A. (2020). Automation of Traditional Exam Invigilation using CCTV and Bio-Metric. International Journal of Advanced Computer Science and Applications, 11(6), 392-399.
• Hylton, K., Levy, Y., & Dringus, L. P. (2016). Utilizing webcam-based proctoring to deter misconduct in online exams. Computers & Education, 92(2016), 53-63. https://doi.org/10.1016/j.compedu.2015.10.002
• Jiang, J., Wu, B., Chang, L., Liu, K., & Hao, T. (2019). The Design and Application of an Web-Based Online Examination System. International Symposium on Emerging Technologies for Education (pp. 246-256). Springer, Cham.
• Jisc (2020). The future of assessment: five principles, five targets for 2025. https://repository.jisc.ac.uk/7733/1/the-future-of-assessment-report.pdf
• Jordan S., & Mitchell T. (2009) e-Assessment for learning? The potential of short-answer free-text questions with tailored feedback. British Journal of Educational Technology, 40(2), 371–385. https://doi.org/10.1111/j.1467-8535.2008.00928.x
• Karabekir, M., Tozlu, E., & Şencan, M. N. M. (2015). Girişimci Adayı Üniversite Öğrencilerinin Girişimcilik Özelliklerinin Odak Grup Görüşmesi ile İncelenmesi [Investigation of Entrepreneurial Characteristics of University Students who are Entrepreneur Candidates with a Focus Group Meeting]. SDÜ Fen Edebiyat Fakültesi, Sosyal Bilimler Dergisi, 1(35), 203-216.
• Karthika, R., Vijayakumar, P., Rawal, B. S., & Wang, Y. (2019). Secure Online examination System for e-learning. 2019 IEEE Canadian Conference of Electrical and Computer Engineering (CCECE) (pp. 1-4). IEEE.
• Kaya, Z. (2016). Biyometrik Güvenlik Sistemleri ve Yüz Tanımaya Dayalı Çevrimiçi Sınav Sistemi [Biometric Security Systems and Online Exam System Based on Face Recognition] [Unpublished doctoral dissertation]. İstanbul Aydın University.
• Kınalıoğlu, İ. H., & Güven, Ş. (2011). Uzaktan Eğitim Sisteminde Öğrenci Başarısını Ölçülmesinde Karşılaşılan Güçlükler ve Çözüm Önerileri [Difficulties Encountered in Measuring Student Achievement in Distance Education System and Solution Suggestions]. XIII. Akademik Bilişim Konferansı, Malatya.
• Kitzinger, J. (1995). Qualitative research: introducing focus groups. British Medical Journal, 31(7000), 299–302.
• Koçak, Ş., Yenilmez, E. D., & Yenilmez, E. (2006). Çevrimiçi Sınav Sistemlerinin Öğrenmeye Olan Etkileri Üzerine Bir Çalışma: Öğrenci Görüşleri [A Study on the Effects of Online Exam Systems on Learning: Student Views]. Çukurova Üniversitesi İlahiyat Fakültesi Dergisi, 6(2), 171-189.
• Krueger, R.A. (1994). Focus Groups: A Practical Guide For Applied Research. SAGE.
• Kuhtman, M. (2004). Review of online student ratings of instruction. College and University Journal, 80(1), 64-67.
• Senthil Kumar, A.V., & Rathi, M. (2021). Keystroke Dynamics: A Behavioral Biometric Model for User Authentication in Online Exams. In M. Khosrow-Pour (Ed.), Research Anthology on Developing Effective Online Learning Courses (pp. 1137-1162). IGI Global.
• Kundu, A., & Bej, T. (2021). Experiencing e-assessment during COVID-19: an analysis of Indian students' perception. Higher Education Evaluation and Development, 15(2), 114-134. https://doi.org/10.1108/HEED-03-2021-0032
• Liu, M., Papathanasiou E., & Yung-Wei H. (2001) Exploring the use of multimedia examination formats in undergraduate teaching: results from the fielding testing. Computers in Human Behavior, 17(3), 225–248. https://doi.org/10.1016/S0747-5632(01)00008-5
• Marriot, P. (2009). Students’ Evaluation of The Use of Online Summative Assessment on an Undergraduate Financial Accounting Module. British Journal of Educational Technology, 40(2), 237–254. https://doi.org/10.1111/j.1467-8535.2008.00924.x
• Ortner T. M. & Caspers, J. (2001). Consequences of test anxiety on adaptive versus fixed item testing. European Journal of Psychological Assessment, 27(3),157–163.
• Özen, Z. (2016). Kimlik Doğrulaması için Tuş Vuruş Dinamiklerine Dayalı Bir Güvenlik Sisteminin Yapay Sinir Ağları ile Geliştirilmesi [Development of a Security System Based on Keystroke Dynamics for Authentication with Artificial Neural Networks] [Unpublished doctoral dissertation]. İstanbul University.
• Parker, D. (2010). Our excessively simplistic information security model and how to fix it. ISSA Journal, 8(7), 12-21.
• Patil, S., Sharma, Y. K., & Patil, R. (2019). Implications of Deep Learning-Based Methods for Face Recognition in Online examination System. International Journal of Recent Technology and Engineering, 8(3), 14-27.
• Saban, A., Özer, H. İ., & Tümer, A. E. (2010). Çevrimiçi ders materyalleri ve çevrimiçi sınav sistemi ile ilgili öğrenci görüşleri [Student views on online course materials and online exam system]. E-Journal of New World Sciences Academy, 5(4), 2238-2244.
• Schmidt, R. A., Lee, T. D., Winstein, C. J., Wulf, G., Zelaznik, H. N. (2019). Motor control and learning: a behavioral emphasis. Human Kinetics.
• Senel, S., & Senel, H. C. (2021). Remote Assessment in Higher Education during COVID-19 Pandemic. International Journal of Assessment Tools in Education, 8(2), 181-199. https://doi.org/10.21449/ijate.820140
• Şimşek, İ., Balaban, M., & Ergin, H. (2016). Eğitimde Ölçme ve Değerlendirme Çalışmalarında Web Tabanlı Uzman Sınav Sisteminin Kullanımı Üzerine Bir Araştırma [A Research on the Use of Web Based Examination System in Education Measurement and Evaluation Studies]. Hasan Ali Yücel Eğitim Fakültesi Dergisi, 13(26), 165-179.
• Sindre, G., & Vegendla, A. (2015). E-exams versus paper exams: A comparative analysis of cheating-related security threats and countermeasures. NISK Journal, 8(1), 34-45.
• Sırakaya, M., Sırakaya, D. A., & Çakmak, E. K. (2015). Uzaktan Eğitim Öğrencilerinin Çevrimiçi Sinava Yönelik Tutum Düzeylerinin İncelenmesi [Investigation of Distance Education Students' Attitudes towards Online Exam]. Kastamonu Eğitim Dergisi, 23(1), 87-104.
• Solak, H.İ., Ütebay, G., & Yalçın, B. (2020). Uzaktan eğitim öğrencilerinin basılı ve dijital ortamdaki sınav başarılarının karşılaştırılması [Comparison of distance education students' exam success in print and digital media]. AUAd, 6(1), 41-52.
• UN (1948). Universal Declaration of Human Rights. https://www.un.org/en/universal-declaration-human-rights/index.html
• UNESCO (2020a). School closures caused by Coronavirus (Covid 19). https://en.unesco.org/covid19/educationresponse
• UNESCO (2020b). Exams and assessments in COVID-19 crisis: fairness at the centre. https://en.unesco.org/news/exams-and-assessments-covid-19-crisis-fairness-centre
• UNESCO (2020c). Distance Learning Strategies, What do we know about effectiveness? http://www.unesco.org/covid19EDwebinar
• Wang, F. (2017). Research on the paperless examination in the university public computer laboratory. Proceedings of the 2017 International Conference on E-Society, E-Education and E-Technology (pp. 52-55). ACM Digital Library.
• Yağcı, M. Ekiz, H., & Gelbal, S. (2011). Çevrimiçi sınav ortamlarının öğrencilerin akademik başarılarına etkisi [The effect of online exam environments on students' academic success]. 5th International Computer & Instructional Technologies Symposium, Fırat University, Elazığ, Turkey.
• Yağcı, M., Ekiz, H.,& Gelbal, S. (2015). Yeni Bir Çevrimiçi Sınav Modeli Geliştirilmesi ve Uygulanması [Developing and Implementing a New Online Exam Model]. Ahi Evran Üniversitesi Kırşehir Eğitim Fakültesi Dergisi, 16(1), 269-288.
• Yıldırım, A., & Şimşek, H. (2008). Sosyal Bilimlerde Nitel Araştırma Yöntemleri (6. Baskı) [Qualitative research methods in the social sciences. (6th Edition)]. Seçkin Yayınevi.
• Yıldırım, A., & Şimşek, H. (2011). Sosyal Bilimlerde Nitel Araştırma Yöntemleri (8. Basım) [Qualitative research methods in the social sciences. (8th Edition)]. Seçkin Yayınları
• Yılmaz, Ö. (2016). Çevrimiçi sınav görüş anketi [Online examination assessment survey]. e-Kafkas Eğitim Araştırmaları Dergisi, 3(3), 26-33.
• Yin, R. K. (1984). Case Study Research: Design and Methods. Sage Publications.
• Yin, R. K. (2003). Case study research: Design and methods (3rd ed.). Sage.
• Zhao, Y. (2020). COVID-19 as a catalyst for educational change. Prospects, 49(1), 29-33. https://doi.org/10.1007/s11125-020-09477-y