Öznitelik Belirleme Yöntemleriyle Farklı Türde Hata Ayıklamada Önemli Göz Hareket Metriklerinin Belirlenmesi

Year 2023, Volume: 43 Issue: 2, 757 - 784, 02.09.2023

Arif Akçay , Arif Altun

https://doi.org/10.17152/gefad.1199096

Abstract

Bir programlama süreci içerisinde karşılaşılan olumsuz durumlara hata; hataların fark edilmesi, aranması, bulunması, çözümlenmesi ve tekrar oluşmaması için önlem alınması işlemlerine hata ayıklama denir. Bir programlama sürecinde farklı türde hatalar (derleme zamanı hataları, çalışma zamanı hataları ve mantık hataları) ile karşılaşılmaktadır. Her hata farklı özelliklere sahip olmakla birlikte bireylerin hata giderirken sergiledikleri davranışlar araştırmalarda ilgi çekicidir. Hata ayıklarken bireylerin bilişsel performanslarına ilişkin çıkarım yapmak için göz izleme yöntemi kullanılır. Araştırmalarda kullanılan göz hareket metrikleri birbirlerinden farklıdır ve neden kullanıldığına yönelik belirsizlik vardır. Bu çalışmanın amacı farklı türde hata ayıklamada önemli göz hareketlerinin belirlenmesidir. Araştırmaya mesleki ve teknik liselerin Bilişim Teknolojileri Bölümlerinde okuyan 51 öğrenci katılmıştır. Öğrencilerin hata ayıklama performanslarını belirlemek için Hata Ayıklama Performansı Testi kullanılmıştır. Göz hareketlerini belirlemek için Gazepoint GP3 göz izleme aracı kullanılmıştır. Araştırmanın analizlerinde ilgi alanları belirlenmiş ve göz hareketleri, makine öğrenmesinde kullanılan öznitelik belirleme yöntemlerinden olan Bilgi Kazancı ve Gini Katsayısı ile incelenmiştir. Araştırma sonucunda öğrencilerin farklı türde hata ayıklarken sergiledikleri önemli göz hareketlerinin hata türlerinin özelliklerini yansıttığı belirlenmiştir. Son olarak, bu sonuca göre önerilerde bulunulmuştur.

Keywords

Hata, hata ayıklama, hata türleri, göz izleme, göz hareketleri

Determining Important Eye Movement Metrics in Different Types of Debugging with Feature Selection Methods

Abstract

The negative situations encountered in a programming process are called errors; the processes of recognizing, searching, finding, solving and taking measures to prevent recurrence of errors are called debugging. While programming, different types of errors (compile-time bugs, run-time bugs, and logical errors) are encountered. Each type of bug has different characteristics and the behavior of individuals while debugging is interesting in research. The eye tracking method is used to make inferences about the cognitive performance of individuals while debugging. Eye tracking metrics used in research are different and there is uncertainty as to why they are used. The aim of this study is to determine the important eye movements in different types of debugging. 51 students studying at the Information Technologies Departments of vocational and technical high schools participated in the research. The Debugging Performance Test was used to determine the debugging performance of the students. Also, the Gazepoint GP3 Eye Tracker tool was used to detect eye movements. In the analysis of the research, areas of interest were determined and eye movements were examined with Information Gain and Gini Index. The results indicated that eye movements while debugging change according to the characteristics of the bug types. Thus, certain metrics should be utilized depending on the debugging types. Finally, suggestions were made according to this result.

Keywords

Bug, debugging, types of bugs, eye-tracking, eye movements

References

• Ahmadzadeh, M., Elliman, D., and Higgins, C. (2005). An analysis of patterns of debugging among novice computer science students. ACM SIGCSE Bulletin, 37(3), 84–88. https://doi.org/10.1145/1151954.1067472
• Akçay, A., ve Altun, A. (2021). Hata ayıklama performansı testi geliştirme: Geçerlik ve güvenirlik çalışması. Erzincan University Journal of Education Faculty, 23(3), 667-685. https://doi.org/10.17556/erziefd. 815922
• Baig, Z. A., Shaheen, A. S., and AbdelAal, R. (2012). One-dependence estimators for accurate detection of anomalous network traffic. International Journal for Information Security Research, 2(4), 202–210. https://doi.org/10.20533/ijisr.2042.4639.2012.0025
• Bednarik, R., and Tukiainen, M. (2004). Visual attention and representation switching in Java program debugging: A study using eye movement tracking. In E. Dunican ve T. R. G. Green (Eds.), Proceedings of 16th Workshop of Psychology of Programming Interest Group (ss. 159–169). Berlin, Germany: Springer. https://www.ppig.org/papers/2004-ppig-16th-bednarik/
• Bednarik, R., and Tukiainen, M. (2005). Effects of display blurring on the behavior of novices and experts during program debugging. In G. van der Veer (Ed.), CHI 2005 Conference on Human Factors in Computing Systems (ss. 1204–1207). New York, USA: Association for Computing Machinery. https://doi.org/10.1145/1056808.1056877
• Bednarik, R., and Tukiainen, M. (2007). Validating the restricted focus viewer: A study using eye-movement tracking. Behavior Research Methods, 39(2), 274–282. https://doi.org/10.3758/BF03193158
• Breiman, L. (2001). Random forests. Machine Learning, 45, 5-32. https://doi.org/10.1023/A:1010933404324
• Chandrika, K. R., and Amudha, J. (2017). An eye tracking study to understand the visual perception behavior while source code comprehension. International Journal of Control Theory and Applications, 10(19), 169–175. https://serialsjournals.com/abstract/11377_20-chandrika_k.r..pdf
• Chen, M., and Lim, V. (2013). Eye gaze and mouse cursor relationship in a debugging task. In Stephanidis C. (Ed.), HCI International 2013 - Posters’ Extended Abstracts (ss. 468–472). Berlin, Germany: Springer. https://doi.org/10.1007/978-3-642-39473-7_93
• Ciolkowski, M., Laitenberger, O., Rombach, D. H., Shull, F., and Perry, D. (2002). Software inspections, reviews and walkthroughs. In W. Tracz (Ed.), Proceedings of the 24th International Conference on Software Engineering (ss. 641–642). New York, USA: Association for Computing Machinery. https://doi.org/10.1145/581339.581422
• Downey, A. B., and Mayfield, C. (2016). Think Java: How to think like a computer scientist. Green Tea Press. https://open.umn.edu/opentextbooks/textbooks/think-java-how-to-think-like-a-computer-scientist
• Duraes, J., Madeira, H., Castelhano, J., Duarte, C., and Branco, M. C. (2016). WAP: Understanding the brain at software debugging. Proceedings - International Symposium on Software Reliability Engineering, ISSRE, 87–92. https://doi.org/10.1109/ISSRE.2016.53
• Elith, J., Leathwick, J. R., and Hastie, T. (2008). A working guide to boosted regression trees. Journal of Animal Ecology, 77, 802–813. https://doi.org/10.1111/j.1365-2656.2008.01390.x
• Fitzgerald, S., Lewandowski, G., McCauley, R., Murphy, L., Simon, B., Thomas, L., and Zander, C. (2008). Debugging: Finding, fixing and flailing, a multi-institutional study of novice debuggers. Computer Science Education, 18(2), 93–116. https://doi.org/10.1080/08993400802114508
• Holland, M. K., and Tarlow, G. (1972). Blinking and mental load. Psychological Reports, 31(1), 119–127. https://doi.org/10.2466/pr0.1972.31.1.119
• Hristova, M., Misra, A., Rutter, M., and Mercuri, R. (2003). Identifying and correcting Java programming errors for introductory computer science students. In S. Grissom (Ed.), Proceedings of the 34th SIGCSE Technical Symposium on Computer Science Education (ss. 153–156). New York, USA: Association for Computing Machinery. https://doi.org/10.1145/611892.611956
• Institute of Electrical and Electronics Engineers. (2010). IEEE standard classification for software anomalies. https://ieeexplore.ieee.org/document/5399061
• Jacob, R. J. K., and Karn, K. S. (2003). Eye tracking in human-computer-interaction and usability in research: Ready to deliver the promises. In R. Radach, J. Hyona, ve H. Deubel (Eds.), The Minds Eyes: Cognitive and Applied Aspects of Eye Movements (ss. 573–605). Amsterdam, Holland: Elsevier. https://doi.org/10.1016/B978-044451020-4/50031-1
• Jian, Y. C., Wu, C. J., and Su, J. H. (2014). Learners’ eye movements during construction of mechanical kinematic representations from static diagrams. Learning and Instruction, 32, 51–62. https://doi.org/10.1016/j.learninstruc.2014.01.005
• Johnson, W. L., Soloway, E., Cutler, B., and Draper, S. (1983). Bug catalogue: I. https://cpsc.yale.edu/research/technical-reports/1983-technical-reports
• Kaliyeva, S. (2013). Bilimsel makalelerin metin işleme yöntemleri ile sınıflandırılması (Yüksek Lisans Tezi). Fen Bilimleri Enstitüsü, Ankara.
• Kaynar, O., Arslan, H., Görmez, Y., ve Işık, Y. E. (2018). Makine öğrenmesi ve öznitelik seçim yöntemleriyle saldırı tespiti. Bilişim Teknolojileri Dergisi, 11(2), 175–185. https://doi.org/10.17671/gazibtd.368583
• Kovari, A., Katona, J., and Costescu, C. (2020). Evaluation of eye-movement metrics ina software debugging task using GP3 eye tracker. Acta Polytechnica Hungarica, 17(2), 57–76. https://doi.org/10.12700/APH.17.2.2020.2.4
• Ladha, L., and Deepa, T. (2011). Feature selection methods and algorithms. International Journal on Computer Science and Engineering, 3(5), 1787–1797. http://www.enggjournals.com/ijcse/doc/IJCSE11-03-05-051.pdf
• Lin, Y. T., Wu, C. C., Hou, T. Y., Lin, Y. C., Yang, F. Y., and Chang, C. H. (2016). Tracking students’ cognitive processes during program debugging-An eye-movement approach. IEEE Transactions on Education, 59(3), 175–186. https://doi.org/10.1109/TE.2015.2487341
• Majooni, A., Masood, M., and Akhavan, A. (2016). An eye tracking experiment on strategies to minimize the redundancy and split attention effects in scientific graphs and diagrams. In G. di Bucchianico ve P. Kercher (Eds.), Advances in Design for Inclusion (ss. 529–539). Berlin, Germany: Springer. https://doi.org/10.1007/978-3-319-41962-6_47
• Mangaroska, K., Sharma, K., Giannakos, M., Traeteberga, H., and Dillenbourg, P. (2018). Gaze-driven design insights to amplify debugging skills: A learner-centred analysis approach. Journal of Learning Analytics, 5(3), 98–119. https://doi.org/10.18608/jla.2018.53.7
• Millî Eğitim Bakanlığı (MEB). (2011). Bilişim teknolojileri̇ alanı çerçeve öğretim programı. http://www.megep.meb.gov.tr/dokumanlar/Çerçeve Öğretim Programı/BİLİŞİM TEKNOLOJİLERİ_ÇÖP_10.pdf adresinden erişilmiştir.
• Obaidellah, U., al Haek, M., and Cheng, P. C.-H. (2018). A survey on the usage of eye-tracking in computer programming. ACM Computing Surveys, 51(1), 1–58. https://doi.org/10.1145/3145904
• Peng, F., Li, C., Song, X., Hu, W., and Feng, G. (2016). An eye tracking research on debugging strategies towards different types of bugs. In S. I. Ahamed, C. K. Chang, W. Chu, I. Crnkovic, P.-A. Hsiung, G. Huang, ve J. Yang (Eds.), 2016 IEEE 40th Annual Computer Software and Applications Conference (ss. 130–134). New York, USA: IEEE. https://doi.org/10.1109/COMPSAC.2016.57
• Porkodi, R. (2014). Comparison of filter based feature selection algorithms: An overview. International Journal of Innovative Research in Technology ve Science, 2(2), 108–113. http://ijirts.org/volume2issue2/IJIRTSV2I2034.pdf
• Robins, A., Haden, P., and Garner, S. (2006). Problem distributions in a CS1 course. In D. Tolhurst ve S. Mann (Eds.), Conferences in Research and Practice in Information Technology Series (ss. 165–173). New York, USA: Association for Computing Machinery. https://dl.acm.org/doi/10.5555/1151869.1151891
• Scheiter, K., and Eitel, A. (2015). Signals foster multimedia learning by supporting integration of highlighted text and diagram elements. Learning and Instruction, 36, 11–26. https://doi.org/10.1016/j.learninstruc.2014.11.002
• Scheiter, K., and Eitel, A. (2017). The use of eye tracking as a research and instructional tool in multimedia learning. In C. Was, F. Sansosti, ve B. Morris (Eds.), Eye-Tracking Technology Applications in Educational Research (ss. 143–164). Hershey, PA, USA: IGI Global. https://doi.org/10.4018/978-1-5225-1005-5.ch008
• Schwonke, R., Berthold, K., and Renkl, A. (2009). How multiple external representations are used and how they can be made more useful. Applied Cognitive Psychology, 23, 1227–1243. https://doi.org/10.1002/acp.1526
• Sharafi, Z., Marchetto, A., Susi, A., Antoniol, G., and Guéhéneuc, Y.-G. (2013). An empirical study on the efficiency of graphical vs. textual representations in requirements comprehension. In H. Kagdi, D. Poshyvanyk, ve M. di Penta (Eds.), 2013 21st International Conference on Program Comprehension (ICPC) (ss. 33–42). New York, USA: IEEE. https://doi.org/10.1109/ICPC.2013.6613831
• Sharafi, Z., Soh, Z., Guéhéneuc, Y. G., and Antoniol, G. (2012). Women and men - Different but equal: On the impact of identifier style on source code reading. 2012 20th IEEE International Conference on Program Comprehension (ICPC), 27–36. https://doi.org/10.1109/ICPC.2012.6240505
• Sharif, B., Falcone, M., and Maletic, J. I. (2012). An eye-tracking study on the role of scan time in finding source code defects. In S. N. Spencer (Ed.), Proceedings of the Symposium on Eye Tracking Research and Applications, (ss. 381–384). New York, USA: Association for Computing Machinery. https://doi.org/https://doi.org/10.1145/2168556.2168642
• Sharma, K., Mangaroska, K., Giannakos, M., and Dillenbourg, P. (2018). Interlacing gaze and actions to explain the debugging process. In J. Kay, ve R. Luckin (Ed.) 13th International Conference of the Learning Sciences (ss. 640–647). London, UK: International Society of the Learning Sciences. http://hdl.handle.net/11250/2582513
• Shojaeizadeh, M., Djamasbi, S., and Trapp, A. C. (2016). Density of gaze point within a fixation and information processing behavior. In G. Goos, J. Hartmanis, ve J. van Leeuwen (Eds.), Universal Access in Human-Computer Interaction: Methods, Techniques, and Best Practices (ss. 465–471). Berlin, Germany: Springer. https://doi.org/10.1007/978-3-319-40250-5_44
• Turner, R., Falcone, M., Sharif, B., and Lazar, A. (2014). An eye-tracking study assessing the comprehension of C++ and Python source code. In P. Qvarfordt ve D. W. Hansen (Eds.), Proceedings of the Symposium on Eye Tracking Research and Applications - ETRA ’14 (ss. 231–234). New York, USA: Association for Computing Machinery. https://doi.org/https://doi.org/10.1145/2578153.2578218
• van Gog, T., Kester, L., Nievelstein, F., Giesbers, B., ve Paas, F. (2009). Uncovering cognitive processes: Different techniques that can contribute to cognitive load research and instruction. Computers in Human Behavior, 25(2), 325–331. https://doi.org/10.1016/j.chb.2008.12.021
• van Meeuwen, L. W., van Merriënboer, J. J. G., Jarodzka, H., Brand-Gruwel, S., Kirschner, P. A., ve de Bock, J. J. P. R. (2014). Identification of effective visual problem solving strategies in a complex visual domain. Learning and Instruction, 32, 10–21. https://doi.org/10.1016/j.learninstruc.2014.01.004
• Yen, C. Z., Wu, P. H., and Lin, C. F. (2012). Analysis of experts’ and novices’ thinking process in program debugging. In K. C. Li, F. L. Wang, K. S. Yuen, S. K. S. Cheung, ve R. Kwan (Eds.), Engaging Learners Through Emerging Technologies (ss. 122–134). Berlin, Germany: Springer. https://doi.org/10.1007/978-3-642-31398-1_12