Comparison of Methods of Affect Transition Analysis: An Example of SimInClass Dataset

Year 2022, Volume: 13 Issue: 3, 232 - 243, 30.09.2022

Şeyma Çağlar Özhan , Arif Altun

https://doi.org/10.21031/epod.1051716

Abstract

Studies of emotional-cognitive sequences are the growing body of research area in educational context. These studies focus on how emotions change during the learning-teaching process due to their dynamic nature. In affect transition studies, the change of emotion, depending on the event and time, is usually analyzed by using (a) lag sequential analysis (LSA), (b) L metric, (c) L* metric, and (d) Yule's Q metric. Yet, various methodological criticisms exist in the literature while utilizing these sequential analysis methods. In this study, it is aimed to comparatively examine lag analysis, L metric, L* metric, and Yule’s Q in terms of proportion of invalid values, maximum transition metrics, minimum transition metrics, and analysis results. For this reason, the emotional states of the fifteen prospective teachers were collected and their emotions were labeled every 0.5 seconds with EEG (Electroencephalogram), GSR (Galvanic Skin Response), and Microsoft Kinect in a teacher training simulator (SimInClass). The dataset contained 17570 emotions, and the data were analyzed by utilizing lag analysis, L, L* and Yule's Q. The results showed that LSA had yielded the most proportion of invalid results. In addition, it was observed that the number of invalid values increased as the segment length became shorter in all analysis methods. When the maximum and minimum transition metric values were examined, it was found that as the sequence length increased in L and L* analyses, the value of the metrics approached 1, which is the largest value they can reach. However, it was noted that the lag analysis maximum-minimum transition metrics fluctuate independently from the sequence length. It was concluded that there were differences in the analysis results produced by the four sequential analysis methods with the same functions. It was thought that this situation might be due to the different invalid results produced by the analyses. When the results were compared with the studies in the literature, it was thought that it would be beneficial to pay attention to the nature of the data (emotional or behavioral), the data type (singe modality or multimodal modality), the amount of data (short sequences or long sequences), the environment in which the dataset was created (computer-based or not), and the sampling rate (automated data collection tool or observation) when choosing sequential analysis methods.

Keywords

affect, affect transition, sequential analysis

Supporting Institution

TÜBİTAK

Project Number

117R036

Thanks

This work was supported by the Scientific and Technological Research Council of Turkey (TÜBİTAK), through the project titled “Investigating the Effects of Computer Based Affective Recommendation System on Teacher Trainees Cognitive-Emotional Development” (Grant No:117R036).

References

• Bakeman, R., & Gottman, J. M. (1997). Observing interaction: An introduction to sequential analysis. Cambridge University Press. https://doi.org/10.1017/CBO9780511527685
• Baker, R. S., D'Mello, S. K., Rodrigo, M. M. T., & Graesser, A. C. (2010). Better to be frustrated than bored: The incidence, persistence, and impact of learners’ cognitive–affective states during interactions with three different computer-based learning environments. International Journal of Human-Computer Studies, 68(4), 223-241. https://doi.org/10.1016/j.ijhcs.2009.12.003
• Baker, R. S., Rodrigo, M. M. T., & Xolocotzin, U. E. (2007). The dynamics of affective transitions in simulation problem-solving environments. In A. Paiva, R. Prada & W. Picard (Eds.), International conference on affective computing and intelligent interaction (pp. 666-677). Springer. https://doi.org/10.1007/978-3-540-74889-2_58
• Bayazıt, T. (2018). Event Related Potentials (ERP). Journal of Medical Clinics, 1(1), 59-65. https://dergipark.org.tr/tr/pub/atk/issue/38771/451155
• Benjamini, Y., & Yekutieli, D. (2001). The control of the false discovery rate in multiple testing under dependency. Annals of statistics, 29(4),1165-1188. https://doi.org/10.1214/aos/1013699998
• Bosch, N., & D’Mello, S. (2017). The affective experience of novice computer programmers. International Journal of Artificial Intelligence in Education, 27(1), 181-206. https://doi.org/10.1007/s40593-015-0069-5
• Bosch, N., & Paquette, L. (2021). What’s next? Sequence length and impossible loops in state transition measurement. Journal of Educational Data Mining, 13(1), 1-23. https://eric.ed.gov/?id=EJ1320638
• Botelho, A. F., Baker, R., Ocumpaugh, J., & Heffernan, N. (2018). Studying affect dynamics and chronometry using sensor-free detectors. In E. Boyer & M. Yudelson (Eds.), Proceedings of the 11th international conference on educational data mining (pp. 157–166). EDM. https://files.eric.ed.gov/fulltext/ED593106.pdf
• Cimtay, Y., Ekmekcioglu, E., & Caglar-Ozhan, S. (2020). Cross-subject multimodal emotion recognition based on hybrid fusion. IEEE Access, 8, 168865-168878. https://doi.org/10.1109/ACCESS.2020.3023871
• D’Mello, S., & Graesser, A. (2012). Dynamics of affective states during complex learning. Learning and Instruction, 22(2), 145-157. https://doi.org/10.1016/j.learninstruc.2011.10.001
• D’Mello, S., Taylor, R., & Graesser, A. (2007). Monitoring affective trajectories during complex learning. In McNamara, D., Trafton, J. (Eds.), Proceedings of 29th annual cognitive science society (pp. 203–208). Cognitive Science Society. https://doi.org/10.1007/978-1-4419-1428-6_849
• Ekman (2021, September 7). What is surprise? Paul Ekman. https://www.paulekman.com/universal-emotions/what-is-surprise/
• Frenzel, A. C., Goetz, T., Stephens, E. J., & Jacob, B. (2009). Antecedents and effects of teachers’ emotional experiences: An integrated perspective and empirical test. In P. A. Schutz & M. Zembylas (Eds.), Advances in teacher emotion research: The impact on teachers’ lives (pp. 129-152). Springer. https://doi.org/10.1007/978-1-4419-0564-2_7
• Han, J.-H., Shubeck, K., Shi, G.-H., Hu, X.-E., Yang, L., Wang, L.-J., Zhao, W., Jiang, Q., & Biswas, G. (2021). Teachable agent improves affect regulation: Evidence from Betty’s brain. Educational Technology & Society, 24(3), 194–209. https://www.jstor.org/stable/27032865
• Juslin, P. N., & Sloboda, J. A. (2013). Music and emotion. In D. Deutsch (Ed.), The psychology of music (pp. 583-645). Academic Press. https://doi.org/10.1016/B978-0-12-381460-9.00015-8
• Karumbaiah, S., Baker, R. S., & Ocumpaugh, J. (2019). The case of self-transitions in affective dynamics. In S. Isotani, E. Millán, A. Ogan, P. Hastings, B. McLaren, & R. Luckin (Eds.), International conference on artificial intelligence in education (pp. 172-181). Springer. https://doi.org/10.1007/978-3-030-23204-7_15
• Karumbaiah, S., Baker, R. B., Ocumpaugh, J., & Andres, A. (2021). A re-analysis and synthesis of data on affect dynamics in learning. IEEE Transactions on Affective Computing. https://doi.org/10.1109/TAFFC.2021.3086118
• Kleinginna, P. R., & Kleinginna, A. M. (1981). A categorized list of emotion definitions, with suggestions for a consensual definition. Motivation and Emotion, 5(4), 345-379. https://doi.org/10.1007/BF00992553
• Lajoie, S. P., Zheng, J., Li, S., Jarrell, A., & Gube, M. (2021). Examining the interplay of affect and self regulation in the context of clinical reasoning. Learning and Instruction, 72, 101219. https://doi.org/10.1016/j.learninstruc.2019.101219
• Liu, Z., Zhang, N., Liu, S., & Liu, S. (2021). Development trajectory of student cognitive behaviors in a SPOC forum: An integrated approach combining epistemic network analysis and lag sequential analysis. In L. Lee, L. Wang, Y. Kato & S. Sato (Eds.), 2021 International symposium on educational technology (ISET) (pp. 26-30). IEEE. https://doi.org/10.1109/ISET52350.2021.00016.
• Matayoshi, J., & Karumbaiah, S. (2020). Adjusting the L statistic when self-transitions are excluded in affect dynamics. Journal of Educational Data Mining, 12(4), 1-23. https://eric.ed.gov/?id=EJ1298368
• Matayoshi, J., & Karumbaiah, S. (2021). Using marginal models to adjust for statistical bias in the analysis of state transitions. In M. Scheffel, N. Dowell, S. Joksimovic & G. Siemens (Eds.), LAK21: 11th International learning analytics and knowledge conference (pp. 449-455). Association for Computing Machinery. https://doi.org/10.1145/3448139.3448182
• Rebolledo-Mendez, G., Huerta-Pacheco, N. S., Baker, R. S., & du Boulay, B. (2022). Meta-affective behaviour within an intelligent tutoring system for mathematics. International Journal of Artificial Intelligence in Education, 32(1), 174-195. https://doi.org/10.1007/s40593-021-00247-1
• Pohl, M., Wallner, G., & Kriglstein, S. (2016). Using lag-sequential analysis for understanding interaction sequences in visualizations. International Journal of Human-Computer Studies, 96, 54-66. https://doi.org/10.1016/j.ijhcs.2016.07.006
• Scherer, K. R. (1993). Studying the emotion-antecedent appraisal process: An expert system approach. Cognition and Emotion, 7(3), 325–355. https://doi.org/10.1080/02699939308409192
• Sebe, N., Cohen, I., & Huang, T. S. (2005). Multimodal emotion recognition. In C. Chen & P. Wang (Eds.), Handbook of pattern recognition and computer vision (pp. 387-409). World Scientific. https://doi.org/10.1142/1802
• Sun, J. C. Y., Kuo, C. Y., Hou, H. T., & Lin, Y. Y. (2017). Exploring learners' sequential behavioral patterns, flow experience, and learning performance in an anti-phishing educational game. Journal of Educational Technology & Society, 20(1), 10-20. https://www.proquest.com/scholarly-journals/exploring-learners-sequential-behavioral-patterns/docview/2147743221/se-2
• Sun, Z., Lin, C. H., Lv, K., & Song, J. (2021). Knowledge-construction behaviors in a mobile learning environment: A lag-sequential analysis of group differences. Educational Technology Research and Development, 69(2), 533-551. https://doi.org/10.1007/s11423-021-09938-x
• Wu, S. Y., & Hou, H. T. (2015). How cognitive styles affect the learning behaviors of online problem-solving based discussion activity: A lag sequential analysis. Journal of Educational Computing Research, 52(2), 277-298. https://eric.ed.gov/?id=EJ1076314
• Yang, X., Song, S., Zhao, X., & Yu, S. (2018). Understanding user behavioral patterns in open knowledge communities. Interactive Learning Environments, 26(2), 245-255. https://doi.org/10.1080/10494820.2017.1303518
• Yang, X., Li, J., & Xing, B. (2018). Behavioral patterns of knowledge construction in online cooperative translation activities. The Internet and Higher Education, 36, 13-21. https://doi.org/10.1016/j.iheduc.2017.08.003
• Yule, G. U. (1900). On the association of attributes in statistics: With illustrations from the material of the childhood society. Philosophical Transactions of the Royal Society of London. 66(194), 252–261. https://doi.org/10.1098/rspl.1899.0067