Modelling the Differences in Social and Emotional Skills with Polytomous Explanatory IRT: The Example of Assertiveness Skill

Year 2023, Volume: 14 Issue: 4, 508 - 523, 31.12.2023

Fatma Nur Aydın , Kübra Atalay Kabasakal

https://doi.org/10.21031/epod.1299468

Abstract

Explanatory item response theory models can simultaneously decompose the covariance between persons and items, as well as analyze items by adding item-related predictors for differences between item difficulties and/or person-related predictors for differences between individuals. In the current study, we calculated the parameter estimations regarding the skill of assertiveness according to the rating scale model and partial credit model, which are descriptive (traditional) item response theory models as well as latent regression partial credit model including only person-level predictors, and then examined the results comparatively. We used the raw score belonging to the skill of assertiveness of Türkiye belonging to the OECD Social and Emotional Skills Study, and we included gender, socioeconomic level, perceived relationships with teachers, bullying at school, sense of belonging at school, global mindedness, and test anxiety as person-level predictors. Current study findings suggests that; (1) the latent regression partial credit model produces a better data fit when compared to the rating scale model and partial credit model, and (2) sense of belonging at school, global mindedness, and socioeconomic level are significant predictors to explain the differences between persons. We discussed the current study findings in terms of the rich body of knowledge provided by explanatory item response theory and presented some suggestions.

Keywords

polytomous IRT models, polytomous explanatory IRT models, social and emotional skills, assertiveness

References

• Akaike, H. (1974). A new look at the statistical model identification. IEEE Transactions on Automatic Control, 19(6), 716-723. https://doi.org/10.1109/TAC.1974.1100705.
• Andrich, D. (1978a). Application of a psychometric model to ordered categories which are scored with successive integers. Applied Psychological Measurement, 2(4), 581-594. https://doi.org/10.1177/014662167800200413.
• Andrich, D. (1978b). A rating formulation for ordered response categories. Psychometrika, 43 (4), 561-573. https://doi.org/10.1007/BF02293814.
• Atar, B. (2011). Tanımlayıcı ve açıklayıcı madde tepki modellerinin TIMSS 2007 Türkiye matematik verisine uyarlanması. Eğitim ve Bilim, 36 (159), 255-269.
• Atar, B., & Çobanoğlu Aktan, D. (2013). Birey açıklayıcı madde tepki kuramı analizi: Örtük regresyon iki parametreli lojistik modeli. Eğitim ve Bilim, 38 (168), 59-68.
• Baker, F. B. (2016). Madde tepki kuramının temelleri. (N., Güler, Trans Ed.) & (M., İlhan, Trans.). Ankara. https://doi.org/10.14527/9786053185673.
• Bates, D., Maechler, M., Bolker, B., & Walker, S. (2015). Fitting linear mixed-effects models using lme4. Journal of Statistical Software, 67(1), 1-48. https://doi.org/10.18637/jss.v067.i01.
• Briggs, D. C. (2008). Using explanatory item response models to analyze group differences in science achievement. Applied Measurement in Education, 21(2), 89 - 118. https://doi.org/10.1080/08957340801926086.
• Bulut, H. (2019). An R Package for Multivariate Hypothesis Tests: MVTests. Technological Applied Sciences, 14 (4), 132-138. https://doi.org/10.12739/NWSA.2019.14.4.2A0175.
• Bulut, O. (2021). eirm: Explanatory item response modeling for dichotomous and polytomous item responses. https://CRAN.R-project.org/package=eirm.
• Bulut, O., Görgün, G., & Yıldırım Erbaşlı, S. N. (2021). Estimating explanatory extensions of dichotomous and polytomous Rasch models: The eirm package in R. Psych, 3(3), 308-321. https://doi.org/10.3390/psych3030023.
• Büyükkıdık, S., & Bulut, O. (2022). Analyzing the effects of test, student, and school predictors on science achievement: An explanatory IRT modelling approach. Journal of Measurement and Evaluation in Education and Psychology, 13(1), 40-53. https://doi.org/10.21031/epod.1013784.
• Büyüköztürk, Ş., Kılıç Çakmak, E., Akgün, Ö., Karadeniz, Ş., & Demirel, F. (2014). Bilimsel araştırma yöntemleri. Pegem Akademi.
• Castedo, A. L., Juste, M. P., & Alonso, J. D. (2015). Social competence: Evaluation of assertiveness in Spanish adolescents. Psychological reports, 116 (1), 219-229. https://doi.org/10.2466/21.PR0.116k12w5.
• Chalmers, R.P. (2012). mirt: A Multidimensional Item Response Theory Package for the R Environment. Journal of Statistical Software, 48 (6), 1-29. https://doi.org/10.18637/jss.v048.i06.
• Christensen, K.B., Makransky, G., & Horton, M.C. (2017) Critical values for yen’s q3: identification of local dependence in the rasch model using residual correlations. Applied Psychological Measurement, 41 (3), 178-194. https://doi.org/10.1177/0146621616677520.
• Crocker, L. & Algina, J. (1986). Introduction to classical and modern test theory. Holt, Rinehart and Winston.
• De Boeck, P., & Wilson, M. (2004). Explanatory item response models: A generalized linear and nonlinear approach. Springer-Verlag.
• Demirkol, S., & Ayvallı Karagöz, M. (2023). PISA 2015 okuma becerisi maddelerinin güçlük indeksini etkileyen madde özelliklerinin incelenmesi. Ana Dili Eğitimi Dergisi, 11(3), 567-579. https://doi.org/10.16916/aded.1212049
• Demirkol, S., & Kelecioğlu, H. (2022). Analyzing the interaction of item position effect and student characteristics within explanatory IRT models. Journal of Measurement and Evaluation in Education and Psychology, 13(4), 282-304. https://doi.org/10.21031/epod.1126368
• Embretson, S. E., & Reise, S. P. (2000). Item response theory for psychologists. Lawrence Erlbaum Associates Inc.
• Eskin, M. (2003). Self‐reported assertiveness in Swedish and Turkish adolescents: A cross‐cultural comparison. Scandinavian Journal of Psychology, 44(1), 7-12. https://doi.org/10.1111/1467-9450.t01-1-00315.
• Hambleton, R. K., & Swaminathan, H. (1985). Item response theory: Principles and applications. Springer Science & Business Media.
• Hebbali A. (2020). olsrr: Tools for building OLS regression models. https://CRAN.R-project.org/package=olsrr.
• Kahraman, N. (2014). An explanatory item response theory approach for a computer-based case simulation test. Eurasian Journal of Educational Research, 14 (54), 117-134. https://doi.org/10.14689/ejer.2014.54.7.
• Kankaraš, M., & Suarez-Alvarez, J. (2019). Assessment framework of the OECD study on social and emotional skills. OECD Publishing. https://doi.org/10.1787/5007adef-en.
• Karaaslan Arkan, R. (2016). Ergenlerin atılganlık, utangaçlık ve yalnızlık düzeyleri. [Unpublished master thesis]. Beykent Üniversitesi.
• Kaya, Z., & Karaca, R. (2018). Ergenlerin atılganlık ve sürekli kaygı düzeylerinin bazı değişkenlere göre incelenmesi. Van Yüzüncü Yıl Üniversitesi Eğitim Fakültesi Dergisi, 15(1), 1490-1516. https://doi.org/10.23891/efdyyu.2018.113
• Keliat, B. A., Tololiu, T. A., Daulima, N. H. C., & Erawati, E. (2015). Effectiveness assertive training of bullying prevention among adolescents in West Java Indonesia. International Journal of Nursing, 2(1), 128-134. https://doi.org/10.15640/ijn.v2n1a14.
• Kılıç, G. (2009). Lise öğrenimi görmekte olan ergenlerin atılganlık düzeylerinin ebeveynlerine bağlanma örüntülerine ve bazı demografik değişkenlere göre incelenmesi: Darıca ilçesi örneği. [Unpublished master thesis]. Maltepe Üniversitesi.
• Kim, J., & Wilson, M. (2020). Polytomous item explanatory item response theory models. Educational and Psychological Measurement, 80(4), 726-755. https://doi.org/10.1177/0013164419892667.
• Masters, G. N. (1982). A Rasch model for partial credit scoring. Psychometrika, 47(2), 149-174. https://doi.org/10.1007/BF02296272.
• Nunnally, J., & Bernstein, I. (1994), Psychometric Theory, McGraw-Hill.
• OECD. (2015). Skills for social progress: The power of social and emotional skills. OECD Publishing. https://doi.org/10.1787/9789264226159-en.
• OECD. (2021a). Beyond academic learning: First results from the survey of social and emotional skills. OECD Publishing. https://doi.org/10.1787/92a11084-en.
• OECD. (2021b). OECD Survey on social and emotional skills technical report. OECD Publishing.
• R Core Team (2022). R: A language and environment for statistical computing. https://www.R-project.org/.
• Revelle, W. (2022) Psych: procedures for personality and psychological research. https://CRAN.R-project.org/package=psych.
• Rizopoulos, D. (2006). ltm: An R package for latent variable modelling and item response theory analyses. Journal of Statistical Software, 17 (5), 1-25. https://doi.org/10.18637/jss.v017.i05.
• Samejima, F. (1969). Estimation of latent ability using a response pattern of graded scores. Psychometrika, 34 (1), 1-97. https://doi.org/10.1007/BF03372160.
• Schwarz, G. (1978). Estimating the dimension of a model. The Annals of Statistics, 6(2), 461–464. https://doi.org/10.1214/aos/1176344136.
• Sitota, G. (2018). Assertiveness and academic achievement motivation of adolescent students in selected secondary schools of Harari peoples regional state, Ethiopia. International Journal of Education and Literacy Studies, 6(4), 40-46. https://doi.org/10.7575/aiac.ijels.v.6n.4p.40.
• Stake, J. E., DeVille, C. J., & Pennell, C. L. (1983). The effects of assertive training on the performance self-esteem of adolescent girls. Journal of Youth and Adolescence, 12(5), 435-442. https://doi.org/10.1007/BF02088725.
• Stanke, L., & Bulut, O. (2019). Explanatory item response models for polytomous item responses. International Journal of Assessment Tools in Education, 6(2), 259–278. https://doi.org/10.21449/ijate.515085.
• Tuerlinckx, F., & Wang, W.C. (2004). Models for polytomous data. In P. De Boeck & M. Wilson (Eds.), Explanatory item response models: A generalized linear and nonlinear approach (pp. 75–109). Springer-Verlag.
• Vansteelandt, K. (2000). Formal models for contextualized personality psychology [Unpublished doctoral dissertation]. K.U.Leuven.
• Wickham H. (2022). stringr: Simple, consistent wrappers for common string operations. https://CRAN.R-project.org/package=stringr.
• Wickham H., François R., Henry L., & Müller K. (2022). dplyr: A grammar of data manipulation. https://CRAN.R-project.org/package=dplyr.
• Wickham H., Miller E., & Smith D. (2022). haven: Import and export 'SPSS', 'Stata' and 'SAS' files. https://CRAN.R-project.org/package=haven.
• Wickham, H. (2016). ggplot2: Elegant graphics for data analysis. Springer.
• Wilson, M., & De Boeck, P. (2004). Descriptive and explanatory item response models. In P. De Boeck & M. Wilson (Eds.), Explanatory item response models: A generalized linear and nonlinear approach (pp. 43–74). Springer-Verlag