An Analysis of Parameter Invariance according to Different Sample Sizes and Dimensions in Parametric and Nonparametric Item Response Theory

Year 2020, Volume: 11 Issue: 2, 98 - 112, 13.06.2020

Çiğdem Reyhanlıoğlu , Nuri Doğan

https://doi.org/10.21031/epod.584977

Cited By: 1

Abstract

The aim of this study is to analyze the invariance of the estimated parameters in regard to Item Response Theory (IRT) applications within different dimensionality and sample characteristics. To realize this aim, students' answers in test booklet 'A' of the first stage of TEOG, Secondary Education Placement Test applied by the Ministry of Education in Turkey, in 2015 were used as the data. The population size of the study was determined as 63,871. The groups of 50, 100, 200, 500, 1000 and 5000 people who were randomly selected from the population comprised the sample of the study. One-dimensional mathematics subtest and artificially formed two-dimensional test results were used within IRT applications. As a result of the findings obtained from the study within the analysis of the one-dimensional test in regard to one-dimensional nonparametric IRT (ODNPIRT), item parameter invariance was obtained from the sample size 200. When it was analyzed in regard to one-dimensional parametric IRT (ODPIRT), it was found that at least a sample size of 1000 has to be used for item parameter estimations nearly as high as the population value (parameter). It was found that invariance for item parameters was not obtained by analyzing the two-parameter test in regard to ODPIRT, ODNPIRT and Multidimensional IRT (MDIRT).

Keywords

Item Response Theory Applications, Sample Size, Parameter Invariance

References

• Ackerman, T. A. (2005). Multidimensional item response theory modeling. In A. Maydeu-Olivares & J. J. McArdle (Eds.), Multivariate applications book series. Contemporary psychometrics: A festschrift for Roderick P. McDonald (p. 3–25). Lawrence Erlbaum Associates Publishers.
• Andries van der Ark, L. (2007). Mokken Scale Analysis in R. Journal of Statistical Software, 20(11).1-19.
• Ansley, T.N. and Forsyth, R.A. (1985). An Examination of the characteristics of Unidimensional IRT estimates derived from two dimensional data. Applied Psychological Measurement, 9(1), 37-48.
• Şengül Avşar A. (2018). Kategori sayısının psikometrik özellikler üzerine etkisinin mokken homojenlik modeli’ne göre incelenmesi. Eğitimde ve Psikolojide Ölçme ve Değerlendirme Dergisi, 9(1), 49-63.
• Şengül Avşar A. , Tavşancıl E. (2017). Examination of polytomous items’ psychometric properties according to nonparametric item response theory models in different test conditions, Kuram ve Uygulamada Egitim Bilimleri, 17, 493-514.
• Cai L. (2017). flexMIRT Version 3.51: Flexible multilevel multidimensional item analysis and test scoring (Computer software). Chapel Hill, NC: Vector Psychometric Group.
• Cai, L. (2008). SEM of Another Flavour: Two new applications of the supplemented em algorithm. British Journal of Mathematical and Statistical Psychology, 61, 309–329. Çokluk, Ö., Şekercioğlu, G. ve Büyüköztürk, Ş. (2010). Sosyal bilimler için çok değişkenli istatistik teknikleri. Ankara: Pegem Akademi.
• Drasgow, F. & Parsons,K. (1983). Application of unidimensional item response theory models to multidimensional data. Applied Psychological Measurement, 7(2), 189-199.
• Embretson, S. E., & Reise, S. P. (2000). Multivariate applications books series. Item response theory for psychologists. Lawrence Erlbaum Associates Publishers.
• Fraenkel, J.R. & Wallen, N.E. ( 2006). How to design and evaluate research in education (Sixth edition). Boston: McGraw-Hill Pub.
• Goldman, S.H., & Raju, N. S. (1986). Recovery of one- and two-parameter logistic item parameters: An empirical study. Educational And Psychological Measurement, 46(1),11-21.
• Hambleton, R. K. & Swaminathan, H. (1985). Item Response Theory: Principles and applications. Boston: Academic Puslishers Group . Hulin, C. L., Lissak, R. I., & Drasgow, F. (1982). Recovery of two and three-parameter logistic item characteristic curves: A Monte Carlo Study. Applied Psychological Measurement, 6(3), 249-260.
• Harman, H. H. (1962). Modern Factor Analysis (2. edition). University of Chicago Press, Chicago.
• Kaptan, S. (1977). Bilimsel Araştırma Teknikleri, Ankara: Tekışık Matbaası ve Rehber Yayınevi.
• Koğar H. (2018). Examining invariant item ordering using mokken scale analysis for polytomously scored items. Eğitimde ve Psikolojide Ölçme ve Değerlendirme Dergisi, 9(4), 312-325.
• Koğar, H. (2014). Madde tepki kuramının farklı uygulamalarından elde edilen parametrelerin ve model uyumlarının örneklem büyüklüğü ve test uzunluğu açısından karşılaştırılması. (Doktora tezi, Hacettepe Üniversitesi, Eğitimde Ölçme ve Değerlendirme Anabilim Dalı, Ankara). http://tez2.yok.gov.tr/ adresinden edinilmiştir.
• Meara, K., Robin, F. & Sireci, S.G. (2000). Using multidimensional scaling to assess the dimensionality of dichotomous item data. Multivariate Behavioral Research, 35(2), 229–259.
• Mor-Dirlik, E. (2017). Parametrik ve parametrik olmayan madde tepki kuramı modellerinden çeşitli faktörlere göre elde edilen madde ve yetenek kestirimlerinin karşılaştırılması. (Doktora tezi, Ankara Üniversitesi, Eğitimde Ölçme ve Değerlendirme Anabilim Dalı, Ankara). http://tez2.yok.gov.tr/ adresinden edinilmiştir.
• Price, L. R. (2017). Psychometric methods: Theory and practice. New York, NY: The Guilford Press.
• Sijtsma, K., & Molenaar, I. W. (2002). Introduction to Nonparametric Item Response Theory. Thousand Oaks, CA: Sage.
• Smits, I. A. M., Timmerman, M. E., & Meijer, R. R. (2012). Exploratory mokken scale analysis as a dimensionality assessment tool: why scalability does not ımply unidimensionality. Applied Psychological Measurement, 36(6), 516-539.
• Sodano, S. M., and Tracey, T. J. G. (2011). A brief inventory of interpersonal problems- circumplex using non-parametric item response theory: introducing the IIP-C- IRT. Journal of Personality Assessment, 93(1), 62-75. doi:10.1080/00223891.2010.528482
• Syu, J. J. (2013). Applying person fit-in faking detection-the simulation and practice of non parametric item response theory. (Doctoral Dissertation, National Chengchi University). Retrieved from http://nccur.lib.nccu.edu.tw/bitstream/140.119/58646/1/251501.pdf
• Sünbül, Ö. (2011). Çeşitli boyutluluk özelliklerine sahip yapılarda, madde parametrelerinin değişmezliğinin klasik test teorisi, tek boyutlu madde tepki kuramı ve çok boyutlu madde tepki kuramı çerçevesinde i̇ncelenmesi. (Doktora tezi, Mersin Üniversitesi, Eğitimde Ölçme ve Değerlendirme Anabilim Dalı, Ankara). http://tez2.yok.gov.tr/ adresinden edinilmiştir.
• Tabachnick, G. B. ve Fidell, S. L. (2001). Using multivariate statistics (4th Edition), Boston MA: Allyn&Bacon.
• Thissen, D., & Wainer, H. (1982). Some standart errors in item response theory. Psychometrika, 47(4), 397-412.