Research Article
BibTex RIS Cite

Year 2021, Volume: 12 Issue: 2, 129 - 146, 30.06.2021

Esra Sözer , Nilüfer Kahraman

https://doi.org/10.21031/epod.819927
Cited By: 2

Abstract

Bu çalışmada, pozitif duygu durumlarının (mutlu, huzurlu gibi) ölçülmesinde kullanılmak üzere geliştirilen 14 maddelik Pozitif Duygu Durum Ölçeği’nde yer alan likert tipi maddelerin beşli tepki kategorileri için maddeler-içi ve maddeler-arası geçişlilik özellikleri Aşamalı Tepki Modeli, Kısmi Puanlama Modeli ve Dereceli Ölçekleme Modeli kullanılarak karşılaştırmalı olarak incelenmiştir. Her bir model için, model veri uyumu, ölçekte yer alan maddelerin tepki kategorileri, kategori eşik (threshold) değerleri ve kategori sıraları hesaplanmış, maddeler arası eşdeğerlik varsayımının ne kadar sağlandığı incelenmiştir. Model veri uyumu, model kestirimlerinden elde edilen -2log likelihood ki-kare değerleri, AIC ve BIC değerleri kullanılarak değerlendirilmiştir. Uygulama verisinde 339 gönüllü katılımcı yer almaktadır ve ölçek için hesaplanan Cronbach α güvenirlik katsayısı 0.92’dir. Analizler ölçeğin tek boyutlu bir yapıya sahip olduğunu göstermektedir. Model-veri uyumu değerlendirmesi sonucunda veriye en iyi uyum sağlayan modelin Aşamalı Tepki Modeli olduğu sonucu elde edilmiştir. Aşamalı Tepki Modeli ile parametre kestirimi yapılan maddeler için gözlenen kategori eşik parametrelerinin hem madde içlerinde beklendiği gibi neredeyse eşit aralıklar ile sıralı olduğu hem de bu aralıkların ölçeği oluşturan tüm maddeler için karşılaştırılabilir olduğu sonucu elde edilmiştir.

Keywords

likert pozitif duygu durumu madde parametre kestirimi çok kategorili madde tepki kuramı

References

  • Andrich, D. (1978). A rating formulation for ordered response categories. Psychometrika, 43(4), 561-573. doi: 10.1007/BF02293814
  • Baker, F. B. (2001). The basis of item response theory. USA: ERIC Clearing house on Assessment and Evaluation.
  • Baker, J. G., Rounds, J. B., & Zevon, M. A. (2000). A comparison of graded response and Rasch partial credit models with subjective well-being. Journal of Educational and Behavioral Statistics, 25(3), 253-270. doi: 10.3102/10769986025003253
  • Brown, T. A. (2015). Confirmatory factor analysis for applied research. New York, NY: Guilford.
  • Caycho-Rodríguez, T., Vilca, L. W., Carbajal-León, C., White, M., Vivanco-Vidal, A., Saroli-Araníbar, D., ..., Moreta-Herrera, R. (2021). Coronavirus anxiety scale: New psychometric evidence for the Spanish version based on CFA and IRT models in a Peruvian sample. Death Studies. doi: 10.1080/07481187.2020.1865480
  • Chalmers, R. P. (2012). mirt: A multidimensional item response theory package for the R environment. Journal of Statistical Software, 48(6), 1-29. doi: 10.18637/jss.v048.i06
  • Chernyshenko, O. S., Stark, S., Chan, K. Y., Drasgow, F., & Williams, B. (2001). Fitting item response theory models to two personality inventories: issues and insights. Multivariate Behavioral Research, 36(4), 523-562. doi: 10.1207/S15327906MBR3604_03
  • Cho, S., Drasgow, F., & Cao, M. (2015). An investigation of emotional intelligence measures using item response theory. Psychological Assessment, 27(4), 1241-1252. doi: 10.1037/pas0000132
  • de Ayala, R. J., Dodd, B. G., & Koch, W. R. (1990, April). A comparison of the partial credit and graded response model in computerized adaptive testing. Paper presented at the AERA Annual Meeting. Boston.
  • DeMars, C. (2010). Item response theory: Understanding statistics measurement. Oxford: Oxford University Press.
  • Demirtaşlı, N., Yalçın, S., & Ayan, C. (2016). The development of irt based attitude scale towards educational measurement course. Eğitimde ve Psikolojide Ölçme ve Değerlendirme Dergisi, 1(7), 133-144. doi: 10.21031/epod.43804
  • Embretson, S. E., & Reise, S. P. (2000). Item response theory for psychologists. New Jersey: LEA publishers.
  • Ferrando, P., Lorenzo, U., & Molina, G. (2001). An item response theory analysis of response stability in personality measurement. Applied Psychological Measurement, 25(1), 3-17. doi: 10.1177/01466216010251001
  • Flannery, W. P., Reise, S. P., & Widaman, K. F. (1995). An item response theory analysis of the general and academic scales of the self-description questionnaire II. Journal of Research in Personality, 29(2), 168-188. doi: 10.1006/jrpe.1995.1010
  • Glass, G. V., & Hopkins, K. D. (1984). Statistical methods in education and psychology. Englewood Cliffs, NJ: Prentice Hall.
  • Gray-Little, B., Williams, V., & Hancock, T. (1997). An item response theory analysis of the Rosenberg self - esteem scale. Personality and Social Psychology Bulletin, 23(5), 443-451. doi: 10.1177/0146167297235001
  • Hambleton, R. K., & Swaminathan, H. (1985). Item response theory: Principles and applications. New York: Springer Science and Business Media.
  • Hattie, J. (1992). Self-concept. Hillsdale, NJ: Erlbaum.
  • Hu, L. T., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6(1), 1-55. doi: 10.1080/10705519909540118
  • Kahraman, N., Akbaş, D., & Sözer, E. (2019). Bilişsel-olmayan öğrenme durum ve süreçlerini ölçme ve değerlendirmede boylamsal yaklaşımlar: Duygu Cetveli Alan Uygulaması örneği. Bolu Abant İzzet Baysal Üniversitesi Eğitim Fakültesi Dergisi, 19(1), 257-269. doi: 10.17240/aibuefd.2019.19.43815-459831
  • Kaptan, S. (1995). Bilimsel araştırma ve istatistik teknikleri (10. Basım). Ankara: Rehber Yayınevi.
  • Koch, W. R. (1983). Likert scaling using the graded response latent trait model. Applied Psychological Measurement, 7(1), 15-32. doi: 10.1177/014662168300700104
  • Köse, İ. A. (2015). Aşamalı tepki modeli ve klasik test kuramı altında elde edilen test ve madde parametrelerinin karşılaştırılması. Abant İzzet Baysal Üniversitesi Eğitim Fakültesi Dergisi, 15(2), 184-197. Retrieved from https://dergipark.org.tr/tr/download/article-file/17439
  • Linacre, J. M. (2002). Optimizing rating scale category effectiveness. Journal of Applied Measurement, 3(1), 85-106. Retrieved from http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.424.2811&rep=rep1&type=pdf
  • Lord, F. M. (1980). Applications of item response theory practical testing problems. Hillsdale, NJ: Erlbaum.
  • Messick, S. (1995). Validity of psychological assessment: Validation of inferences from persons’ responses and performances as scientific inquiry into score meaning. American Psychologist, 50, 741-749. doi: 10.1002/j.2333-8504.1994.tb01618.x
  • Min, S., & Aryadoust, V. (2021). A systematic review of item response theory in language assessment: implications for the dimensionality of language ability. Studies in Educational Evaluation, 68. doi: 10.1016/j.stueduc.2020.100963
  • Muraki, E. (1992). A generalized partial credit model: Application of an EM algorithm. Applied Psychological Measurement, 16(2), 159-176. doi: 10.1177/014662169201600206
  • R Core Team. (2016). R: A language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing. Retrieved from https://www.R-project.org/
  • Roskam, E. E. (1985). Current issues in item response theory. In E. E. Roskam (Ed.), Measurement and personality assessment (pp. 3-19). Amsterdam: North Holland.
  • Rubio, V. J., Aguado, D., Hontangas, P. M., & Hernandez, J. M. (2007). Psychometric properties of an emotional adjustment measure: an application of the Graded Response Model. European Journal of Psychological Assessment, 23(1), 39-46. doi: 10.1027/1015-5759.23.1.39
  • Samejima, F. (1969). Estimation of latent ability using a response pattern of graded scores (Psychometric Monography No. 17). Retrieved from https://www.psychometricsociety.org/sites/main/files/file-attachments/mn17.pdf?1576606975
  • Samejima, F. (1996). Evaluation of mathematical models for ordered polychotomous responses. Behaviormetrika, 23, 17-35. doi: 10.2333/bhmk.23.17
  • Silvia, P. J. (2021). The self-reflection and insight scale: applying item response theory to craft an efficient short form. Current Psychology. doi: 10.1007/s12144-020-01299-7
  • Steiger, J. H., & Lind, J. M. (1980, May). Statistically based tests for the number of common factors. Paper presented in Psychometric Society. Iowa City.
  • Wang, W. C., Wilson, M., & Shih, C. L. (2006). Modeling randomness in judging rating scales with a random-effects rating scale model. Journal of Educational Measurement, 43(4), 335-353. doi: 10.1111/j.1745-3984.2006.00020.x
  • Watson, D., Clark, L. A., & Tellegen, A. (1988). Development and validation of brief measures of positive and negative affect: The PANAS scales. Journal of Personality and Social Psychology, 54(6), 1063-1070. doi: 10.1037/0022-3514.54.6.1063
  • Wu, M., & Adams, R. (2006). Modelling mathematics problem solving item responses using a multidimensional IRT model. Mathematics Education Research Journal, 18(2), 93-113. doi: 10.1007/BF03217438
  • Yaşar, M., & Aybek, E. C. (2019). Üniversite öğrencileri için bir yılmazlık ölçeğinin geliştirilmesi: Madde tepki kuramı temelinde geçerlilik ve güvenilirlik çalışması. İlköğretim Online, 18(4), 1687-1699. Retrieved from https://ilkogretim-online.org/fulltext/218-1597121020.pdf?1618815938
  • Yen, W. M. (1984). Effects of local item dependence on the fit and equating performance of the three-parameter logistic model. Applied Psychological Measurement, 8(2), 125-145. doi: 10.1177/014662168400800201

Investigation of Psychometric Properties of Likert Items with Same Categories Using Polytomous Item Response Theory Models

Year 2021, Volume: 12 Issue: 2, 129 - 146, 30.06.2021

Esra Sözer , Nilüfer Kahraman

https://doi.org/10.21031/epod.819927
Cited By: 2

Abstract

The purpose of this study was to investigate within- and between-threshold parameter invariance for items of a fourteen-item Positive Affect Scale developed to assess positive moods (like happy, peaceful, etc.) of university students. To test whether the estimated threshold parameters were as expected (1 to 5, with increments of 1) across all the 14 items, Graded Response, Partial Credit, and Rating Scale Models were fit the response data collected from 326 students. A comparison of the model fit statistics, such as the negative 2log likelihood and chi-square values, revealed that the Graded Response Model had the best fit and that the thresholds estimates for all the items in the Positive Affective Scale were reasonably close to the expected 1 to 5 values with increments of 1. The study illustrates how polytomous response models can be used to test the psychometric quality of items with ordinal rating scales.

Keywords

likert positive affect item parameter estimation polytomous ıtem response theory

References

  • Andrich, D. (1978). A rating formulation for ordered response categories. Psychometrika, 43(4), 561-573. doi: 10.1007/BF02293814
  • Baker, F. B. (2001). The basis of item response theory. USA: ERIC Clearing house on Assessment and Evaluation.
  • Baker, J. G., Rounds, J. B., & Zevon, M. A. (2000). A comparison of graded response and Rasch partial credit models with subjective well-being. Journal of Educational and Behavioral Statistics, 25(3), 253-270. doi: 10.3102/10769986025003253
  • Brown, T. A. (2015). Confirmatory factor analysis for applied research. New York, NY: Guilford.
  • Caycho-Rodríguez, T., Vilca, L. W., Carbajal-León, C., White, M., Vivanco-Vidal, A., Saroli-Araníbar, D., ..., Moreta-Herrera, R. (2021). Coronavirus anxiety scale: New psychometric evidence for the Spanish version based on CFA and IRT models in a Peruvian sample. Death Studies. doi: 10.1080/07481187.2020.1865480
  • Chalmers, R. P. (2012). mirt: A multidimensional item response theory package for the R environment. Journal of Statistical Software, 48(6), 1-29. doi: 10.18637/jss.v048.i06
  • Chernyshenko, O. S., Stark, S., Chan, K. Y., Drasgow, F., & Williams, B. (2001). Fitting item response theory models to two personality inventories: issues and insights. Multivariate Behavioral Research, 36(4), 523-562. doi: 10.1207/S15327906MBR3604_03
  • Cho, S., Drasgow, F., & Cao, M. (2015). An investigation of emotional intelligence measures using item response theory. Psychological Assessment, 27(4), 1241-1252. doi: 10.1037/pas0000132
  • de Ayala, R. J., Dodd, B. G., & Koch, W. R. (1990, April). A comparison of the partial credit and graded response model in computerized adaptive testing. Paper presented at the AERA Annual Meeting. Boston.
  • DeMars, C. (2010). Item response theory: Understanding statistics measurement. Oxford: Oxford University Press.
  • Demirtaşlı, N., Yalçın, S., & Ayan, C. (2016). The development of irt based attitude scale towards educational measurement course. Eğitimde ve Psikolojide Ölçme ve Değerlendirme Dergisi, 1(7), 133-144. doi: 10.21031/epod.43804
  • Embretson, S. E., & Reise, S. P. (2000). Item response theory for psychologists. New Jersey: LEA publishers.
  • Ferrando, P., Lorenzo, U., & Molina, G. (2001). An item response theory analysis of response stability in personality measurement. Applied Psychological Measurement, 25(1), 3-17. doi: 10.1177/01466216010251001
  • Flannery, W. P., Reise, S. P., & Widaman, K. F. (1995). An item response theory analysis of the general and academic scales of the self-description questionnaire II. Journal of Research in Personality, 29(2), 168-188. doi: 10.1006/jrpe.1995.1010
  • Glass, G. V., & Hopkins, K. D. (1984). Statistical methods in education and psychology. Englewood Cliffs, NJ: Prentice Hall.
  • Gray-Little, B., Williams, V., & Hancock, T. (1997). An item response theory analysis of the Rosenberg self - esteem scale. Personality and Social Psychology Bulletin, 23(5), 443-451. doi: 10.1177/0146167297235001
  • Hambleton, R. K., & Swaminathan, H. (1985). Item response theory: Principles and applications. New York: Springer Science and Business Media.
  • Hattie, J. (1992). Self-concept. Hillsdale, NJ: Erlbaum.
  • Hu, L. T., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6(1), 1-55. doi: 10.1080/10705519909540118
  • Kahraman, N., Akbaş, D., & Sözer, E. (2019). Bilişsel-olmayan öğrenme durum ve süreçlerini ölçme ve değerlendirmede boylamsal yaklaşımlar: Duygu Cetveli Alan Uygulaması örneği. Bolu Abant İzzet Baysal Üniversitesi Eğitim Fakültesi Dergisi, 19(1), 257-269. doi: 10.17240/aibuefd.2019.19.43815-459831
  • Kaptan, S. (1995). Bilimsel araştırma ve istatistik teknikleri (10. Basım). Ankara: Rehber Yayınevi.
  • Koch, W. R. (1983). Likert scaling using the graded response latent trait model. Applied Psychological Measurement, 7(1), 15-32. doi: 10.1177/014662168300700104
  • Köse, İ. A. (2015). Aşamalı tepki modeli ve klasik test kuramı altında elde edilen test ve madde parametrelerinin karşılaştırılması. Abant İzzet Baysal Üniversitesi Eğitim Fakültesi Dergisi, 15(2), 184-197. Retrieved from https://dergipark.org.tr/tr/download/article-file/17439
  • Linacre, J. M. (2002). Optimizing rating scale category effectiveness. Journal of Applied Measurement, 3(1), 85-106. Retrieved from http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.424.2811&rep=rep1&type=pdf
  • Lord, F. M. (1980). Applications of item response theory practical testing problems. Hillsdale, NJ: Erlbaum.
  • Messick, S. (1995). Validity of psychological assessment: Validation of inferences from persons’ responses and performances as scientific inquiry into score meaning. American Psychologist, 50, 741-749. doi: 10.1002/j.2333-8504.1994.tb01618.x
  • Min, S., & Aryadoust, V. (2021). A systematic review of item response theory in language assessment: implications for the dimensionality of language ability. Studies in Educational Evaluation, 68. doi: 10.1016/j.stueduc.2020.100963
  • Muraki, E. (1992). A generalized partial credit model: Application of an EM algorithm. Applied Psychological Measurement, 16(2), 159-176. doi: 10.1177/014662169201600206
  • R Core Team. (2016). R: A language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing. Retrieved from https://www.R-project.org/
  • Roskam, E. E. (1985). Current issues in item response theory. In E. E. Roskam (Ed.), Measurement and personality assessment (pp. 3-19). Amsterdam: North Holland.
  • Rubio, V. J., Aguado, D., Hontangas, P. M., & Hernandez, J. M. (2007). Psychometric properties of an emotional adjustment measure: an application of the Graded Response Model. European Journal of Psychological Assessment, 23(1), 39-46. doi: 10.1027/1015-5759.23.1.39
  • Samejima, F. (1969). Estimation of latent ability using a response pattern of graded scores (Psychometric Monography No. 17). Retrieved from https://www.psychometricsociety.org/sites/main/files/file-attachments/mn17.pdf?1576606975
  • Samejima, F. (1996). Evaluation of mathematical models for ordered polychotomous responses. Behaviormetrika, 23, 17-35. doi: 10.2333/bhmk.23.17
  • Silvia, P. J. (2021). The self-reflection and insight scale: applying item response theory to craft an efficient short form. Current Psychology. doi: 10.1007/s12144-020-01299-7
  • Steiger, J. H., & Lind, J. M. (1980, May). Statistically based tests for the number of common factors. Paper presented in Psychometric Society. Iowa City.
  • Wang, W. C., Wilson, M., & Shih, C. L. (2006). Modeling randomness in judging rating scales with a random-effects rating scale model. Journal of Educational Measurement, 43(4), 335-353. doi: 10.1111/j.1745-3984.2006.00020.x
  • Watson, D., Clark, L. A., & Tellegen, A. (1988). Development and validation of brief measures of positive and negative affect: The PANAS scales. Journal of Personality and Social Psychology, 54(6), 1063-1070. doi: 10.1037/0022-3514.54.6.1063
  • Wu, M., & Adams, R. (2006). Modelling mathematics problem solving item responses using a multidimensional IRT model. Mathematics Education Research Journal, 18(2), 93-113. doi: 10.1007/BF03217438
  • Yaşar, M., & Aybek, E. C. (2019). Üniversite öğrencileri için bir yılmazlık ölçeğinin geliştirilmesi: Madde tepki kuramı temelinde geçerlilik ve güvenilirlik çalışması. İlköğretim Online, 18(4), 1687-1699. Retrieved from https://ilkogretim-online.org/fulltext/218-1597121020.pdf?1618815938
  • Yen, W. M. (1984). Effects of local item dependence on the fit and equating performance of the three-parameter logistic model. Applied Psychological Measurement, 8(2), 125-145. doi: 10.1177/014662168400800201
There are 40 citations in total.

Details

Primary Language English
Journal Section Articles
Authors

Esra Sözer 0000-0002-4672-5264

Nilüfer Kahraman 0000-0003-2523-0155

Publication Date June 30, 2021
Acceptance Date April 16, 2021
Published in Issue Year 2021 Volume: 12 Issue: 2

Cite

APA Sözer, E., & Kahraman, N. (2021). Investigation of Psychometric Properties of Likert Items with Same Categories Using Polytomous Item Response Theory Models. Journal of Measurement and Evaluation in Education and Psychology, 12(2), 129-146. https://doi.org/10.21031/epod.819927

Cited By

How can organizations measure the integration of environmental, social, and governance (ESG) criteria? Validation of an instrument using item response theory to capture workers' perception
Business Strategy and the Environment
https://doi.org/10.1002/bse.3675
The Effect of Option Differences on Psychometric Properties of Items in Likert-Type Scales
Sakarya University Journal of Education
https://doi.org/10.19126/suje.1253876
 Download Cover Image