Comparison of Classification Accuracy and Parameters of DINA, DINO, HO-DINA and HO-DINO Models in the Framework of Cognitive Diagnosis in Health Education

Year 2025, , 48 - 66, 28.01.2025

Sena Gencan , Şeref Tan

https://doi.org/10.24106/kefdergi.1628232

Abstract

Purpose: This study aims to compare the parameters of DINA, DINO, HO-DINA and HO-DINO models according to different sample sizes (500, 2000, 5000) and different item numbers (60, 120) based on the Q matrices created for different attributes in health education based on simulation data.
Design/Methodology/Approach: In the simulation data, 50 replications were performed for each condition. In the study, two different Q-Matrixes were determined based on the learning domain determined by considering the 2018 TUS Spring Assessment Report and the taxonomy included in the Clinical assessment framework determined in Miller's 1990 study as the attributes dimension in the Q-Matrix in which matching of attribute and item is carried out. In the study, RMSEA, g and s parameters and classification accuracies were compared and under which conditions DINA, DINO, HO-DINA and HO-DINO models gave similar or different results were investigated.
Findings: According to the research findings, the Q-Matrix, in which Fields levels were used as the attribute dimension, was the matrix that gave the best parameter results in all models. In addition, it has been determined that the models that give the best RMSEA, g and s parameters and classification accuracies are DINO and HO-DINO models in the analysis.
Highlights: Based on the findings, when analyzing the results for the Basic Medical Sciences and Clinical Medical Sciences tests, it is evident that the Q matrix determined by Fields provides a better fit to the data, and moreover, it is advantageous for the Q matrix determined by Fields to be used for the TUS exam.

Keywords

Cognitive Diagnostic Models, DINA model, DINO model, Q matrix, Health Education

References

• Aryadoust, V. (2018). A cognitive diagnostic assessment study of the listening test of the Singapore–Cambridge General Certificate of Education O-Level: Application of DINA, DINO, G-DINA, HO-DINA, and RRUM. International Journal of Listening, 35(1), 29-52.
• Aydın, S., & Demir, M. (2006). Sağlıkta performans yönetimi: Performansa dayalı ek ödeme sistemi. Sağlık Bakanlığı.
• Bakan, İ., Erşahan, B., Kefe, İ. & Bayat, M. (2011). Kamu ve özel hastanelerde tedavi gören hastaların sağlıkta hizmet kalitesine ilişkin algılamaları. Kahramanmaraş Sütçü İmam Üniversitesi İktisadi ve İdari Bilimler Fakültesi Dergisi, 1(2), 1-26.
• Bakanlığı, T. S. (2001). Sağlık Hizmetlerinin Yürütülmesi Hakkında Yönerge.
• Bakanlığı, T. S., & Müdürlüğü, T. S. H. G. (2011). Sağlığın teşviki ve geliştirilmesi sözlüğü. Bakanlık yayın, 814(1), 23.
• Chiu, C. Y. (2013). Statistical refinement of the Q-matrix in cognitive diagnosis. Applied Psychological Measurement, 37(8), 598-618.
• Collares, C. F. (2022). Cognitive diagnostic modelling in healthcare professions education: an eye-opener. Advances in Health Sciences Education, 27(2), 427-440.
• DeCarlo, L. T. (2011). On the analysis of fraction subtraction data: The DINA model, classification, latent class sizes, and the Q-matrix. Applied Psychological Measurement, 35(1), 8-26.
• De La Torre, J. (2009). DINA model and parameter estimation: A didactic. Journal of educational and behavioral statistics, 34(1), 115-130.
• De La Torre, J., & Douglas, J. A. (2004). Higher-order latent trait models for cognitive diagnosis. Psychometrika, 69(3), 333-353.
• De La Torre, J., Hong, Y., & Deng, W. (2010). Factors affecting the item parameter estimation and classification accuracy of the DINA model. Journal of Educational Measurement, 47(2), 227-249.
• De La Torre, J., & Minchen, N. (2014). Cognitively diagnostic assessments and the cognitive diagnosis model framework. Psicología Educativa, 20(2), 89-97.
• Doğan, N. Ö., & Gencan, S. (2014). VZA/AHP bütünleşik yöntemi ile performans ölçümü: Ankara’daki kamu hastaneleri üzerine bir uygulama. Gazi Üniversitesi İktisadi ve İdari Bilimler Fakültesi Dergisi, 16(2), 88-112.
• Grégoire, J. (1997). Diagnostic assessment of learning disabilities: From assessment of performance to assessment of competence. European Journal of Psychological Assessment, 13(1), 10-20.
• Haertel, E. H. (1989). Using restricted latent class models to map the skill structure of achievement items. Journal of Educational Measurement, 26(4), 301-321.
• Henson, R. A., Templin, J. L., & Willse, J. T. (2009). Defining a family of cognitive diagnosis models using log-linear models with latent variables. Psychometrika, 74(2), 191-210.
• Hu, J., Miller, M. D., Huggins-Manley, A. C., & Chen, Y. H. (2016). Evaluation of model fit in cognitive diagnosis models. International Journal of Testing, 16(2), 119-141.
• Jang, E. E., & Wagner, M. (2014). Diagnostic feedback in the classroom. The companion to language assessment, 2, 157-175.
• Junker, B. W., & Sijtsma, K. (2001). Cognitive assessment models with few assumptions, and connections with nonparametric item response theory. Applied Psychological Measurement, 25(3), 258-272.
• Kalkan, Ö. K., & Başokçu, T. O. (2019). The Effect of the Item–Attribute Relation on the DINA Model Estimations in the Presence of Missing Data. Pamukkale Üniversitesi Eğitim Fakültesi Dergisi, 46(46), 290-306.
• Ma, W., & Guo, W. (2019). Cognitive diagnosis models for multiple strategies. British Journal of Mathematical and Statistical Psychology, 72(2), 370-392.
• Ma, C., Ouyang, J., & Xu, G. (2022). Learning latent and hierarchical structures in cognitive diagnosis models. Psychometrika, 1-33.
• Miller, G. E. (1990). The assessment of clinical skills/competence/performance. Academic medicine, 65(9), S63-7.
• Rupp, A. A., & Templin, J. (2007). The effects of Q-matrix misspecification on parameter estimates and classification accuracy in the DINA model. Educational and Psychological Measurement, 68(1), 78-96.
• Rupp, A. A., & Templin, J. L. (2008). Unique characteristics of diagnostic classification models: A comprehensive review of the current state-of-the-art. Measurement, 6(4), 219-262.
• Rupp, A. A., Templin, J., & Henson, R. A. (2010). Diagnostic measurement: Theory, methods and applications. Guilford Press.
• Su, Y. L. (2013). Cognitive diagnostic analysis using hierarchically structured skills. The University of Iowa.
• Sünbül, S. Ö., & Adnan, K. A. N. (2013). Bilişsel Tanı Modellerinde Parametre Kestirimini ve Sınıflama Tutarlılığını Etkileyen Faktörlerin İncelenmesi Factors Affecting the Item Parameter Estimation and Classification Accuracy of the Cognitive Diagnostic Models.
• Tatsuoka, K. K. (1995). Architecture of knowledge structures and cognitive diagnosis: A statistical pattern recognition and classification approach. Cognitively diagnostic assessment, 327-359.
• Tatsuoka, K. K., & Tatsuoka, M. M. (1997). Computerized cognitive diagnostic adaptive testing: Effect on remedial instruction as empirical validation. Journal of Educational Measurement, 34(1), 3-20.
• Templin, J., & Bradshaw, L. (2014). Hierarchical diagnostic classification models: A family of models for estimating and testing attribute hierarchies. Psychometrika, 79(2), 317-339.
• Templin, J. L., & Henson, R. A. (2006). Measurement of psychological disorders using cognitive diagnosis models. Psychological methods, 11(3), 287.
• WORLD HEALTH ORGANIZATION, et al. World Conference on Medical Education: Edinburgh, 8-12 August 1988. In: Executive Board Session, 83. World Health Organization, 1988.