Adaptation of Teachers' Perceptions of Grading Practices Scale to Turkish and Examination of Measurement Invariance

Year 2022, Volume: 9 Issue: Special Issue, 300 - 316, 29.11.2022

Yeşim Özer Özkan , Meltem Acar Güvendir , Emre Guvendir

https://doi.org/10.21449/ijate.1115317

Abstract

Grading is a complicated decision-making process that needs teachers to make value judgements on the learning, accomplishment, and development of their students. When assigning grades, teachers tend to examine a variety of criteria, including students' efforts, work habits, and accomplishments. Grades are a reflection of the value judgements made about students based on the evaluation of their academic performance. Consequently, describing how to reach a value judgment utilizing general measures will contribute to a better understanding of the difficulties encountered throughout the grading process. The purpose of this research is to adapt the Teacher Perceptions of Grading Practices Scale into Turkish and to examine the measurement invariance. This scale, which examines teachers' perceptions of grading methods, has six components: importance, usefulness, student effort, student ability, teacher's grading patterns, and perceived self-efficacy of the grading process. Before adapting the scale, permission was first acquired from the researcher who developed it. To ensure linguistic comparability, bilingual translators were recruited in the second phase. The semantic, experiential, conceptual, and idiomatic equivalence between the two variants of the scale were evaluated. The original and adapted scales were administered to a group of English teachers twice at a predetermined interval, and the consistency between the two applications was analyzed due to the fact that the language employed in the original test was a widely spoken group. Confirmatory Factor Analysis (CFA) was used to examine the factor structure of the original scale. Cronbach's alpha and Omega coefficients were calculated for the reliability of the data obtained from the scale. Finally, the measurement invariance of the scale according to gender was examined by using Multiple Group Confirmatory Factor Analysis (MGCFA), and it was determined that the measurement model fulfilled the criteria of complete gender-group invariance.

Keywords

Grading Practices, Scale Adaptation, Teachers’ Perceptions, Student Achievement, Measurement Invariance, Not verme uygulamaları, Ölçek uyarlama, Öğretmen algıları, Öğrenci başarısı, Ölçme değişmezliği

Not verme uygulamalarına ilişkin öğretmen algıları ölçeğinin Türkçe’ye uyarlanması ve ölçme değişmezliğinin incelenmesi

Abstract

Bu araştırmanın amacı, not verme uygulamalarına ilişkin öğretmen algıları ölçeğinin Türkçe'ye uyarlanması ve ölçme değişmezliğinin incelenmesidir. Öğretmenlerin not verme yöntemlerine ilişkin algılarını inceleyen bu ölçeğin, önem, yarar, öğrenci çabası, öğrenci yeteneği, öğretmenin not verme alışkanlıkları ve not verme sürecinin algılanan öz yeterliği olmak üzere altı faktörü vardır. Ölçeğin uyarlanmasından önce öncelikle ölçeği geliştiren araştırmacıdan izin alınmıştır. Dilsel eşdeğerliliği sağlamak için ikinci aşamada iki dilli çevirmenlerden yardım alınmıştır. Ölçeğin iki formu arasındaki anlamsal, deneyimsel, kavramsal ve deyimsel eşdeğerlik değerlendirilmiştir. Orijinal ve uyarlanmış ölçekler, bir grup İngilizce öğretmenine önceden belirlenmiş aralıklarla iki kez uygulanmış ve orijinal testte kullanılan dilin yaygın olarak konuşulan bir grup olması nedeniyle iki uygulama arasındaki tutarlılık analiz edilmiştir. Orijinal ölçeğin faktör yapısını incelemek için Doğrulayıcı Faktör Analizi (DFA) kullanılmıştır. Ölçekten elde edilen verilerin güvenirliği için Cronbach alfa ve Omega katsayıları hesaplanmıştır. Son olarak ölçeğin cinsiyete göre ölçme değişmezliği Çoklu Grup doğrulayıcı Faktör Analizi (ÇGDFA) ile incelenmiş ve ölçme modelinin cinsiyet-grup değişmezliği ölçütlerini tam olarak karşıladığı belirlenmiştir.

Keywords

Not verme uygulamaları, Ölçek uyarlama, Öğretmen algıları, Öğrenci başarısı, Ölçme değişmezliği

References

• Acar Güvendir, M., & Özer Özkan, Y. (2016). Practicality of measurement and evaluation course in education. Paper presented at V. Congress of Measurement and Evaluation in Education and Psychology, 1-3 September 2016, Antalya.
• Adıyaman, Y. (2005). İlköğretim 4., 6. ve 8. sınıflarında Türkçe dersine giren öğretmenlerin ölçme değerlendirme düzeyleri [The measurement and evaluation levels of teachers teach Turkish course in 4th, 6th and 8th classes in primary school] [Unpublished Master Thesis, Kocatepe University].
• Andersson, A. (1998). The dimensionality of the leaving certificate. Scandinavian Journal of Educational Research, 42(1), 25-40. https://doi.org/10.1080/0031383980420102
• Anderson, L.W. (2018). A Critique of grading: Policies, practices, and technical matters. Education Policy Analysis Archives, 26(49), 1 31. http://dx.doi.org/10.14507/epaa.26.3814
• Bailey, J.M., & McTighe, J. (1996). Reporting achievement at the secondary level: What and how. In T.R. Guskey (Ed.), Communicating student learning: 1996 Yearbook of the ASCD (pp. 119–140). ASCD.
• Bollen, K.A. (1989). Structural equations with latent variables, Wiley.
• Brewer, T.J., & deMarrais, K. (2015). Teach for America counter-narratives: Alumni speak up and speak out. Peter Lang Incorporated, International Academic Publishers. https://doi.org/10.3726/978-1-4539-1556-1
• Biggs, J. (2001) Assessment of student learning: Where did we go wrong? Assessment Update, 13(6), 6-11.
• Brookhart, S.M., Guskey, T.R., Bowers, A.J., McMillan, J.H., Smith, J.K., Smith, L.F., Stevens, M.T., & Welsh, M.J. (2016). A century of grading research: Meaning and value in the most common educational measure. Review of Educational Research, 86(4), 803-848. https://doi.org/10.3102/0034654316672069
• Büyüköztürk, Ş. (2002). Faktör analizi: Temel kavramlar ve ölçek geliştirmede kullanımı. [Factor analysis: basic concepts and using to development scale]. Educational Administration in Theory & Practice, 32(32), 470 483. https://dergipark.org.tr/en/pub/kuey/issue/10365/126871
• Campbell, A.L. (1921). Keeping the score. School Review, 29(7), 510-519.
• Chen, F.F. (2007). Sensitivity of goodness of fit indexes to lack of measurement invariance. Structural Equation Modeling: A Multidisciplinary Journal, 14(3), 464 504. https://doi.org/10.1080/10705510701301834
• Cheung, G.W., & Rensvold, R.B. (2002). Evaluating goodness-of-fit indexes for testing measurement invariance. Structural Equation Modeling, 9(2), 233 255. https://doi.org/10.1207/S15328007SEM0902_5
• Crocker, L., & Algina, J. (1986). Introduction to classical and modern test theory. Holt, Rinehart and Winston.
• Çakan, M. (2004). Öğretmenlerin ölçme-değerlendirme uygulamaları ve yeterlik düzeyleri: ilk ve ortaöğretim. [Comparison of elementary and secondary school teachers in terms of their assessment practices and perceptions toward their qualification levels]. Ankara University, Journal of Faculty of Educational Sciences, 37(2), 99 114. https://doi.org/10.1501/Egifak_0000000101
• Deniz, Z. (2007). Psikolojik ölçme aracı uyarlama. [The Adaptation of psychological scales]. Ankara Üniversitesi Eğitim Bilimleri Fakültesi Dergisi, 40(1), 1-16.
• Eraut, M. (2004) A wider perspective on assessment, Medical Education, 38(1), 803-804. https://doi.org/10.1111/j.1365-2929.2004.01930.x
• Erdal, H. (2007). 2005 ilköğretim matematik programı ölçme değerlendirme kısmının incelenmesi (Afyonkarahisar ili örneği). [The investigation of measurement & evaluation parts in the new elementary school mathematics curriculum (case of Afyonkarahisar)] [Unpublished Master Thesis, Kocatepe University].
• Erdemir, Z.A. (2007). İlköğretim ikinci kademe öğretmenlerinin ölçme değerlendirme tekniklerini etkin kullanabilme yeterliklerinin araştırılması (Kahramanmaraş örneği). [Searching for the secondary education teachers' competence of being able to use the techniques of measurement and evaluation (example of Kahramanmaraş)] [Unpublished master thesis, Kahramanmaraş Sütçü İmam University].
• Frisbie, D., Diamond, N.A., & Ory, J.C. (1979). Assigning course grades, Urbana, IL: University of Illinois Office of Instructional Resources. (ERIC Document Reproduction Service No. ED285496)
• Gardner, W., Demirtaş, A., & Doğanay, A. (1997). Sosyal bilimler öğretimi. [Social sciences teaching] YÖK-Dünya Bankası. MEGEP.
• Green, S.K., Johnson, R.L., Kim, D., & Pope, N.K. (2006). Ethics in classroom assessment practices: Issues and attitudes. Teacher and Teacher Education, 23(7), 999-1011. https://doi.org/10.1016/j.tate.2006.04.042
• Guskey, T.R. (2011). Five obstacles to grading reform. Educational Leadership, 69(3), 16-21. https://uknowledge.uky.edu/edp_facpub/6
• Guskey, T.R. (2015). On your mark: Challenging the conventions of grading and reporting. Bloomington, IN: Solution Tree Press.
• Guskey, T.R., & Bailey, J.M. (2001). Developing grading and reporting systems for student learning, Corwin Press.
• Guskey, T.R., & Link, L.J. (2019). Exploring the factors teachers consider in determining students’ grades. Assessment in Education: Principles, Policy & Practice, 26(3), 303-320. https://doi.org/10.1080/0969594X.2018.1555515
• Hambleton, R.K. (1996). Guidelines for adapting educational and psychological test. National Center for Education Statistics (ED).
• Hambleton, R.K. & Patsula, L. (1999). Increasing the validity of adapted tests: Myths to be avoided and guidelines for improving test adaptation practices. Journal of Applied Testing Technology, 1(1), 1-30.
• Hambleton, R.K., Merenda, P., & Spielberger, C. (Eds.). (2005). Adapting educational and psychological tests for cross-cultural assessment. Lawrence S. Erlbaum Publishers.
• Harlen, W. (2005) Teachers’ summative practices and assessment for learning- Tensions and synergies, The Curriculum Journal, 16(2), 207 223. https://doi.org/10.1080/09585170500136093
• Jöreskog, K.G., & Sörbom, D. (1993). LISREL 8: Structural equation modeling with the SIMPLIS command language. Scientific Software International.
• Jöreskog, K., & Sörbom, D. (2004). LISREL [Computer Software]. Lincolnwood, IL: Scientific Software, Inc. https://doi.org/10.1002/0471667196.ess1481
• Kan, A. (2005). Yazılı yoklamaların puanlanmasında puanlama cetveli ve yanıt anahtarı kullanımının (aynı) puanlayıcı güvenirliğine etkisi [The effect of using grading scale and answer key to grader’s reliability]. Eurasian Journal of Educational Research, 20(1), 166-177.
• Koç, N. (1981). Liselerde öğrencilerin akademik başarılarının değerlendirilmesi uygulamalarının etkinliğine ilişkin bir araştırma [A research on the effectiveness of the applications of evaluating the academic achievement of students in high schools] [Unpublished Master Thesis, Ankara University].
• Küçükahmet, L. (2005). Öğretimde planlama ve değerlendirme [Planning and evaluation in instruction]. Nobel Yayınları.
• Leech, N.L., Barrett, K.C., & Morgan, G.A. (2005) SPSS for intermediate statistics,use and interpretation (2nd Edition). Lawrence Erlbaum.
• Liu, X. (2004). The initial validation of teacher’s perception of grading practices. Paper presented at the 2004 Northeastern Educational Research Association annual meeting, Measuring Teachers’ Perceptions 14.
• Liu, X. (2008). Assessing measurement ınvariance of the teachers’ perceptions of grading practices scale across cultures. NERA Conference Proceedings 2008. 3. https://opencommons.uconn.edu/nera_2008/3.
• Liu, X., O’Connell, A.A., & McCoach, D.B. (2006). The initial validation of teachers’ perceptions of grading practices. Paper presented at the 2006 Annual Meeting of American Educational Research Association (AERA).
• Masters, G. (1987). New views of student learning: Implications for educational measurement. Research working paper 87.11. University of Melbourne: Centre for the Study of Higher Education.
• McMillan, J.H., Myran, S., & Workman, D. (2002). Elementary teachers’ classroom assessment and grading practices. The Journal of Educational Research, 95(4), 203–213. https://doi.org/10.1080/00220670209596593
• Ministry of National Education's [MoNE] (2005). EARGED ilköğretim 1.-5. sınıf pilot uygulama sonuçlarının değerlendirilmesi [EARGED primary education 1.-5. Evaluation of class pilot application results. MoNE Publications.
• Messick, S. (1984). The psychology of educational measurement. Journal of Educational Measurement, 21, 215-237. https://doi.org/10.1111/j.1745-3984.1984.tb01030.x
• Meyer, M. (1908). The grading of students. Science, 28(712), 243 250. https://doi.org/10.1126/science.28.712.243
• Öztürk, B. (1988). Lise sosyal bilimler dersleri öğretmenlerinin başarı testi hazırlamadaki yeterliliklerine ilişkin bir araştırma [A research on the competencies of high school social science teachers in preparing achievement tests] [Unpublished Master Thesis, Gazi University].
• Redding, C., & Smith, T.M. (2016). Easy in, easy out: Are alternatively certified teachers turning over at increased rates? American Educational Research Journal, 53(4), 1086-1125. https://doi.org/10.3102/0002831216653206
• Salend, S.J., & Duhaney, L.M.G. (2002). Grading students in inclusive settings. Teaching Exceptional Children, 34(3), 8-15. https://doi.org/10.1177/004005990203400301
• Schermelleh-Engel, K., Moosbrugger, H., & Müller, H. (2003). Evaluating the fit of structural equation models: Tests of significance and descriptive goodness-of-fit measures. Methods of Psychological Research Online, 8(2), 23-74. http://www.mpr-online.de
• Semerci, Ç. (1993). Fırat Üniversitesinde öğrenci başarısının ölçülmesinde kullanılan yöntemler ile ölçme ve değerlendirmeye ilişkin görüşler [Opinions on the methods used in measuring student achievement at Fırat University and on measurement and evaluation] [Unpublished Master Thesis, Fırat University].
• Serban, A.M. (2004) Assesment of student learning outcomes at the institutional level. New Directions For Comminity Colleges, 2004(126), 17-27. https://doi.org/10.1002/cc.151
• Sümer, N. (2000). Yapısal eşitlik modelleri: Temel kavramlar ve örnek uygulamalar [Structural equation modeling: Basic concepts and applications]. Türk Psikoloji Yazıları, 3(6), 49–74.
• Tabachnick, B.G. & Fidell, L.S. (2019). Using multivariate statistics (7th edition). Pearson.
• The jamovi project (2021). Jamovi. (Version 1.8) [Computer Software]. Retrieved from https://www.jamovi.org
• Thompson, B. (2004). Exploratory and confirmatory factor analysis: Understanding concepts and applications. American Psychological Association. https://doi.org/10.1037/10694-000
• Topal, T. (2020). Öğretmen adaylarının bakış açısından sınıf öğretmenlerinin öğretim sürecinde gösterdikleri dönüt ve düzeltme davranışları [Feedback and correction behavior of the classroom teachers during the teaching process from the perspective of the teacher candidates]. OPUS International Journal of Society Researches, Eğitim ve Toplum Özel Sayısı, 16, 6150-6166. https://doi.org/10.26466/opus.825157
• Vandenberg, R.J., & Lance, C.E. (2000). A review and synthesis of the measurement invariance literature: Suggestions, practices, and recommendations for organizational research. Organizational Research Methods, 3(1), 4-70. https://doi.org/10.1177/109442810031002