Investigation of the Items Prepared by Teachers Who Received Training on Item Preparation Measuring Higher-Order Thinking Skills

Year 2024, Volume: 11 Issue: 6, 189 - 215, 01.11.2024

Ahmet Bolat , Sevilay Karamustafaoğlu

https://doi.org/10.17275/per.24.86.11.6

Abstract

This study was conducted to determine how the in-service training program for science teachers affects teachers' competencies in preparing items measuring higher order learning levels. The study was conducted with the exploratory special case design, which is one of the special case study models. The study group consisted of 8 science teachers who participated in the training program implemented within the scope of the study. The training program is an 8-week question preparation training that measures higher-order thinking skills. The training was carried out theoretically and practically by science education, assessment and evaluation and curriculum experts. During the study, the teachers prepared items to measure higher order thinking skills related to the Earth and Universe learning area in the science curriculum for six months. A 21-item checklist developed by the researcher was used as a data collection tool in the study. The data obtained in the scope of the program were analyzed descriptively. The analyses were reported as frequency and percentage, and the results were visualised with graphs. At the end of the program, it was determined that 43% of the items prepared by the teachers measured higher-order thinking skills. This finding suggests that the training program positively affected the teachers' competence in writing items that measure higher-order thinking skills. At the end of the study, a fair number of suggestions were made, the most important being the widespread implementation of the training program.

Keywords

Item preparation, Higher-order thinking skills, Science education, In-service training

Ethical Statement

Amasya University Social Sciences Ethics Committee has an ethics committee permit dated 15.04.2021 and numbered E-30640013-108.01-13400.

Thanks

Thanks. Enjoy your work.

References

• Abdullah, A. H., Mokhtar, M., Abd Halim, N. D., Ali, D. F., Tahir, L. M., & Kohar, U. H. A. (2016). Mathematics teachers’ level of knowledge and practice on the implementation of higher-order thinking skills (HOTS). Eurasia Journal of Mathematics, Science and Technology Education, 13(1), 3-17. https://doi.org/10.12973/eurasia.2017.00601a
• Abosalem, Y. (2016). Assessment techniques and students’ higher-order thinking skills. International Journal of Secondary Education, 4(1), 1-11. https://doi.org/ 10.11648/j.ijsedu.20160401.11
• Afifah, I. R. N., & Retnawati, H. (2019, October). Is it difficult to teach higher order thinking skills? In Journal of Physics: Conference Series (Vol. 1320, No. 1, p. 012098). IOP Publishing.
• Akpınar, E., & Ergin, Ö. (2006). Fen bilgisi öğretmenlerinin yazılı sınav sorularının değerlendirilmesiEvaluation of science teachers' written exam items. Milli Eğitim DergisiJournal of National Education, 35(172), 225-231.
• Anderson, L.W., Krathwohl, D.R., Airasian, P.W., Cruikshank, K.A., Mayer, R.E., Pintrich, P.R., Raths, J. and Wittrock. (2018). Bloom’un eğitim hedefleri ile ilgili sınıflamasının güncelleştirilmiş biçimiAn updated version of Bloom's classification of educational objectives (Third Edition). Durmuş Ali Özçelik (Trans.). Ankara: Pegem Academy, XX-115.
• Antonio, R. P., & Prudente M. S. (2024). Effects of inquiry-based approaches on students’ higher-order thinking skills in science: A meta-analysis. International Journal of Education in Mathematics, Science, and Technology (IJEMST), 12(1), 251-281. https://doi.org/10.46328/ijemst.3216
• Ar, M. E. (2019). Fen Bilimleri Öğretmenlerine Yönelik Geliştirilen Nitelikli Yaşam Temelli Açık Uçlu Soru Hazırlama Kursunun Uygulanması ve DeğerlendirilmesiImplementation and Evaluation of Qualified Life-Based Open-Ended Item Preparation Course Developed for Science Teachers. Yayınlanmamış Yüksek Lisans TeziUnpublished Master Thesis, Uludağ Üniversitesi Eğitim Bilimleri EnstitüsüUludağ University Institute of Educational Sciences, Bursa.
• Arı, A., & İnci, T. (2015). Sekizinci sınıf fen ve teknoloji dersine ilişkin ortak sınav sorularının değerlendirilmesiEvaluation of the common exam items related to the eighth grade science and technology course. Uşak Üniversitesi Sosyal Bilimler DergisiUsak University Journal of Social Sciences, 8(4), 17-50. https://doi.org/10.12780/uusbd.75570
• Ataş, E., & Güneş, P. (2020). Altıncı sınıf fen bilimleri dersi sınav sorularının yeniden yapılandırılmış Bloom taksonomisine göre değerlendirilmesiEvaluation of sixth grade science exam items according to the restructured Bloom's taxonomy. Bolu Abant İzzet Baysal Üniversitesi Eğitim Fakültesi DergisiBolu Abant Izzet Baysal University Journal of Faculty of Education, 20 (2), 1066-1078. https://doi.org/10.17240/aibuefd.2020..-632040
• Ayvacı, H. Ş., & Türkdoğan, A. (2009). Yeniden yapılandırılan bloom taksonomisine göre fen ve teknoloji dersi yazılı sorularının incelenmesiAnalysing science and technology course written items according to the restructured bloom taxonomy. Türk Fen Eğitimi Dergisi Journal of Turkish Science Education, 7(1), 13-25.
• Boyd, B. T. (2008). Effects of state tests on classroom test items in mathematics. School Science and Mathematics, 108(6), 251-262.
• Brookhart, S. M. (2010). How to assess higher-order thinking skills in your classroom (First Edition). USA: ASCD.
• Cangüven, H. D., Öz, O., Binzet, G., & Avcı, G. (2017). Millî Eğitim Bakanlığı 2017 fen bilimleri taslak programının Yenilenmiş Bloom Taksonomisi’ne göre incelenmesiExamining the draft science program of the Ministry of National Education 2017 according to the Revised Bloom's Taxonomy. International Journal of Eurasian Education and Culture, 2, 2-80.
• Cansüngü Koray, Ö. & Yaman, S. (2002). Fen bilgisi öğretmenlerinin soru sorma becerilerinin bloom taksonomisine göre değerlendirilmesiEvaluation of science teachers' item asking skills according to bloom taxonomy. Kastamonu Eğitim DergisiKastamonu Education Journal, 10(2), 317-324.
• Chaffee, J. (1994). Thinking critically (Fourth Edition). Boston: Houghlon Mifflin Company.
• Çepni, S., & Çoruhlu, T. Ş. (2010). Alternatif ölçme ve değerlendirme tekniklerine yönelik hazırlanan hizmet içi eğitim kursundan öğretime yansımalarReflections on teaching from the in-service training course on alternative assessment and evaluation techniques. Pamukkale Üniversitesi Eğitim Fakültesi Dergisi Pamukkale University Journal of Education, 28(28), 117-128.
• Çoruhlu, T. Ş., Nas, S. E., & Çepni, S. (2008). Fen ve teknoloji öğretmenleri için alternatif ölçme ve değerlendirme tekniklerine yönelik bir hizmet içi eğitim programından yansımalar: Trabzon örneğiReflections from an in-service training program on alternative assessment and evaluation techniques for science and technology teachers: Trabzon case. Necatibey Eğitim Fakültesi Elektronik Fen ve Matematik Eğitimi Dergisi Necatibey Faculty of Education Electronic Journal of Science and Mathematics Education, 2(2), 1-22.
• Davey, L. (2009). The application of case study evaluations. Elementary Education Online, 8(2),1-3.
• Dindar, H., & Demir, M. (2006). Beşinci sınıf öğretmenlerinin fen bilgisi dersi sınav sorularının Bloom Taksonomisi’ne göre değerlendirilmesiEvaluation of fifth grade teachers' science course exam items according to Bloom's Taxonomy. Gazi Eğitim Fakültesi Dergisi Gazi University Journal of Gazi Educational Faculty (GUJGEF, 26(3), 87-96.
• Driana, E., & Ernawati, E. (2019). Teachers' understanding and practices in assessing higher order thnking skills at primary schools. Acitya: Journal of Teaching and Education, 1(2), 110-118. https://doi.org/10.30650/ajte.v1i2.233.
• Ernst-Slavit, G., & Pratt, K. L. (2017). Teacher items: Learning the discourse of science in a linguistically diverse elementary classroom. Linguistics and Education, 40, 1-10.
• Gillham, B. (2000). Case Study Research Methods. British Library Cataloguing in Publication Data. Typeset by Paston Pre Press Ltd., Beccles, Suffolk. Printed and bound in Great Britain by TJ International Ltd., Padstow, Cornwall.
• Gökulu, A. (2015). Fen ve teknoloji öğretmenlerinin yazılı sınav soruları ile TEOG sınavlarında sorulan fen ve teknoloji sorularının Yenilenmiş Bloom Taksonomisi’ne göre incelenmesiExamination of science and technology teachers' written exam items and science and technology items asked in TEOG exams according to the Revised Bloom's Taxonomy. Route Educational and Social Science Journal, 2(2), 434-446.
• Güleryüz, H., & Erdoğan, İ. (2018). Orta okul fen bilimleri dersi sınav sorularının Bloom’un bilişsel alan taksonomisine göre değerlendirilmesi: Muş İli örneğiEvaluation of secondary school science exam items according to Bloom's cognitive domain taxonomy: The case of Muş Province.. Anemon Muş Alparslan Üniversitesi Sosyal Bilimler DergisiThe Journal of Social Sciences, 6(1), 43-49. https://doi.org/10.18506/anemon.354846.
• Güven, Ç. (2014). 6, 7, 8. Sınıflar Fen ve Teknoloji Dersi Öğretim Programındaki Soruların Yenilenmiş Bloom Taksonomisi’ne Göre İncelenmesiInvestigation of the Items in the 6th, 7th, 8th Grades Science and Technology Course Curriculum According to the Revised Bloom's Taxonomy. Yayımlanmamış Yüksek Lisans TeziUnpublished Master Thesis, Ahi Evran Üniversitesi Fen Bilimleri EnstitüsüAhi Evran University Institute of Science and Technology, Kırşehir.
• Güneş, F. (2012). Öğrencilerin düşünme becerilerini geliştirmeDeveloping students' thinking skills. Türklük Bilimi Araştırmaları Journal of Turkology Research , 32, 127-146.
• İnceçam, B., Demir, E., & Demir, E. (2018). Ortaokul Öğretmenlerinin Sınıf İçi Ölçme ve Değerlendirmelerde Yazılı Yoklamalarda Kullandıkları Açık Uçlu Maddeleri Hazırlama YeterlikleriSecondary School Teachers' Competencies of Preparing Open-Ended Items Used in Written Assignments in Classroom Measurement and Assessment. İlköğretim OnlinePrimary Education Online, 17(4). https://doi.org/10.17051/ilkonline.2019.506900
• Karabulut, A. (2017). MEB’in yayınlamış olduğu örnek pisa sorularının 2015 Türkçe öğretim programı okuma anlama kazanımları çerçevesinde analiziAnalysis of sample Pisa items published by MoNE within the framework of reading comprehension acquisitions of 2015 Turkish curriculum. Uluslararası Eğitim Bilim ve Teknoloji DergisiInternational Journal of Educational Science and Technology, 3(3), 166-174.
• Kartal, M., & Dibek, M. İ. (2021). Madde yazma eğitiminin özel okulda çalışan ilköğretim İngilizce öğretmenlerinin maddenin kalitesini değerlendirmeye ilişkin yeterliklerine etkisiThe effect of item writing training on the competences of primary school English teachers working in private schools on evaluating item quality. Milli Eğitim DergisiJournal of National Education, 51(236), 3345-3358. https://doi.org/10.37669/milliegitim.991124
• Keskin, M. Ö., & Aydın, S. (2011). Seviye Belirleme Sınavı 6. sınıf fen ve teknoloji testinde çıkan biyoloji sorularının revize edilmiş taksonomi’ye göre incelenmesiExamination of biology items in the 6th grade science and technology test of the Placement Test according to the revised taxonomy. Gazi Eğitim Fakültesi DergisiGazi University Journal of Gazi Educational Faculty (GUJGEF, 31(3), 727-742.
• Koray, Ö., Altunçekiç, A., & Yaman, S. (2005). Fen bilgisi öğretmen adaylarının soru sorma becerilerinin Bloom taksonomisine göre değerlendirilmesiEvaluation of pre-service science teachers' item asking skills according to Bloom's taxonomy. Pamukkale Üniversitesi Eğitim Fakültesi DergisiPamukkale University Journal of Education, 17(17), 33-39.
• Kurnaz Adıbatmaz, B., & Kutlu, Ö. (2020). Bilimsel düşünme becerilerinin ölçülmesiMeasurement of scientific thinking skills (Birinci BaskıFirst Edition). Ankara:Pegem AkademiAnkara:Pegem Academy.
• Lewis, A., & Smith, D. (1993). Defining higher order thinking. Theory into practice, 32(3), 131-137.
• Marso, R. N. & Pigge, F. L. (1988, Nisan). An analysis of teacher-made tests: Testing practices, cognitive demands, and item construction errors. The annual meeting of the national council on measurement in education, New Orleans.
• Millî Eğitim BakanlığıMinistry of National Education. (2018). İlköğretim kurumları fen bilimleri dersi öğretim programıPrimary school science curriculum, Talim ve Terbiye Kurulu BaşkanlığıPresidency of the Board of Education, Ankara.
• Miri, B., David, B. C., & Uri, Z. (2007). Purposely teaching for the promotion of higher-order thinking skills: a case of critical thinking. Research in Science Education, 37(4), 353-369. https://doi.org/10.1007/s11165-006-9029-2
• Mitana, J. M. V., Muwagga, A. M., & Ssempala, C. (2018). Assessment of higher order thinking skills: A case of Uganda Primary Leaving Examinations. African Educational Research Journal, 6(4): 240-249. https://doi.org/ 10.30918/AERJ.64.18.083
• Mutlu, M., Uşak, M., & Aydoğdu, M. (2003). Fen bilgisi sorularının bloom taksonomisine göre değerlendirilmesiEvaluation of science items according to bloom taxonomy. Kırşehir Eğitim Fakültesi Dergisi Ahi Evran University Journal of Kırşehir Faculty of Education, 4(2), 87-95.
• National Research Council. (2007). Taking Science to School; NRC. Washington, DC: The National Academy Press.
• Oescher, J., & Kirby, P. C. (1990, April). Assessing teacher-made tests in secondary math and science classrooms. The anual meeting of the national council on measurement in education, Boston.
• Özcan, S., & Akcan, K. (2010). Fen bilgisi öğretmen adaylarının hazırladığı soruların içerik ve Bloom taksonomisi'ne uygunluk yönünden incelenmesiAnalysing the items prepared by prospective science teachers in terms of content and appropriateness to Bloom's taxonomy. Kastamonu Eğitim DergisiKastamonu Education Journal, 18(1), 323-330.
• Punch, K. F. (2013). Introduction To Social Research: Quantitative and Qualitative Approaches. New York: Sage.
• Şanlı, C., & Pınar, A. (2017). Sosyal bilgiler dersi sınav sorularının Yenilenen Bloom Taksonomisi’ne göre incelenmesiAnalysing social studies course exam items according to Bloom's Taxonomy. İlköğretim OnlinePrimary Education Online, 16(3), 949-959. https://doi.org/10.17051/ilkonline.2017.330234
• Tanık, N., & Saraçoğlu, S. (2011). Fen ve teknoloji dersi yazılı sorularının Yenilenmiş Bloom Taksonomisi’ne göre incelenmesiAnalysing science and technology course written items according to the Revised Bloom's Taxonomy. TÜBAV Bilim Dergisi Journal of TUBAV Science, 4(4), 235-246.
• Üstüner, A., & Şengül, M. (2004). Çoktan seçmeli test tekniğinin türkçe öğretimine olumsuz etkileriNegative effects of multiple choice test technique on Turkish language teaching. Fırat Üniversitesi Sosyal Bilimler DergisiJournal of Social Sciences, 14 (2), 197-208.
• Villarroel, V., Bruna, D., Brown, G. T. L., & Bustos, C. (2021). Changing the quality of teachers’ written tests by implementing an authentic assessment teachers’ training program. International Journal of Instruction, 14(2), 987-1000. https://doi.org/10.29333/iji.2021.14256a
• Wilson, D. M., & Narasuman, S. (2020). Investigating teachers' implementation and strategies on higher order thinking skills in school-based assessment instruments. Asian Journal of University Education, 16(1), 70-84. https://doi.org/10.24191/ajue.v16i1.8991
• Yenilmez, K., & Yolcu, B. (2007). Öğretmen davranışlarının yaratıcı düşünme becerilerinin gelişimine katkısıÖğretmen davranışlarının yaratıcı düşünme becerilerinin gelişimine katkısı. Manas Üniversitesi Sosyal Bilimler DergisiManas Journal of Social Studies, 9(18), 95-105.
• Yıldız, D. Ç. (2015). The analysis of Turkish course exam items according to re-constructed Bloom’s taxonomy. Gaziantep University Journal of Social Sciences, 14(2), 479-497. https://doi.org/10.21547/jss.256771
• Yin, R. K. (2003). Case study research design and methods (Third Edition). London: Sage Publications.
• Yip, D. Y. (2004). Questioning skills for conceptual change in science instruction. Journal of Biological Education, 38(2), 76-83.
• Yurdakul, B., Başokçu, T. O. & Yazıcılar, Ü. (2020). Evaluation of the professional development program for secondary math teachers on item writing related to higher order thinking skills. Journal of Teacher Education and Educators, 9(1), 83-106.
• Yüksel, M. (2022). PISA 2018 araştırma sonuçlarına göre ülkelerin bileşik PISA performans sıralamasıRanking the composite PISA performance of countries according to PISA 2018 research results. Muğla Sıtkı Koçman Üniversitesi Eğitim Fakültesi Dergisi MSKU Journal of Education, 9(2), 788-821.
• Zorluoğlu, S. L., Kızılaslan, A. & Sözbilir, M. (2016). School chemistry curriculum according to revised bloom taxonomy. Necatibey Faculty of Education Electronic Journal of Science and Mathematics Education, 10(1), 260-279.
• Zorluoğlu, S. L., Şahintürk A. & Bağrıyanık K. E. (2017). 2013 yılı Fen Bilimleri Öğretim Programı kazanımlarının Yenilenmiş Bloom Taksonomisi’ne göre analizi ve değerlendirilmesiAnalysis and evaluation of 2013 Science Curriculum outcomes according to the Revised Bloom's Taxonomy.. Bartın Üniversitesi Eğitim Fakültesi DergisiBartın University Journal of Faculty of Education, 6(1), 1-15. Doi: https://doi.org/10.14686/buefad.267190