The Whole is Greater Than the Sum of Its Parts: How to Write A Systematic Review in Educational Sciences?

Year 2023, Volume: 25 Issue: 3, 538 - 550, 29.09.2023

Münevver İlgün Dibek , Burcu Toptaş

https://doi.org/10.17556/erziefd.1176257

Cited By: 1

Abstract

Increasing research in social sciences offers researchers the opportunity to conduct comparative studies in different contexts. As the number of studies increases, both researchers and practitioners are conflicted about the effectiveness of the interventions. In fact, the outcomes may be complicated, conflicting, or inconsistent. Therefore, further studies are required. The aim of this study is to provide information on how systematic reviews can be used in the educational sciences. Systematic reviews offer comprehensive and in-depth findings on a subject to researchers, policymakers, and practitioners. In this study, the definition of systematic review, its application in educational sciences, and its stages were compiled in light of the discussions in the literature. Accordingly, a systematic review can be divided into three stages: designing of the systematic review, search and screening, and review reporting. When conducting a systematic review, some biases need to be taken into account, including selection, publication, time-lag, selective outcome reporting, lack of risk of bias assessment, duplication, language, duplicate publication, and database biases. Various platforms and software can be used while preparing a systematic review; however, these should be used carefully, especially to avoid bias.

Keywords

traditional review, systematic review, qualitative and quantitative systematic review

Bütün Parçaların Toplamından Daha Fazladır: Eğitim Bilimlerinde Sistematik Derleme Nasıl Yazılır?

Abstract

Sosyal bilimlerde her geçen gün artan araştırmalar, araştırmacılara farklı bağlamlarda karşılaştırmalı çalışmalar yapma fırsatı sunmaktadır. Araştırma sayısı arttıkça hem araştırmacılar hem uygulayıcılar, müdahale programlarının (uygulamaların) etkililiğinin ne olduğu konusunda çelişkiye düşmektelerdir. Nitekim sonuçlar karmaşık, tutarsız ya da çelişkili olabilmektedir. Bu noktada, araştırmalar üstü çalışmaların yapılmasına gerek duyulmaktadır. Bu çalışmanın amacı sistematik derlemelerin eğitim bilimlerinde kullanımı hakkında bilgi vermektir. Sistematik derlemeler, araştırmacılara, politika yapıcılara ve uygulayıcılara bir konuda derinlemesine ve detaylı sonuçlar sunmaktadır. Bu çalışmada sistematik derlemenin tanımı, eğitim bilimlerinde kullanımı ve aşamaları alanyazındaki tartışmalar ışığında derlenmiştir. Buna göre sistematik derlemenin üç aşamasından söz edilebilir. Bu aşamalar; derlemenin tasarlanması, arama ve tarama ile derlemenin raporlaştırılmasıdır. Sistematik derleme tasarlanırken dikkat edilmesi gereken bazı yanlılık türleri vardır. Bu yanlılık türleri ise seçim, basım, gecikme, seçici sonuç raporlama, yanlılık riski değerlendirmesinin azlığı, dil, duplikasyon ve veritabanı yanlılığıdır. Sistematik derleme hazırlanırken çeşitli yazılımlar ve platformalardan yararlanılabilir; ancak özellikle yanlılık oluşmaması açısından bunların dikkatli kullanılması gerekmektedir.

Keywords

geleneksel derleme, sistematik derleme, nitel ve nicel sistematik derleme

References

• Amir-Behghadami, M., & Janati, A. (2020). Population, intervention, comparison, outcomes and study (PICOS) design as a framework to formulate eligibility criteria in systematic reviews. Emergency Medicine Journal, 37(6), 387. http://dx.doi.org/10.1136/emermed-2020-209567
• Bakken, R. K., Næss, K. A. B., Lemons, C. J., & Hjetland, H. N. (2021). A systematic review and meta analysis of reading and writing interventions for students with disorders of intellectual development. Education Sciences, 11, 638. https://doi.org/10.3390/educsci11100638
• Boland, A., Dickson, R., & Cherry, G. (2017). Doing a systematic review: A student's guide (2nd edition). Sage Publications.
• Boutron, I., Page, M. J., Higgins, J. P. T., Altman, D. G., Lundh, A., & Hróbjartsson, A. (2022). Considering bias and conflicts of interest among the included studies. In J. P. T. Higgins, J. Thomas, J. Chandler, M. Cumpston, T. Li, M. J. Page, V. A. Welch (Eds.), Cochrane handbook for systematic reviews of interventions (pp. 177-199). Wiley-Blackwell.
• Brunton, G., Thomas, J., O'Mara-Eves, A., Jamal, F., Oliver, S., & Kavanagh, J. (2017). Narratives of community engagement: a systematic review-derived conceptual framework for public health interventions. BMC Public Health, 17(1), 944. https://doi.org/1186/12889-017-4958-4
• Coughlan, M., & Cronin, P. (2017). Doing a literature review in nursing, health and social care (2nd Edition). Sage Publications.
• Çınar N. (2021). İyi bir sistematik derleme nasıl yazılmalı? Online Türk Sağlık Bilimleri Dergisi, 6(2), 310-314. https://doi.org/10.26453/otjhs.888569
• Domecq, J. P., Prutsky, G., Elraiyah, T., Wang, Z., Nabhan, M., Shippee, N., ... & Murad, M. H. (2014). Patient engagement in research: a systematic review. BMC Health Services Research, 14(1), 1-9.
• Dwan, K., Altman, D. G., Arnaiz, J. A., Bloom, J., Chan, A. W., Cronin, E., ... & Williamson, P. R. (2008). Systematic review of the empirical evidence of study publication bias and outcome reporting bias. PloS One, 3(8), 1-31. https://doi.org/10.1371/journal.pone.0003081
• Eagly, A. H., & Wood, W. (1994). Using research to plan future research. In H. Cooper & L. V. Hedges (Eds.), The handbook of research synthesis (pp. 485-500). Russell Sage Foundation.
• Esfijani, A. (2018). Measuring quality in online education: A meta-synthesis. American Journal of Distance Education, 32(1), 57-73.
• Finfgeld-Connett, D. (2018). A guide to qualitative meta-synthesis. Routledge.
• Foo, Y. Z., O’Dea, R. E., Koricheva, J., Nakagawa, S., & Lagisz, M. (2021). A practical guide to question formation, systematic searching and study screening for literature reviews in ecology and evolution. Methods in Ecology and Evolution, 12, 1705-1720. https://doi.org/10.1111/2041-210X.13654
• Gessler, M., & Siemer, C. (2020). Umbrella review: Methodological review of reviews published in peer-reviewed journals with a substantial focus on vocational education and training research. International Journal for Research in Vocational Education and Training, 7(1), 91-125.
• Gough, D., Oliver, S., & Thomas, J. (2012). Introducing systematic reviews. In D. Gough, S. Oliver & J. Thomas (Eds.), An introduction to systematic reviews (pp. 1-16). Sage Publications.
• Grant, M. J., & Booth, A. (2009). A typology of reviews: An analysis of 14 review types and associated methodologies. Health Information and Librairies Journal, 26(2), 91-108. https://doi.org/10.1111/j.1471-1842.2009.00848.x
• Higgins, J. P. T., Thomas, J., Chandler, J., Cumpston, M., Li, T., Page, M. J., & Welch, V. A. (Eds.) (2019). Cochrane handbook for systematic reviews of interventions (2nd Edition). Wiley-Blackwell.
• Hong, Q. N., Pluye, P., Fabregues, S., Barlett, G., Boardman, F., Cargo, M., Dagenais, P., Gagnon, M.P., Griffiths, F., Nicolau, B., O’Cathain, A., Rousseau, M.C., & Isabelle, V. (2018). Mixed methods appraisal tool (MMAT) version 2018 user guide. McGill University.
• Jaafar, M. H., Mahadeva, S., Morgan, K., & Tan, M. P. (2016). Systematic review of qualitative and quantitative studies on the attitudes and barriers to percutaneous endoscopic gastrostomy feeding. Clinical Nutrition, 35(6), 1226-1235. https://doi.org/10.1016/j.clnu.2016.04.019
• Jesson, J., Matheson, L., & Lacey, F. M. (2011). Doing your literature review: Traditional and systematic techniques. SAGE Publications.
• Kalantar, M. (2021, January 19). 5 software tools to support your systematic review processes. Research Skills. Retrieved August 15, 2022, from https://www.ifis.org/en/research-skills-blog/software-tools-to-support-your-systematic-review-processes
• Karaçam Z. (2013). Sistematik derleme metodolojisi: Sistematik derleme hazırlamak için bir rehber. Dokuz Eylül Üniversitesi Hemşirelik Fakültesi Elektronik Dergisi, 6(1), 26-33.
• Kazrin, A., Durac, J., & Agteros, T. (1979). Meta-meta analysis: A new method for evaluating therapy outcome. Behaviour Research and Therapy, 17(4), 397–399.
• Khan, M. A. (2021). COVID-19’s impact on higher education: a rapid review of early reactive literature. Education Sciences, 11(8), 421. https://doi.org/10.3390/educsci11080421
• Kocak, O. Goksu, I, & Goktas, Y. (2021). The factors affecting academic achievement: a systematic review of meta analyses. International Online Journal of Education and Teaching (IOJET), 8(1), 454-484.
• Lefebvre, C., Glanville, J., Briscoe, S., Littlewood, A., Marshall, C., Metzendorf, I., Noel-Storr, A., Rader, T., Shokraneh, F., Thomas, J., & Wieland, L. S. (2019). Searching for and selecting studies. In J.P.T. Higgins, J. Thomas, J. Chandler, M. Cumpston, T. Li, M.J. Page, V.A. Welch (Eds.), Cochrane handbook for systematic reviews of interventions (pp. 67-99). Wiley-Blackwell.
• Littell, H. J., Corcoran, J., & Pillai, V. (2008). Systematic reviews and meta-analysis. Oxford University Press.
• Lo, C. K. (2020). Systematic reviews on flipped learning in various education contexts. In O. Zawacki-Richter, M. Kerres, S. Bedenlier, M. Bond, & K. Buntins (Eds.), Systematic reviews in educational research: Methodology, perspectives and application (pp. 129-143). Springer.
• MacFarlane, A., Russell-Rose, T. & Shokraneh, F. (2022). Search strategy formulation for systematic reviews: Issues, challenges and opportunities. Intelligent Systems with Applications, 15, 1-10. https://doi.org/10.1016/j.iswa.2022.200091
• Mallett, R., Hagen-Zanker, J., Slater, R., & Duvendack, M. (2012). The benefits and challenges of using systematic reviews in international development research. Journal of Development Effectiveness, 4(3), 445-455. https://doi.org/10.1080/19439342.2012.711342
• Møiniche, S., Kehlet, H., & Dahl, J. B. (2002). A qualitative and quantitative systematic review of preemptive analgesia for postoperative pain relief: the role of timing of analgesia. The Journal of the American Society of Anesthesiologists, 96(3), 725-741. https://doi.org/10.1097/00000542-200203000-00032
• Munn, Z., Stern, C., Aromataris, E., Lockwood, C., & Jordan, Z. (2018). What kind of systematic review should I conduct? A proposed typology and guidance for systematic reviewers in the medical and health sciences. BMC Medical Research Methodology, 18(1), 1-9. https://doi.org/10.1186/s12874-017-0468-4
• Naisbitt, J. (1982). Megatrends: Ten new directions transforming our live. Warner Books, Inc.
• Newman, M., & Gough, D. (2020). Systematic reviews in educational research: methodology, perspectives and application. In O. Zawacki-Richter, M. Kerres, S. Bedenlier, M. Bond & K. Buntins (Eds.), Systematic reviews in educational research: Methodology, perspectives and application (pp. 3-22). Springer.
• Norman, C. R. (2020). Systematic review automation methods. [Doctoral dissertation, University of Amsterdam]. Proquest Dissertations and Theses.
• Özdemir, N., Gümüş, S., Kılınç, A. Ç., & Bellibaş, M. Ş. (2022). A systematic review of research on the relationship between school leadership and student achievement: An updated framework and future direction. Educational Management Administration & Leadership, 1-27. https://doi.org/10.1177/17411432221118662
• Page, M. J., McKenzie, J. E., Bossuyt, P. M., Boutron, I., Hoffmann, T. C., Mulrow, C. D. et al. (2021). The PRISMA 2020 statement: An updated guideline for reporting systematic reviews, BMJ, 372. https://doi.org/10.1136/bmj.n71
• Patino, C. M., & Ferreira, J. C. (2018). Inclusion and exclusion criteria in research studies: Definitions and why they matter. Journal Brasileiro de Pneumologia Bras Pneumol, 44(2), 84. https://doi.org/10.1590/s1806-37562018000000088
• Petticrew, M., & Roberts, H. (2006). Systematic reviews in the social sciences: A practical guide. Blackwell Publishing.
• PRISMA 2020 expanded checklist (2020). Retrieved September 9, 2022 from https://prisma-statement.org/documents/PRISMA_2020_expanded_checklist.pdf
• Reitsma, J. B., Rutjes A. W. S., Whiting, P., Vlassov, V. V., Leeflang, M. M. G., & Deeks, J. J. (2009). Assessing methodological quality. In: J. J. Deeks, P. M. Bossuyt, C. Gatsonis (Eds.), Cochrane handbook for systematic reviews of diagnostic test accuracy. The Cochrane Collaboration.
• ReviewManager (RevMan) is Cochrane's bespoke software for writing Cochrane Reviews. (n. d.). Retrieved September 9, 2022 from https://training.cochrane.org/online-learning/core-software/revman
• ROBINS-I tool (n. d.) Retrieved September, 7, 2022 from https://methods.cochrane.org/methods-cochrane/robins-i-tool
• Schlosser, R. W. (2007). Appraising the quality of systematic reviews. Focus, 17, 1-8. Retrieved September 9, 2022 from https://ktdrr.org/ktlibrary/articles_pubs/ncddrwork/focus/focus17/Focus17.pdf Shamseer, L. (2015). Planning a systematic review? Think protocols. Retrieved September 9, 2022 from http://blogs.biomedcentral.com/bmcblog/2015/01/05/planning-a-systematic-review-think-protocols/
• Sims, S., Fletcher-Wood, H., O’Mara-Eves, A., Cottingham, S., Stansfield, C., Van Herwegen, J., & Anders, J. (2021). What are the characteristics of teacher professional development that increase pupil achievement? A systematic review and meta-analysis. Education Endowment Foundation.
• Smith, S.M., Wallace, E., Clyne, B., Boland, F., & Fortin, M. (2021). Interventions for improving outcomes in patients with multimorbidity in primary care and community setting: A systematic review. Systematic Review, 10, 271. https://doi.org/10.1186/s13643-021-01817-z
• Song, F., Parekh, S., Hooper, L., Loke, Y. K, Ryder, J., Sutton, A. J., Hing, C., Kwok, C. S, Pang, C. & Harvey, I. (2010). Dissemination and publication of research findings: an updated review of related biases. Health Technology Assessment, 14(8), 1-193. https://doi.org/10.3310/hta14080
• Stern, C., & Kleijnen, J. (2020). Language bias in systematic reviews: you only get out what you put in. JBI Evidence Synthesis, 18(9), 1818-1819. https://doi.org/10.11124/JBIES-20-00361
• Sun, J., & Leithwood, K. (2015). Direction-setting school leadership practices: A meta-analytical review of evidence about their influence. School Effectiveness and School Improvement, 26(4), 499-523. https://doi.org/10.1080/09243453.2015.1005106
• Sur, H. (2020). Ethical considerations of conducting systematic reviews in educational research. In O. Zawacki-Richter, M. Kerres, S. Bedenlier, M. Bond & K. Buntins (Eds.), Systematic reviews in educational research: Methodology, perspectives and application (pp. 41-54). Springer.
• Tricco, A. C., Langlois, E., & Straus, S. E. (2017). Rapid reviews to strengthen health policy and systems: a practical guide; world health organization, alliance for health policy and systems research. World Health Organization.
• Tsafnet, G., Glasziou, P., Choong, M., Dunn, K., Galgani, A., & Coeira, E. (2014). Systematic review automation technologies. Systematic Review, 3(74), 1-15. https://doi.org/10.1186/2046-4053-3-74
• Vassar, M., Yerokhin, V., Sinnett, P. M., Weiher, M., Muckelrath, H., Carr, B., ... & Cook, G. (2017). Database selection in systematic reviews: an insight through clinical neurology. Health Information & Libraries Journal, 34(2), 156-164.
• Wright, R. W., Brand, R. A., Dunn, W., & Spindler, K. P. (2007). How to write a systematic review. Clinical Orthopaedics and Related Research, 455, 23-29.
• Wu, H., & Shen, J. (2021). The association between principal leadership and student achievement: A multivariate meta-meta-analysis. Educational Research Review, 35, 1-17. https://doi.org/10.1016/j.edurev.2021.100423
• Yannascoli, S. M., Schenker, M. L., Carey, J. L., Ahn, J., & Baldwin, K. D. (2013). How to write a systematic review: A step-by-step guide. University of Pennsylvania Orthopaedic Journal, 23, 64-69.
• Yavuz, N. (2022). Sosyal bilimlerde sistematik literatür analizi. Pamukkale Üniversitesi Sosyal Bilimler Enstitüsü Dergisi, 2022(51), Ö347-Ö360. https://doi.org/10.30794/pausbed.1134606
• Yılmaz, K. (2021). Sosyal bilimlerde ve eğitim bilimlerinde sistematik derleme, meta değerlendirme ve bibliyometrik analizler. Manas Sosyal Araştırmalar Dergisi, 10(2), 1457-1490. https://doi.org/10.33206/mjss.791537
• Young, C. (2021). Conditions in which a flipped classroom can successfully be implemented: A traditional literature review. Retrieved August 15, 2022, from https://scholar.uwindsor.ca/cgi/viewcontent.cgi?article=1044&context=itos21
• Yu, T., Enkh-Amgalan, N., & Zorigt, G. (2017). Methods to perform systematic reviews of patient preferences: A literature survey. BMC Medical Research Methodology, 17(166), 1-10. https://doi.org/10.1186/s12874-017-0448-8
• Zhang, Y., Luo, R., Zhu, R.& Yin, Y. (2021). Educational robots improve K-12 students’ computational thinking and STEM attitudes: Systematic review. Journal of Educational Computing Research, 59(7), 1450–1481. https://doi.org/10.1177/0735633121994070
• http://eppi.ioe.ac.uk/cms/Default.aspx?alias=eppi.ioe.ac.uk/cms/er4