Research Article
BibTex RIS Cite

Comparison of ongoing COVID-19 pandemic confirmed cases/deaths weekly forecasts on continental basis using R statistical models

Year 2021, , 635 - 644, 29.09.2021
https://doi.org/10.24012/dumf.1002160

Abstract

The aim of this study is to contribute to the literature by estimating the 5-weeks number of cases/deaths for each continent by using statistical-based prediction models, which are quite effective on simple but small-scale datasets. While Auto.arima, Tbats, Naive, Holt, Thetaf and, Drift models were used for prediction processes root mean square error (RMSE), mean absolute error (MAE), and mean absolute percent error (MAPE) metrics were used for evaluating estimates. According to the confirmed cases MAPE metric values of the 5 continents analyzed, the best predictions for Asia, Africa, Europe, America, and Oceania were done by Thetaf, Naive, Thetaf, Auto.arima, and Auto.arima models, respectively. The use of very limited data for time series estimates such as 57-weeks in the estimation process was a disadvantage. Most models require at least two cycles, 104 weeks of data, to run. Therefore, we could not use models such as neural network autoregressive, multilayer perceptrons, extreme learning machines.
The results obtained with the prediction models used in this study aim to make more accurate decisions for the authorized persons dealing with health to be more prepared for future conditions and health systems.

References

  • [1] S. D. Pitlik, “Covid-19 compared to other pandemic diseases,” Rambam Maimonides Med. J., vol. 11, no. 3, pp. 1–17, 2020.
  • [2] D. A. Tyrrell and M. L. Bynoe, “Cultivation of viruses from a high proportion of patients with colds.,” Lancet, vol. 1, no. 7428, pp. 76–77, 1966.
  • [3] N. Zhu et al., “A Novel Coronavirus from Patients with Pneumonia in China, 2019,” N. Engl. J. Med., vol. 382, no. 8, pp. 727–733, 2020.
  • [4] J. Cui, F. Li, and Z. L. Shi, “Origin and evolution of pathogenic coronaviruses,” Nat. Rev. Microbiol., vol. 17, no. 3, pp. 181–192, 2019.
  • [5] R. Wölfel et al., “Virological assessment of hospitalized patients with COVID-2019,” Nature, vol. 581, no. 7809, pp. 465–469, 2020.
  • [6] D. Sornette, E. Mearns, M. Schatz, K. Wu, and D. Darcet, “Interpreting, analysing and modelling COVID-19 mortality data,” Nonlinear Dyn., vol. 101, no. 3, pp. 1751–1776, 2020.
  • [7] S. Perlman, “Another Decade, Another Coronavirus,” N. Engl. J. Med., vol. 382, no. 8, pp. 760–762, 2020.
  • [8] C. Wang, P. W. Horby, F. G. Hayden, and G. F. Gao, “A novel coronavirus outbreak of global health concern,” Lancet, vol. 395, no. 10223, pp. 470–473, 2020.
  • [9] Fang, “Sensitivity of Chest CT for COVID-19: Comparison to RT-PCR,” Radiology, vol. 395, no. 3, pp. A1–A2, 2020.
  • [10] N. Chen et al., “Epidemiological and clinical characteristics of 99 cases of 2019 novel coronavirus pneumonia in Wuhan, China: a descriptive study,” Lancet, vol. 395, no. 10223, pp. 507–513, 2020.
  • [11] W. Wang et al., “Detection of SARS-CoV-2 in Different Types of Clinical Specimens,” JAMA - J. Am. Med. Assoc., vol. 323, no. 18, pp. 1843–1844, 2020.
  • [12] T. Singhal, “Review on COVID19 disease so far,” Indian J. Pediatr., vol. 87, no. April, pp. 281–286, 2020.
  • [13] D. Wang et al., “Clinical Characteristics of 138 Hospitalized Patients with 2019 Novel Coronavirus-Infected Pneumonia in Wuhan, China,” JAMA - J. Am. Med. Assoc., vol. 323, no. 11, pp. 1061–1069, 2020.
  • [14] Ö. Açikgöz and A. Günay, “The early impact of the Covid-19 pandemic on the global and Turkish economy,” Turkish J. Med. Sci., vol. 50, no. SI-1, pp. 520–526, 2020.
  • [15] S. Boccaletti, W. Ditto, G. Mindlin, and A. Atangana, “Modeling and forecasting of epidemic spreading: The case of Covid-19 and beyond,” Chaos, Solitons and Fractals, vol. 135, 2020.
  • [16] W. Guan et al., “Clinical Characteristics of Coronavirus Disease 2019 in China,” N. Engl. J. Med., vol. 382, no. 18, pp. 1708–1720, 2020.
  • [17] T. P. Velavan and C. G. Meyer, “The COVID-19 epidemic,” Trop. Med. Int. Heal., vol. 25, no. 3, pp. 278–280, 2020.
  • [18] C. Huang et al., “Clinical features of patients infected with 2019 novel coronavirus in Wuhan, China,” Lancet, vol. 395, no. 10223, pp. 497–506, 2020.
  • [19] M. L. Holshue et al., “First Case of 2019 Novel Coronavirus in the United States,” N. Engl. J. Med., vol. 382, no. 10, pp. 929–936, 2020.
  • [20] K. Q. Kam et al., “A well infant with coronavirus disease 2019 with high viral load,” Clin. Infect. Dis., vol. 71, no. 15, pp. 847–849, Aug. 2020.
  • [21] S. R. Weiss and J. L. Leibowitz, Coronavirus pathogenesis, 1st ed., vol. 81. Elsevier Inc., 2011.
  • [22] M. L. McGowan, A. H. Norris, and D. Bessett, “Care Churn — Why Keeping Clinic Doors Open Isn’t Enough to Ensure Access to Abortion,” N. Engl. J. Med., vol. 383, no. 6, pp. 508–510, 2020.
  • [23] H. Yao, J. H. Chen, and Y. F. Xu, “Patients with mental health disorders in the COVID-19 epidemic,” The Lancet Psychiatry, vol. 7, no. 4, p. e21, 2020.
  • [24] M. Nicola et al., “The socio-economic implications of the coronavirus pandemic ( COVID-19 ): A review,” Int. J. Surg., vol. 78, no. April, pp. 185–193, 2020.
  • [25] Z. Pala and O. Özkan, “Artificial Intelligence Helps Protect Smart Homes against Thieves,” DÜMF Mühendislik Derg., vol. 11, no. 3, pp. 945–952, 2020.
  • [26] S. S. Mohamed Ali, A. H. Alsaeedi, D. Al-Shammary, H. H. Alsaeedi, and H. W. Abid, “Efficient intelligent system for diagnosis pneumonia (SARSCOVID19) in X-ray images empowered with initial clustering,” Indones. J. Electr. Eng. Comput. Sci., vol. 22, no. 1, pp. 241–251, 2021.
  • [27] Z. Pala and M. Şana, “Attackdet: Combining web data parsing and real-time analysis with machine learning,” J. Adv. Technol. Eng. Res., vol. 6, no. 1, pp. 37–45, 2020.
  • [28] B. Saman, M. M. A. Eid, and M. M. Eid, “Recently employed engineering techniques to reduce the spread of COVID-19 (Corona Virus disease 2019): A review study,” Indones. J. Electr. Eng. Comput. Sci., vol. 22, no. 1, pp. 277–286, 2021.
  • [29] İ. H. Ünlük and Z. Pala, “Prediction of monthly electricity consumption used in Mu ş Alparslan University Complex by means of Classical and Deep Learning methods,” Int. Conf. Data Sci. Mach. Learn. Stat. - 2019, vol. 1, no. 1, pp. 237–239, 2019.
  • [30] Z. Pala, “Using forecastHybrid Package to Ensemble Forecast Functions in the R,” Int. Conf. Data Sci. Mach. Learn. Stat. - 2019, vol. 1, no. 1, pp. 45–47, 2019.
  • [31] E. Yaldız and Z. Pala, “Time Series Analysis of Radiological Data of Outpatients and Inpatients in Emergency Department of Mus State Hospital,” Int. Conf. Data Sci. Mach. Learn. Stat. - 2019, pp. 234–236, 2019.
  • [32] F. Jiang et al., “Artificial intelligence in healthcare: Past, present and future,” Stroke and Vascular Neurology, vol. 2, no. 4. BMJ Publishing Group, pp. 230–243, 01-Dec-2017.
  • [33] S. Chakraborti et al., “Evaluating the plausible application of advanced machine learnings in exploring determinant factors of present pandemic: A case for continent specific COVID-19 analysis,” Sci. Total Environ., vol. 765, p. 142723, 2020.
  • [34] L. Zhong, L. Mu, J. Li, J. Wang, Z. Yin, and D. Liu, “Early Prediction of the 2019 Novel Coronavirus Outbreak in the Mainland China Based on Simple Mathematical Model,” IEEE Access, vol. 8, pp. 51761–51769, 2020.
  • [35] T. Dehesh, H. A. Mardani-Fard, and P. Dehesh, “Forecasting of COVID-19 Confirmed Cases in Different Countries with ARIMA Models,” medRxiv. medRxiv, 18-Mar-2020.
  • [36] M. Maleki, M. R. Mahmoudi, M. H. Heydari, and K. H. Pho, “Modeling and forecasting the spread and death rate of coronavirus (COVID-19) in the world using time series models,” Chaos, Solitons and Fractals, vol. 140, p. 110151, Nov. 2020.
  • [37] A. Zeroual, F. Harrou, A. Dairi, and Y. Sun, “Deep learning methods for forecasting COVID-19 time-Series data: A Comparative study,” Chaos, Solitons and Fractals, vol. 140, p. 110121, Nov. 2020.
  • [38] T. B. Alakus and I. Turkoglu, “Comparison of deep learning approaches to predict COVID-19 infection,” Chaos, Solitons and Fractals, vol. 140, 2020.
  • [39] J. A. Doornik, J. L. Castle, and D. F. Hendry, “Short-term forecasting of the coronavirus pandemic,” Int. J. Forecast., 2020.
  • [40] Z. Pala and A. F. Pala, “Perform Time-series Predictions in the R Development Environment by Combining Statistical-based Models with a Decomposition-based Approach,” J. Muş Alparslan Univ. Fac. Eng. Archit., vol. 1, no. 1, pp. 1–13, 2020.
  • [41] Y. Yang, J. Dong, X. Sun, E. Lima, Q. Mu, and X. Wang, “A CFCC-LSTM Model for Sea Surface Temperature Prediction,” IEEE Geosci. Remote Sens. Lett., vol. 15, no. 2, pp. 207–211, 2018.
  • [42] Y. Lecun, Y. Bengio, and G. Hinton, “Deep learning,” Nature, vol. 521, no. 7553, pp. 436–444, 2015.
  • [43] P. M. Maçaira, A. M. Tavares Thomé, F. L. Cyrino Oliveira, and A. L. Carvalho Ferrer, “Time series analysis with explanatory variables: A systematic literature review,” Environmental Modelling and Software, vol. 107. Elsevier Ltd, pp. 199–209, 01-Sep-2018.
  • [44] Z. Pala, İ. H. Ünlük, and E. Yaldız, “Forecasting of electromagnetic radiation time series: An empirical comparative approach,” Appl. Comput. Electromagn. Soc. J., vol. 34, no. 8, 2019.
  • [45] Z. Pala, “Examining EMF Time Series Using Prediction Algorithms With R,” vol. 44, no. 2, pp. 223–227, 2021.
  • [46] Z. Pala and R. Atici, “Forecasting Sunspot Time Series Using Deep Learning Methods,” Sol. Phys., vol. 294, no. 5, 2019.
  • [47] R. Atıcı and Z. Pala, “Prediction of the Ionospheric foF2 Parameter Using R Language Forecasthybrid Model Library Convenient Time,” Wirel. Pers. Commun., no. doi.org/10.1007/s11277-021-09050-6 Prediction, pp. 1–20, 2021.
  • [48] R. Calegari, F. S. Fogliatto, F. R. Lucini, J. Neyeloff, R. S. Kuchenbecker, and B. D. Schaan, “Forecasting Daily Volume and Acuity of Patients in the Emergency Department,” 2016.
  • [49] R. J. Hyndman and G. Athanasopoulos, Forecasting : Principles and Practice, 2nd editio. Australia: Monash University, 2018.
  • [50] R. J. Hyndman, A. B. Koehler, R. D. Snyder, and S. Grose, “A state space framework for automatic forecasting using exponential smoothing methods,” Int. J. Forecast., vol. 18, no. 3, pp. 439–454, 2002.
  • [51] Rob J. Hyndman and Yeasmin Khandakar, “Automatic Time Series Forecasting: The forecast Package for R,” J. Stat. Softw., vol. 27, no. 3, p. 22, 2008.
  • [52] F. Petropoulos and S. Makridakis, “Forecasting the novel coronavirus COVID-19,” PLoS One, vol. 15, no. 3, pp. 1–8, 2020.
Year 2021, , 635 - 644, 29.09.2021
https://doi.org/10.24012/dumf.1002160

Abstract

References

  • [1] S. D. Pitlik, “Covid-19 compared to other pandemic diseases,” Rambam Maimonides Med. J., vol. 11, no. 3, pp. 1–17, 2020.
  • [2] D. A. Tyrrell and M. L. Bynoe, “Cultivation of viruses from a high proportion of patients with colds.,” Lancet, vol. 1, no. 7428, pp. 76–77, 1966.
  • [3] N. Zhu et al., “A Novel Coronavirus from Patients with Pneumonia in China, 2019,” N. Engl. J. Med., vol. 382, no. 8, pp. 727–733, 2020.
  • [4] J. Cui, F. Li, and Z. L. Shi, “Origin and evolution of pathogenic coronaviruses,” Nat. Rev. Microbiol., vol. 17, no. 3, pp. 181–192, 2019.
  • [5] R. Wölfel et al., “Virological assessment of hospitalized patients with COVID-2019,” Nature, vol. 581, no. 7809, pp. 465–469, 2020.
  • [6] D. Sornette, E. Mearns, M. Schatz, K. Wu, and D. Darcet, “Interpreting, analysing and modelling COVID-19 mortality data,” Nonlinear Dyn., vol. 101, no. 3, pp. 1751–1776, 2020.
  • [7] S. Perlman, “Another Decade, Another Coronavirus,” N. Engl. J. Med., vol. 382, no. 8, pp. 760–762, 2020.
  • [8] C. Wang, P. W. Horby, F. G. Hayden, and G. F. Gao, “A novel coronavirus outbreak of global health concern,” Lancet, vol. 395, no. 10223, pp. 470–473, 2020.
  • [9] Fang, “Sensitivity of Chest CT for COVID-19: Comparison to RT-PCR,” Radiology, vol. 395, no. 3, pp. A1–A2, 2020.
  • [10] N. Chen et al., “Epidemiological and clinical characteristics of 99 cases of 2019 novel coronavirus pneumonia in Wuhan, China: a descriptive study,” Lancet, vol. 395, no. 10223, pp. 507–513, 2020.
  • [11] W. Wang et al., “Detection of SARS-CoV-2 in Different Types of Clinical Specimens,” JAMA - J. Am. Med. Assoc., vol. 323, no. 18, pp. 1843–1844, 2020.
  • [12] T. Singhal, “Review on COVID19 disease so far,” Indian J. Pediatr., vol. 87, no. April, pp. 281–286, 2020.
  • [13] D. Wang et al., “Clinical Characteristics of 138 Hospitalized Patients with 2019 Novel Coronavirus-Infected Pneumonia in Wuhan, China,” JAMA - J. Am. Med. Assoc., vol. 323, no. 11, pp. 1061–1069, 2020.
  • [14] Ö. Açikgöz and A. Günay, “The early impact of the Covid-19 pandemic on the global and Turkish economy,” Turkish J. Med. Sci., vol. 50, no. SI-1, pp. 520–526, 2020.
  • [15] S. Boccaletti, W. Ditto, G. Mindlin, and A. Atangana, “Modeling and forecasting of epidemic spreading: The case of Covid-19 and beyond,” Chaos, Solitons and Fractals, vol. 135, 2020.
  • [16] W. Guan et al., “Clinical Characteristics of Coronavirus Disease 2019 in China,” N. Engl. J. Med., vol. 382, no. 18, pp. 1708–1720, 2020.
  • [17] T. P. Velavan and C. G. Meyer, “The COVID-19 epidemic,” Trop. Med. Int. Heal., vol. 25, no. 3, pp. 278–280, 2020.
  • [18] C. Huang et al., “Clinical features of patients infected with 2019 novel coronavirus in Wuhan, China,” Lancet, vol. 395, no. 10223, pp. 497–506, 2020.
  • [19] M. L. Holshue et al., “First Case of 2019 Novel Coronavirus in the United States,” N. Engl. J. Med., vol. 382, no. 10, pp. 929–936, 2020.
  • [20] K. Q. Kam et al., “A well infant with coronavirus disease 2019 with high viral load,” Clin. Infect. Dis., vol. 71, no. 15, pp. 847–849, Aug. 2020.
  • [21] S. R. Weiss and J. L. Leibowitz, Coronavirus pathogenesis, 1st ed., vol. 81. Elsevier Inc., 2011.
  • [22] M. L. McGowan, A. H. Norris, and D. Bessett, “Care Churn — Why Keeping Clinic Doors Open Isn’t Enough to Ensure Access to Abortion,” N. Engl. J. Med., vol. 383, no. 6, pp. 508–510, 2020.
  • [23] H. Yao, J. H. Chen, and Y. F. Xu, “Patients with mental health disorders in the COVID-19 epidemic,” The Lancet Psychiatry, vol. 7, no. 4, p. e21, 2020.
  • [24] M. Nicola et al., “The socio-economic implications of the coronavirus pandemic ( COVID-19 ): A review,” Int. J. Surg., vol. 78, no. April, pp. 185–193, 2020.
  • [25] Z. Pala and O. Özkan, “Artificial Intelligence Helps Protect Smart Homes against Thieves,” DÜMF Mühendislik Derg., vol. 11, no. 3, pp. 945–952, 2020.
  • [26] S. S. Mohamed Ali, A. H. Alsaeedi, D. Al-Shammary, H. H. Alsaeedi, and H. W. Abid, “Efficient intelligent system for diagnosis pneumonia (SARSCOVID19) in X-ray images empowered with initial clustering,” Indones. J. Electr. Eng. Comput. Sci., vol. 22, no. 1, pp. 241–251, 2021.
  • [27] Z. Pala and M. Şana, “Attackdet: Combining web data parsing and real-time analysis with machine learning,” J. Adv. Technol. Eng. Res., vol. 6, no. 1, pp. 37–45, 2020.
  • [28] B. Saman, M. M. A. Eid, and M. M. Eid, “Recently employed engineering techniques to reduce the spread of COVID-19 (Corona Virus disease 2019): A review study,” Indones. J. Electr. Eng. Comput. Sci., vol. 22, no. 1, pp. 277–286, 2021.
  • [29] İ. H. Ünlük and Z. Pala, “Prediction of monthly electricity consumption used in Mu ş Alparslan University Complex by means of Classical and Deep Learning methods,” Int. Conf. Data Sci. Mach. Learn. Stat. - 2019, vol. 1, no. 1, pp. 237–239, 2019.
  • [30] Z. Pala, “Using forecastHybrid Package to Ensemble Forecast Functions in the R,” Int. Conf. Data Sci. Mach. Learn. Stat. - 2019, vol. 1, no. 1, pp. 45–47, 2019.
  • [31] E. Yaldız and Z. Pala, “Time Series Analysis of Radiological Data of Outpatients and Inpatients in Emergency Department of Mus State Hospital,” Int. Conf. Data Sci. Mach. Learn. Stat. - 2019, pp. 234–236, 2019.
  • [32] F. Jiang et al., “Artificial intelligence in healthcare: Past, present and future,” Stroke and Vascular Neurology, vol. 2, no. 4. BMJ Publishing Group, pp. 230–243, 01-Dec-2017.
  • [33] S. Chakraborti et al., “Evaluating the plausible application of advanced machine learnings in exploring determinant factors of present pandemic: A case for continent specific COVID-19 analysis,” Sci. Total Environ., vol. 765, p. 142723, 2020.
  • [34] L. Zhong, L. Mu, J. Li, J. Wang, Z. Yin, and D. Liu, “Early Prediction of the 2019 Novel Coronavirus Outbreak in the Mainland China Based on Simple Mathematical Model,” IEEE Access, vol. 8, pp. 51761–51769, 2020.
  • [35] T. Dehesh, H. A. Mardani-Fard, and P. Dehesh, “Forecasting of COVID-19 Confirmed Cases in Different Countries with ARIMA Models,” medRxiv. medRxiv, 18-Mar-2020.
  • [36] M. Maleki, M. R. Mahmoudi, M. H. Heydari, and K. H. Pho, “Modeling and forecasting the spread and death rate of coronavirus (COVID-19) in the world using time series models,” Chaos, Solitons and Fractals, vol. 140, p. 110151, Nov. 2020.
  • [37] A. Zeroual, F. Harrou, A. Dairi, and Y. Sun, “Deep learning methods for forecasting COVID-19 time-Series data: A Comparative study,” Chaos, Solitons and Fractals, vol. 140, p. 110121, Nov. 2020.
  • [38] T. B. Alakus and I. Turkoglu, “Comparison of deep learning approaches to predict COVID-19 infection,” Chaos, Solitons and Fractals, vol. 140, 2020.
  • [39] J. A. Doornik, J. L. Castle, and D. F. Hendry, “Short-term forecasting of the coronavirus pandemic,” Int. J. Forecast., 2020.
  • [40] Z. Pala and A. F. Pala, “Perform Time-series Predictions in the R Development Environment by Combining Statistical-based Models with a Decomposition-based Approach,” J. Muş Alparslan Univ. Fac. Eng. Archit., vol. 1, no. 1, pp. 1–13, 2020.
  • [41] Y. Yang, J. Dong, X. Sun, E. Lima, Q. Mu, and X. Wang, “A CFCC-LSTM Model for Sea Surface Temperature Prediction,” IEEE Geosci. Remote Sens. Lett., vol. 15, no. 2, pp. 207–211, 2018.
  • [42] Y. Lecun, Y. Bengio, and G. Hinton, “Deep learning,” Nature, vol. 521, no. 7553, pp. 436–444, 2015.
  • [43] P. M. Maçaira, A. M. Tavares Thomé, F. L. Cyrino Oliveira, and A. L. Carvalho Ferrer, “Time series analysis with explanatory variables: A systematic literature review,” Environmental Modelling and Software, vol. 107. Elsevier Ltd, pp. 199–209, 01-Sep-2018.
  • [44] Z. Pala, İ. H. Ünlük, and E. Yaldız, “Forecasting of electromagnetic radiation time series: An empirical comparative approach,” Appl. Comput. Electromagn. Soc. J., vol. 34, no. 8, 2019.
  • [45] Z. Pala, “Examining EMF Time Series Using Prediction Algorithms With R,” vol. 44, no. 2, pp. 223–227, 2021.
  • [46] Z. Pala and R. Atici, “Forecasting Sunspot Time Series Using Deep Learning Methods,” Sol. Phys., vol. 294, no. 5, 2019.
  • [47] R. Atıcı and Z. Pala, “Prediction of the Ionospheric foF2 Parameter Using R Language Forecasthybrid Model Library Convenient Time,” Wirel. Pers. Commun., no. doi.org/10.1007/s11277-021-09050-6 Prediction, pp. 1–20, 2021.
  • [48] R. Calegari, F. S. Fogliatto, F. R. Lucini, J. Neyeloff, R. S. Kuchenbecker, and B. D. Schaan, “Forecasting Daily Volume and Acuity of Patients in the Emergency Department,” 2016.
  • [49] R. J. Hyndman and G. Athanasopoulos, Forecasting : Principles and Practice, 2nd editio. Australia: Monash University, 2018.
  • [50] R. J. Hyndman, A. B. Koehler, R. D. Snyder, and S. Grose, “A state space framework for automatic forecasting using exponential smoothing methods,” Int. J. Forecast., vol. 18, no. 3, pp. 439–454, 2002.
  • [51] Rob J. Hyndman and Yeasmin Khandakar, “Automatic Time Series Forecasting: The forecast Package for R,” J. Stat. Softw., vol. 27, no. 3, p. 22, 2008.
  • [52] F. Petropoulos and S. Makridakis, “Forecasting the novel coronavirus COVID-19,” PLoS One, vol. 15, no. 3, pp. 1–8, 2020.
There are 52 citations in total.

Details

Primary Language English
Journal Section Articles
Authors

Zeydin Pala This is me 0000-0002-2642-7788

Ahmet Faruk Pala This is me 0000-0001-5841-1943

Publication Date September 29, 2021
Submission Date August 22, 2021
Published in Issue Year 2021

Cite

IEEE Z. Pala and A. F. Pala, “Comparison of ongoing COVID-19 pandemic confirmed cases/deaths weekly forecasts on continental basis using R statistical models”, DÜMF MD, vol. 12, no. 4, pp. 635–644, 2021, doi: 10.24012/dumf.1002160.
DUJE tarafından yayınlanan tüm makaleler, Creative Commons Atıf 4.0 Uluslararası Lisansı ile lisanslanmıştır. Bu, orijinal eser ve kaynağın uygun şekilde belirtilmesi koşuluyla, herkesin eseri kopyalamasına, yeniden dağıtmasına, yeniden düzenlemesine, iletmesine ve uyarlamasına izin verir. 24456