Research Article
BibTex RIS Cite

Yoğun bakım ünitesindeki sepsis hastalarının klinik sonuçlarını tahmin etmede hangi hemogramdan türetilmiş indeksler yararlı olabilir?

Year 2021, Volume: 46 Issue: 2, 532 - 539, 30.06.2021
https://doi.org/10.17826/cumj.856741

Abstract

Amaç: Çalışmamızın amacı, yoğun bakım ünitesinde (YBÜ) sepsis nedeniyle yatan hastalarda nötrofil lenfosit oranı (NLO), monosit lenfosit oranı (MLO) ve trombosit lenfosit oranı (TLO mortaliteyi belirlemedeki prognostik değerini araştırmaktır.
Gereç ve Yöntem: Çalışmamızda, Şubat 2017 – Nisan 2018 tarihleri arasında YBÜ’de yatan tüm sepsisli hastalar retrospektif olarak değerlendirilmiştir. Hastaların demografik ve klinik özelliklerine ek olarak tam kan sayımı parametreleri kayıt edildi. Mortalite olan ve olmayan sepsisli hastaların başlıca NLO, MLO ve TLO sonuçları olmak üzere demografik ve klinik özellikleri ile diğer laboratuvar sonuçları karşılaştırıldı.
Bulgular: Çalışmaya 411 hasta dahil edildi. Sepsisli hastaların %55,7’si (229/411) öldü ve %44,3 hasta (182/411) sağ taburcu edildi. Mortalite olan grupta yaşayan gruba göre NLO, MLO ve TLO daha yüksek bulundu. Sepsisli hastalarda mortalite tahmini için eşik NLO için ≥9,2, MLO için ≥0,8, TLO için ≥187,3 olarak bulundu. Nötrofil lenfosit oranı için eğri altında kalan alana (AUC) değeri 0,825, MLO için AUC değeri 0,835 ve TLO için AUC değeri 0,720 olarak bulundu.
Sonuç: Yoğun bakım ünitesinde yatan sepsis hastalarında yüksek NLO, MLO ve TLO değerleri mortalite ile ilişkili olup, üç oran arasından mortalite göstergesi için en anlamlı parametre AUC değeri en yüksek olan MLO olarak bulundu.

Supporting Institution

yok

Project Number

yok

Thanks

yok

References

  • Referans 1 Chen CM, Cheng KC, Chan KS, Yu WL. Age May Not Influence the Outcome of Patients with Severe Sepsis in Intensive Care Units. International Journal of Gerontology. 2014; 8: 22-26.
  • Referans 2 Nasa P, Juneja D, Singh O, Dang R, Arora V. Severe sepsis and its impact on outcome in elderly and very elderly patients admitted in intensive care unit. J Intensive Care Med. 2012;27:179-83. 21436163.
  • Referans 3 Martin GS, Mannino DM, Moss M. The effect of age on the development and outcome of adult sepsis. Crit Care Med. 2006;34:15-21.
  • Referans 4 Armstrong BA, Betzold RD, May AK. Sepsis and Septic Shock Strategies. Surg Clin North Am. 2017;97:1339-1379.
  • Referans 5 Geyik FD, Yuce, Y, Erkal KH, Cevik B, Citak N. The use of quick sofa (qsofa) in elderly patients with sepsis in the intensive care unit. Turkish Journal of Geriatrics. 2018;21:143-149.
  • Referans 6 Yu W, Ash AS, Levinsky NG, Moskowitz MA. Intensive care unit use and mortality in the elderly. J Gen Intern Med. 2000;15:97-102.
  • Referans 7 Rowe T, Araujo KLB, Van Ness PH, Pisani MA, Juthani-Mehta M. Outcomes of Older Adults With Sepsis at Admission to an Intensive Care Unit. Open Forum Infectious Diseases. 2016;3: ofw010.
  • Referans 8 Westerdijk K, Simons KS, Zegers M, Wever PC, Pickkers P, de Jager CPC. The value of the neutrophil-lymphocyte count ratio in the diagnosis of sepsis in patients admitted to the Intensive Care Unit: A retrospective cohort study. PLoS One. 2019;14:e0212861.
  • Referans 9 Sarı R, Karakurt Z, Ay M, Çelik ME, Yalaz Tekan Ü, Çiyiltepe F et al. Neutrophil to lymphocyte ratio as a predictor of treatment response and mortality in septic shock patients in the intensive care unit. Turk J Med Sci. 2019;49:1336-1349.
  • Referans 10 Liu Y, Zheng J, Zhang D, Jing L. Neutrophil-lymphocyte ratio and plasma lactate predict 28-day mortality in patients with sepsis. J Clin Lab Anal. 2019;33:e22942.
  • Referans 11 Zhao C, Wei Y, Chen D, Jin J, Chen H. Prognostic value of an inflammatory biomarker-based clinical algorithm in septic patients in the emergency department: An observational study. Int Immunopharmacol. 2020;80:106145.
  • Referans 12 Singer M, Deutschman CS, Seymour CW, Shankar-Hari M, Annane D, Bauer M et al. The Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis-3). JAMA. 2016;315:801-10.
  • Referans 13 Rhodes A, Evans LE, Alhazzani W, Levy MM, Antonelli M, Ferrer R et al. Surviving Sepsis Campaign: International Guidelines for Management of Sepsis and Septic Shock: 2016. Crit Care Med. 2017;45:486-552.
  • Referans 14 Knaus WA, Draper EA, Wagner DP, Zimmerman JE. APACHE II: a severity of disease classification system. Crit Care Med. 1985;13:818-29. Referans 15 Vincent JL, Moreno R, Takala J, Willatts S, De Mendonça A, Bruining H et al. The SOFA (Sepsis-related Organ Failure Assessment) score to describe organ dysfunction/failure. On behalf of the Working Group on Sepsis-Related Problems of the European Society of Intensive Care Medicine. Intensive Care Med. 1996;22:707-10.
  • Referans 16 Akobeng AK. Understanding diagnostic tests 3: Receiver operating characteristic curves. Acta Paediatr. 2007;96:644-7.
  • Referans 17 Sen P, Demirdal T, Nemli SA, Vardar I, Kizilkaya M, Sencan A et al. Infection markers as predictors of Bacteremia in an Intensive Care Unit: A prospective study. Pak J Med Sci. 2018;34:1517-1524.
  • Referans 18 Djordjevic D, Rondovic G, Surbatovic M, Stanojevic I, Udovicic I, Andjelic T et al. Neutrophil-to-Lymphocyte Ratio, Monocyte-to-Lymphocyte Ratio, Platelet-to-Lymphocyte Ratio, and Mean Platelet Volume-to-Platelet Count Ratio as Biomarkers in Critically Ill and Injured Patients: Which Ratio to Choose to Predict Outcome and Nature of Bacteremia? Mediators Inflamm. 2018;2018:3758068.
  • Referans 19 Dilektasli E, Inaba K, Haltmeier T, Wong MD, Clark D, Benjamin ER et al. The prognostic value of neutrophil-to-lymphocyte ratio on mortality in critically ill trauma patients. J Trauma Acute Care Surg. 2016;81:882-888.
  • Referans 20 Gharebaghi N, Valizade Hasanloei MA, Medizadeh Khalifani A, Pakzad S, Lahooti D. Neutrophil-to-lymphocyte ratio in patients with gram-negative sepsis admitted to intensive care unit. Anaesthesiol Intensive Ther. 2019;51:11-16.
  • Referans 21 Ogun MN, Yaksi E. Association Between Neutrophil-to-Lymphocyte Ratio and Mortality in Neurological Intensive Care Unit Patients. J Coll Physicians Surg Pak. 2019;29:919-922.
  • Referans 22 Figiel W, Grąt M, Niewiński G, Patkowski W, Zieniewicz K. Applicability of common inflammatory markers in diagnosing infections in early period after liver transplantation in intensive care setting. Sci Rep. 2020;10:3918.
  • Referans 23 Martins EC, Silveira LDF, Viegas K, Beck AD, Fioravantti Júnior G, Cremonese RV et al. Neutrophil-lymphocyte ratio in the early diagnosis of sepsis in an intensive care unit: a case-control study. Rev Bras Ter Intensiva. 2019;31:64-70.
  • Referans 24 Zhou D, Wu Y, Zhu Y, Lin Z, Yu D, Zhang T. The Prognostic Value of Neutrophil-to-lymphocyte Ratio and Monocyte-to-lymphocyte Ratio in Metastatic Gastric Cancer Treated with Systemic Chemotherapy. J Cancer. 2020;11:4205-4212.
  • Referans 25 Naess A, Nilssen SS, Mo R, Eide GE, Sjursen H. Role of neutrophil to lymphocyte and monocyte to lymphocyte ratios in the diagnosis of bacterial infection in patients with fever. Infection. 2017;45:299-307.
  • Referans 26 Gameiro J, Fonseca JA, Jorge S, Gouveia J, Lopes JA. Neutrophil, lymphocyte and platelet ratio as a predictor of mortality in septic-acute kidney injury patients. Nefrologia. 2020;40:461-468.
  • Referans 27 Yoldas H, Karagoz I, Ogun MN, Velioglu Y, Yildiz I, Bilgi M et al. Novel Mortality Markers for Critically III Patients. J Intensive Care Med. 2020;35:383-385.
  • Referans 28 Kim DH, Oh JH, Kim J, Cho CH, Lee JH. Predictive Values of Neutrophil-to-Lymphocyte Ratio, Platelet-to-Lymphocyte Ratio, and Other Prognostic Factors in Pediatric Patients With Bell's Palsy. Ear Nose Throat J. 2020:145561320922097.
  • Referans 29 Lee S, Oh SY, Kim SH, Lee JH, Kim MC, Kim KH et al. Prognostic significance of neutrophil lymphocyte ratio and platelet lymphocyte ratio in advanced gastric cancer patients treated with FOLFOX chemotherapy. BMC Cancer. 2013;13:350.
  • Referans 30 Karataş MB, İpek G, Onuk T, Güngör B, Durmuş G, Çanga Y et al. Assessment of Prognostic Value of Neutrophil to Lymphocyte Ratio and Platelet to Lymphocyte Ratio in Patients with Pulmonary Embolism. Acta Cardiol Sin. 2016;32:313-20.
  • Referans 31 Ozcan Cetin EH, Cetin MS, Canpolat U, Akdi A, Aras D, Temizhan A et al. Platelet-to-lymphocyte ratio as a novel marker of in-hospital and long-term adverse outcomes among patients with acute pulmonary embolism: A single center large-scale study. Thromb Res. 2017;150:33-40.
  • Referans 32 Zheng CF, Liu WY, Zeng FF, Zheng MH, Shi HY, Zhou Y et al. Prognostic value of platelet-to-lymphocyte ratios among critically ill patients with acute kidney injury. Crit Care. 2017;21:238.
  • Referans 33 Shen Y, Huang X, Zhang W. Platelet-to-lymphocyte ratio as a prognostic predictor of mortality for sepsis: interaction effect with disease severity-a retrospective study. BMJ Open. 2019;9:e022896.
  • Referans 34 Wang Q, Ma J, Jiang Z, Ming L. Prognostic value of neutrophil-to-lymphocyte ratio and platelet-to-lymphocyte ratio in acute pulmonary embolism: a systematic review and meta-analysis. Int Angiol. 2018;37:4-11.
  • Referans 35 Wang Y, Attar BM, Fuentes HE, Jaiswal P, Tafur AJ. Evaluation of the prognostic value of platelet to lymphocyte ratio in patients with hepatocellular carcinoma. J Gastrointest Oncol. 2017;8:1065-1071.
  • Referans 36 Huszno J, Kołosza Z, Mrochem-Kwarciak J, Zajusz A. Prognostic Value of the Neutrophil-Lymphocyte, Platelet-Lymphocyte, and Monocyte-Lymphocyte Ratios in Male Breast Cancer Patients. Oncology. 2020;98:487-492.
  • Referans 37 Giray D, Hallioglu O. Are there any novel markers in acute rheumatic fever: neutrophil-to-lymphocyte ratio, platelet-to-lymphocyte ratio, and monocyte-to-lymphocyte ratio. Cardiol Young. 2020;30:717-721.
  • Referans 38 Lu A, Li H, Zheng Y, Tang M, Li J, Wu H et al. Prognostic Significance of Neutrophil to Lymphocyte Ratio, Lymphocyte to Monocyte Ratio, and Platelet to Lymphocyte Ratio in Patients with Nasopharyngeal Carcinoma. Biomed Res Int. 2017;2017:3047802.

Which hemogram-derived indices might be useful in predicting the clinical outcomes of sepsis patients in the intensive care unit?

Year 2021, Volume: 46 Issue: 2, 532 - 539, 30.06.2021
https://doi.org/10.17826/cumj.856741

Abstract

Purpose: The aim of our study is to investigate the prognostic value of Neutrophil-to-lymphocyte ratio (NLR), monocyte-to-lymphocyte ratio (MLR), and platelet-to-lymphocyte ratio (PLR) in determining mortality in patients hospitalized for intensive care unit (ICU) sepsis.
Materials and Methods: This study retrospectively evaluates all patients hospitalized with sepsis in our ICU between February 2017 and April 2018. In addition to the demographic and clinical characteristics of the patients, complete blood count parameters were also recorded. Demographic and clinical characteristics, mainly NLR, MLR and PLR results, and other laboratory results of patients with sepsis were compared between the ones with and without mortality.
Results: Four hundred and eleven patients were included in the study. 55.7% (229/411) of patients with sepsis died and 44.3% (182/411) were discharged alive. NLR, MLR and PLR were higher in the group with mortality compared to the survivor group. The cut-off value for predicting mortality in patients with sepsis was 9.2 for NLR, ≥0.8 for MLR, and ≥187.3 for PLR. The area under the curve (AUC) value for NLR was 0.825, the AUC value for MLR 0.835 and the AUC value for PLR was 0.720.
Conclusion: High NLR, MLR and PLR values are associated with mortality in sepsis patients hospitalized in ICU, and the most significant parameter for mortality indicator among the three rates was found to be MLR with the highest AUC value.

Project Number

yok

References

  • Referans 1 Chen CM, Cheng KC, Chan KS, Yu WL. Age May Not Influence the Outcome of Patients with Severe Sepsis in Intensive Care Units. International Journal of Gerontology. 2014; 8: 22-26.
  • Referans 2 Nasa P, Juneja D, Singh O, Dang R, Arora V. Severe sepsis and its impact on outcome in elderly and very elderly patients admitted in intensive care unit. J Intensive Care Med. 2012;27:179-83. 21436163.
  • Referans 3 Martin GS, Mannino DM, Moss M. The effect of age on the development and outcome of adult sepsis. Crit Care Med. 2006;34:15-21.
  • Referans 4 Armstrong BA, Betzold RD, May AK. Sepsis and Septic Shock Strategies. Surg Clin North Am. 2017;97:1339-1379.
  • Referans 5 Geyik FD, Yuce, Y, Erkal KH, Cevik B, Citak N. The use of quick sofa (qsofa) in elderly patients with sepsis in the intensive care unit. Turkish Journal of Geriatrics. 2018;21:143-149.
  • Referans 6 Yu W, Ash AS, Levinsky NG, Moskowitz MA. Intensive care unit use and mortality in the elderly. J Gen Intern Med. 2000;15:97-102.
  • Referans 7 Rowe T, Araujo KLB, Van Ness PH, Pisani MA, Juthani-Mehta M. Outcomes of Older Adults With Sepsis at Admission to an Intensive Care Unit. Open Forum Infectious Diseases. 2016;3: ofw010.
  • Referans 8 Westerdijk K, Simons KS, Zegers M, Wever PC, Pickkers P, de Jager CPC. The value of the neutrophil-lymphocyte count ratio in the diagnosis of sepsis in patients admitted to the Intensive Care Unit: A retrospective cohort study. PLoS One. 2019;14:e0212861.
  • Referans 9 Sarı R, Karakurt Z, Ay M, Çelik ME, Yalaz Tekan Ü, Çiyiltepe F et al. Neutrophil to lymphocyte ratio as a predictor of treatment response and mortality in septic shock patients in the intensive care unit. Turk J Med Sci. 2019;49:1336-1349.
  • Referans 10 Liu Y, Zheng J, Zhang D, Jing L. Neutrophil-lymphocyte ratio and plasma lactate predict 28-day mortality in patients with sepsis. J Clin Lab Anal. 2019;33:e22942.
  • Referans 11 Zhao C, Wei Y, Chen D, Jin J, Chen H. Prognostic value of an inflammatory biomarker-based clinical algorithm in septic patients in the emergency department: An observational study. Int Immunopharmacol. 2020;80:106145.
  • Referans 12 Singer M, Deutschman CS, Seymour CW, Shankar-Hari M, Annane D, Bauer M et al. The Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis-3). JAMA. 2016;315:801-10.
  • Referans 13 Rhodes A, Evans LE, Alhazzani W, Levy MM, Antonelli M, Ferrer R et al. Surviving Sepsis Campaign: International Guidelines for Management of Sepsis and Septic Shock: 2016. Crit Care Med. 2017;45:486-552.
  • Referans 14 Knaus WA, Draper EA, Wagner DP, Zimmerman JE. APACHE II: a severity of disease classification system. Crit Care Med. 1985;13:818-29. Referans 15 Vincent JL, Moreno R, Takala J, Willatts S, De Mendonça A, Bruining H et al. The SOFA (Sepsis-related Organ Failure Assessment) score to describe organ dysfunction/failure. On behalf of the Working Group on Sepsis-Related Problems of the European Society of Intensive Care Medicine. Intensive Care Med. 1996;22:707-10.
  • Referans 16 Akobeng AK. Understanding diagnostic tests 3: Receiver operating characteristic curves. Acta Paediatr. 2007;96:644-7.
  • Referans 17 Sen P, Demirdal T, Nemli SA, Vardar I, Kizilkaya M, Sencan A et al. Infection markers as predictors of Bacteremia in an Intensive Care Unit: A prospective study. Pak J Med Sci. 2018;34:1517-1524.
  • Referans 18 Djordjevic D, Rondovic G, Surbatovic M, Stanojevic I, Udovicic I, Andjelic T et al. Neutrophil-to-Lymphocyte Ratio, Monocyte-to-Lymphocyte Ratio, Platelet-to-Lymphocyte Ratio, and Mean Platelet Volume-to-Platelet Count Ratio as Biomarkers in Critically Ill and Injured Patients: Which Ratio to Choose to Predict Outcome and Nature of Bacteremia? Mediators Inflamm. 2018;2018:3758068.
  • Referans 19 Dilektasli E, Inaba K, Haltmeier T, Wong MD, Clark D, Benjamin ER et al. The prognostic value of neutrophil-to-lymphocyte ratio on mortality in critically ill trauma patients. J Trauma Acute Care Surg. 2016;81:882-888.
  • Referans 20 Gharebaghi N, Valizade Hasanloei MA, Medizadeh Khalifani A, Pakzad S, Lahooti D. Neutrophil-to-lymphocyte ratio in patients with gram-negative sepsis admitted to intensive care unit. Anaesthesiol Intensive Ther. 2019;51:11-16.
  • Referans 21 Ogun MN, Yaksi E. Association Between Neutrophil-to-Lymphocyte Ratio and Mortality in Neurological Intensive Care Unit Patients. J Coll Physicians Surg Pak. 2019;29:919-922.
  • Referans 22 Figiel W, Grąt M, Niewiński G, Patkowski W, Zieniewicz K. Applicability of common inflammatory markers in diagnosing infections in early period after liver transplantation in intensive care setting. Sci Rep. 2020;10:3918.
  • Referans 23 Martins EC, Silveira LDF, Viegas K, Beck AD, Fioravantti Júnior G, Cremonese RV et al. Neutrophil-lymphocyte ratio in the early diagnosis of sepsis in an intensive care unit: a case-control study. Rev Bras Ter Intensiva. 2019;31:64-70.
  • Referans 24 Zhou D, Wu Y, Zhu Y, Lin Z, Yu D, Zhang T. The Prognostic Value of Neutrophil-to-lymphocyte Ratio and Monocyte-to-lymphocyte Ratio in Metastatic Gastric Cancer Treated with Systemic Chemotherapy. J Cancer. 2020;11:4205-4212.
  • Referans 25 Naess A, Nilssen SS, Mo R, Eide GE, Sjursen H. Role of neutrophil to lymphocyte and monocyte to lymphocyte ratios in the diagnosis of bacterial infection in patients with fever. Infection. 2017;45:299-307.
  • Referans 26 Gameiro J, Fonseca JA, Jorge S, Gouveia J, Lopes JA. Neutrophil, lymphocyte and platelet ratio as a predictor of mortality in septic-acute kidney injury patients. Nefrologia. 2020;40:461-468.
  • Referans 27 Yoldas H, Karagoz I, Ogun MN, Velioglu Y, Yildiz I, Bilgi M et al. Novel Mortality Markers for Critically III Patients. J Intensive Care Med. 2020;35:383-385.
  • Referans 28 Kim DH, Oh JH, Kim J, Cho CH, Lee JH. Predictive Values of Neutrophil-to-Lymphocyte Ratio, Platelet-to-Lymphocyte Ratio, and Other Prognostic Factors in Pediatric Patients With Bell's Palsy. Ear Nose Throat J. 2020:145561320922097.
  • Referans 29 Lee S, Oh SY, Kim SH, Lee JH, Kim MC, Kim KH et al. Prognostic significance of neutrophil lymphocyte ratio and platelet lymphocyte ratio in advanced gastric cancer patients treated with FOLFOX chemotherapy. BMC Cancer. 2013;13:350.
  • Referans 30 Karataş MB, İpek G, Onuk T, Güngör B, Durmuş G, Çanga Y et al. Assessment of Prognostic Value of Neutrophil to Lymphocyte Ratio and Platelet to Lymphocyte Ratio in Patients with Pulmonary Embolism. Acta Cardiol Sin. 2016;32:313-20.
  • Referans 31 Ozcan Cetin EH, Cetin MS, Canpolat U, Akdi A, Aras D, Temizhan A et al. Platelet-to-lymphocyte ratio as a novel marker of in-hospital and long-term adverse outcomes among patients with acute pulmonary embolism: A single center large-scale study. Thromb Res. 2017;150:33-40.
  • Referans 32 Zheng CF, Liu WY, Zeng FF, Zheng MH, Shi HY, Zhou Y et al. Prognostic value of platelet-to-lymphocyte ratios among critically ill patients with acute kidney injury. Crit Care. 2017;21:238.
  • Referans 33 Shen Y, Huang X, Zhang W. Platelet-to-lymphocyte ratio as a prognostic predictor of mortality for sepsis: interaction effect with disease severity-a retrospective study. BMJ Open. 2019;9:e022896.
  • Referans 34 Wang Q, Ma J, Jiang Z, Ming L. Prognostic value of neutrophil-to-lymphocyte ratio and platelet-to-lymphocyte ratio in acute pulmonary embolism: a systematic review and meta-analysis. Int Angiol. 2018;37:4-11.
  • Referans 35 Wang Y, Attar BM, Fuentes HE, Jaiswal P, Tafur AJ. Evaluation of the prognostic value of platelet to lymphocyte ratio in patients with hepatocellular carcinoma. J Gastrointest Oncol. 2017;8:1065-1071.
  • Referans 36 Huszno J, Kołosza Z, Mrochem-Kwarciak J, Zajusz A. Prognostic Value of the Neutrophil-Lymphocyte, Platelet-Lymphocyte, and Monocyte-Lymphocyte Ratios in Male Breast Cancer Patients. Oncology. 2020;98:487-492.
  • Referans 37 Giray D, Hallioglu O. Are there any novel markers in acute rheumatic fever: neutrophil-to-lymphocyte ratio, platelet-to-lymphocyte ratio, and monocyte-to-lymphocyte ratio. Cardiol Young. 2020;30:717-721.
  • Referans 38 Lu A, Li H, Zheng Y, Tang M, Li J, Wu H et al. Prognostic Significance of Neutrophil to Lymphocyte Ratio, Lymphocyte to Monocyte Ratio, and Platelet to Lymphocyte Ratio in Patients with Nasopharyngeal Carcinoma. Biomed Res Int. 2017;2017:3047802.
There are 37 citations in total.

Details

Primary Language English
Subjects Intensive Care
Journal Section Research
Authors

Esra Çakır 0000-0002-6992-5744

Işıl Özkoçak Turan 0000-0002-0405-0107

Project Number yok
Publication Date June 30, 2021
Acceptance Date February 26, 2021
Published in Issue Year 2021 Volume: 46 Issue: 2

Cite

MLA Çakır, Esra and Işıl Özkoçak Turan. “Which Hemogram-Derived Indices Might Be Useful in Predicting the Clinical Outcomes of Sepsis Patients in the Intensive Care Unit?”. Cukurova Medical Journal, vol. 46, no. 2, 2021, pp. 532-9, doi:10.17826/cumj.856741.

Cited By