Artificial Intelligence Başer Pharyngitis Detection Using Smartphone

Yıl 2021, Cilt: 1 Sayı: 2, 14 - 19, 18.08.2021

Vakkas Doğan , Volkan Kılıç

Öz

Pharyngitis is defined as inflammation in the back wall of the nose and mouth cavity. Mobile technologies have received an increasing amount of attention in the recent global epidemic due to their advantage in pre-diagnosis of diseases that show respiratory symptoms such as pharyngitis. In this study, we propose a custom-designed Android application that offers pharyngitis detection based on artificial intelligence using throat images. Deep learning, a subset of artificial intelligence, allows being embedded in Android applications which leads to be giving fast and highly accurate results without an internet connection. Popular deep learning architectures including Inception-v3, MobileNet-v2, Xception, VGGN6, VGG19 and ResNet50, have been trained to evaluate their performance in pharyngitis detection. Detection of pharyngitis for the images could be performed after they were verified as inner of the mouth. Therefore, two sequential classifiers were designed. The first classifiers were trained with the MSCOCO dataset, while the second-ranked classifiers were trained with the dataset, including 131 pharyngitis and 208 non-pharyngitis throat images augmented with specific methods. Among the above architectures, ResNet50 showed the highest performance with 96.20% accuracy. By embedding the ResNet50 architecture into our custom-designed Android application named 'Farenjit Tanımlama', users will be able to pre-diagnose in a practical way, thus contributing to reducing the burden on the health system caused by the epidemic.

Anahtar Kelimeler

Pharyngitis, Deep learning, Rex Ne150, Android, MSCOCO dataset

Akıllı Telefon Kullanarak Yapay Zeka Tabanlı Faranjit Tespiti

Öz

Burun ve ağız boşluğunun arka duvarında oluşan iltihaplanma farenjit olarak tanımlanmaktadır. Son küresel salgınla birlikte solunum semptomları gösteren farenjit gibi hastalıkların ön teşhisinde mobil teknolojilerin kullanımı gittikçe önem kazanmıştır. Bu çalışmada, geliştirdiğimiz Android tabanlı akıllı telefon uygulamasına gömülü yapay zeka algoritması ile çekilen boğaz görüntülerinden farenjitin tespit edilmesi sağlanmıştır. Yapay zeka yöntemlerinden biri olan derin öğrenmenin Android'e gömülebilmesi ile internet bağlantısı olmaksızın hızlı ve yüksek doğrulukla sonuçlar alınabilmektedir. Popüler derin öfsenme yöntemlerinden Inception-v3, MobileNet-v2, Xception, VGG16, VGG19 ve ResNet50 mimarileri farenjit tespitindeki performanslarını değerlendirmek için eğitilmiştir. Çekilen görüntünün ağız içi veya dışı olduğunun tespitinden sonra ağız içi görüntülerinin farenjit tespiti yapılması gerekmektedir. Bu nedenle sıralı olarak çalışan iki sınıflandırıcı tasarlanmıştır. İlk sınıflandırıcılar MSCOCO veri kümesiyle eğitilirken, ikinci sıradaki sınıflandırıcılar çoğaltma yöntemleri ile genişletilen 131 adet farenjit ve 208 adet farenjit olmayan boğaz görüntüsünün olduğu veri kümesiyle eğitilmiştir. Eğitilen mimariler arasından ResNet50 %96.20 doğrulukla en yüksek performansı göstermiştir. Geliştirdiğimiz 'Farenjit Tanımlama' adlı Android uygulamasına ResNet50 mimarisinin gömülmesiyle kullanıcılar pratik bir şekilde ön teşhis yapabilecek, böylelikle salgımdan kaynaklı sağlık sistemindeki yükün azaltılmasıma katkı sağlanacaktır.

Anahtar Kelimeler

Farenjit, Derin öğrenme, ResNet50, Android, MSCOCO veri kümesi

Kaynakça

• 1. Pappot, N., G. Taarnhøj, and H.J.T.J.H. Pappot, Telemedicine and e-health solutions for COVID-19: patients' perspective [published online April 24, 2020]. Telemed JE Health, DOI:
• http://doi.org/10.1089/tmj.2020.0099.
• 2. Maurrasse, S.E., et al., Telemedicine during the COVID-19 pandemic: a pediatric otolaryngology perspective. Otolaryngology-Head Neck Surgery (SAGE), 2020. 163(3): p. 480-481 DOI:
• https://doi.org/10.1177/0194599820931827.
• 3. Celik Ertugrul, D. and A.H.J.E.J. Ulusoy, Development of a knowledge-based medical expert system to infer supportive treatment suggestions for pediatric patients. ETRI Journal, 2019.41(4): p. 515-527 DOI:
• https://doi.org/10.4218/etrij.2018-0428.
• 4. Van, T.T.. K. Mata, and J.D.J.J.o.c.m. Bard, Automated detection of Streptococcus pyogenes pharyngitis by use of colorex strep A CHROMagar and WASPLab artificial intelligence chromogenic detection module software, Journal of clinical microbiology (JCM) 2019.57(11) DOI: Doğan et al.
• https://doi.org/10.1128/JCM.00811-19.
• 5. Shaikh, N., E. Leonard, and J.M.J.P. Martin, Prevalence of streptococcal pharyngitis and streptococcal carriage in children: a meta-analysis. Pediatrics (APC) 2010. 126(3): p. e557-e564 DOI: DOI:
• https://doi.org/10.1542/peds.2009-2648.
• 6. Rao, A., et al., Diagnosis and antibiotic treatment of group a streptococcal pharyngitis in children in a primary care setting: impact of point-of-care polymerase chain reaction. BMC pediatrics, 2019. 19(1): p. 1-8 DOI:
• https://doi.org/10.1186/s12887-019-1393-y.
• 7. Vazquez, M.N. and J.E.J.P.e.m.p. Sanders, Diagnosis and management of group A streptococcal pharyngitis and associated complications. Pediatric emergency medicine practice (EB Medicine) 2017.14(12): p. 1-20.
• 8. Norton, L.E., et al., Improving guideline-based streptococcal pharyngitis testing: a quality improvement initiative. Pediatrics (APC), 2018. 142(1) DOI: DOI:
• https://doi.org/10.1542/peds.2017-2033.
• Doğan et al.
• https://doi.org/10.1128/JCM.00811-19.
• 5. Shaikh, N., E. Leonard, and J.M.J.P. Martin, Prevalence of streptococcal pharyngitis and streptococcal carriage in children: a meta-analysis. Pediatrics (APC) 2010. 126(3): p. e557-e564 DOI: DOI:
• https://doi.org/10.1542/peds.2009-2648.
• 6. Rao, A., et al., Diagnosis and antibiotic treatment of group a streptococcal pharyngitis in children in a primary care setting: impact of point-of-care polymerase chain reaction. BMC pediatrics, 2019. 19(1): p. 1-8 DOI:
• https://doi.org/10.1186/s12887-019-1393-y.
• 7. Vazquez, M.N. and J.E.J.P.e.m.p. Sanders, Diagnosis and management of group A streptococcal pharyngitis and associated complications. Pediatric emergency medicine practice (EB Medicine) 2017.14(12): p. 1-20.
• 8. Norton, L.E., et al., Improving guideline-based streptococcal pharyngitis testing: a quality improvement initiative. Pediatrics (APC), 2018. 142(1) DOI: DOI: https://doi.org/10.1542/peds.2017-2033.
• 9. Mustafa, Z., M.J.F.i.C. Ghaffari, and I. Microbiology, Diagnostic methods, clinical guidelines, and antibiotic treatment for group A streptococcal pharyngitis: a narrative review. Frontiers in Cellular Infection Microbiology (FCIM)
• 2020. 10: p. 644 DOI:
• https://doi.org/10.3389/fcimb.2020.563627.
• 10.de Haan, K., et al., Automated screening of sickle cells using a smartphone-based microscope and deep learning.
• NPJ Digital Medicine, 2020. 3(1): p. 1-9 DOI:
• https://doi.org/10.1038/s41746-020-0282
• 11.Zulkifley, M.A., et al., Pterygium-Net: a deep learning approach to pterygium detection and localization. Multimedia Tools Applications (MTAP) 2019. 78(24): p.
• 34563-34584 DOI:
• https://doi.org/10.1007/s11042-019-08130-х.
• 12.Askarian, B., S.-C. Yoo, and J.W.J.S. Chong, Novel image processing method for detecting strep throat (streptococcal pharyngitis) using smartphone. Sensors (MDPI) 2019. 19(15): p. 3307 DOI:
• https://doi.org/10.3390/s19153307.
• 13. Yoo, T.K., et al., Toward automated severe pharyngitis detection with smartphone camera using deep learning networks. Computers in Biology Medicine (ELSEVIER) 2020. 125: p. 103980 DOI:
• https://doi.org/10.1016/j.compbiomed.2020.103980.
• 14.Zhang, S., et al., A flexible bifunctional sensor based on porous copper nanowire@ IonGel composite films for high-resolution stress/deformation detection. Journal of Materials Chemistry C 2020. 8(12): p. 4081-4092 DOI:
• https://doi.org/10.1039/C9TC06091J
• 15.MSCOCO veri seti. Available from: https://content.alegion.com/datasets/coco-ms-coco-dataset.
• 16.Pharyngitis Dataset 2020 10-06-2020; Available from: https://data.mendeley.com/datasets/8ynyhnj2kz/1.
• 17.Maeda, H., et al., Road Damage Detection Using Deep Neural Networks with Images Captured Through a Smartphone, arvi preprint. arXiv preprint arxiv: 09454, 2018 DOI:
• https://doi.org/10.1111/mice, 12387.
• 18.Ghiasi, V. and M.J.S.A.S. Koushki, Numerical and artificial neural network analyses of ground surface settlement of tunnel in saturated soil. SN Applied Sciences, 2020.2(5): p. 1-14 DOI:
• https://doi.org/10.1007/s42452-020-2742-z.
• 19.Minaee, S., et al., Deep-covid: Predicting covid-19 from chest x-ray images using deep transfer learning. Medical image analysis (MIA), 2020. 65: p. 101794 DOI: https://doi.org/10.1016/j.media.2020.101794.
• 20. Yoo, T.K., et al., The possibility of the combination of OCT and fundus images for improving the diagnostic accuracy of deep learning for age-related macular degeneration: a preliminary experiment. Medical biological engineering computing (MBEC) 2019. 57(3): p. 677-687 DOI:
• https://doi.org/10.1007/s11517-018-1915-z.
• 21.Gómez-Flores, W., W.C.J.C.i.B. de Albuquerque Pereira, and Medicine, A comparative study of pre-trained convolutional neural networks for semantic segmentation of breast tumors in ultrasound. Computers in Biology Medicine (ELSEVIER), 2020. 126: p. 104036 DOI:
• https://doi.org/10.1016/j.compbiomed.2020.104036.
• 22. Kassani, S.H., et al. Diabetic retinopathy classification using a modified xception architecture, in 2019 IFEE International Symposium on Signal Processing and Information Technology (ISSPIT). 2019. IEEE DOL 10.1109/ISSPIT47144.2019.9001846. 23. Malsagov, M.Y., et al., Exponential discretization of
• weights of neural network connections in pre-trained neural
• networks. Optical Memory Neural Networks (OMNN),
• 2019.28(4): p. 262-270 DOI
• https dor.org 11 3103/$1060992X19040106.
• 24. Rounds. J. et al. Probing for Artifacts: Detecting
• Imagenet Model Evasions, in Proceedings of the IEEE CVE
• Conference on Computer Visina and Patern Recognition
• Workshops 2020
• 25.Darwin, I.F., Android Cookbook: Problems and Solutions for Android Developers. 2017: " O'Reilly Media, Inc.".
• 26.Sproull, T., D. Shook, and B. Siever. Machine Learning on the Move: Teaching ML. Kit for Firebase in a Mobile Apps Course. in Proceedings of the 52nd ACM Technical Symposium on Computer Science Education. 2021. DOI:
• https://doi.org/10.1145/3408877.3432496.
• 27. Pienaar, J., Mlir in tensorflow ecosystem. 2020.
• 28. Ribeiro, E., et al., Exploring deep learning and transferlearning for colonic polyp classification. Computational mathematical methods in medicine
• (CMMM) 2016. 2016 DOI:
• https://doi.org/10.1155/2016/6584725