Machine LearningDeep Learning in Rheumatological Screening A Systematic Review

Year 2023, Volume: 16 Issue: 3, 940 - 969, 31.12.2023

Zehra Aysun Altıkardeş , Emre Canayaz , Alparslan Ünsal

https://doi.org/10.18185/erzifbed.1211547

Abstract

Machine learning and deep learning techniques have been used in many fields, especially automatic image processing techniques, in recent years. In light of these developments, it has become inevitable to develop applications in the medical field. This study focuses on the past few years of research using machine learning and deep learning methods in the context of image processing in the field of rheumatology. This review provides researchers with the latest information on the use of deep learning and machine learning and inspires them to generate new ideas in their research by analyzing image processing systems performed by these artificial intelligence methods. In the proposed systematic review, 28 articles covering the application of deep learning and machine learning methods in the domain of rheumatology with the aim of digital image processing in the last 18 years were evaluated. Experiments emphasize that machine learning and deep learning methods provide significant segmentation accuracy and better case classification accuracy for various rheumatologic diseases like rheumatoid arthritis, osteoarthritis, and ankylosing spondylitis. Lastly submitted review presents possible different research ideas for related researchers to concentrate on for their future studies.

Keywords

Artificial Intelligence, Machine Learning, Deep Learning, Rheumatology

Supporting Institution

-

Project Number

-

Thanks

-

References

• Aizenberg, E., Roex, E. A., Nieuwenhuis, W. P., Mangnus, L., van der Helm‐van Mil, A. H., Reijnierse, M., . . . Stoel, B. C. (2018). Automatic quantification of bone marrow edema on MRI of the wrist in patients with early arthritis: a feasibility study. Magnetic resonance in medicine, 79(2), 1127-1134.
• Algan, G., & Ulusoy, I. (2021). Image classification with deep learning in the presence of noisy labels: A survey. Knowledge-Based Systems, 215, 106771.
• Antony, J., McGuinness, K., O'Connor, N. E., & Moran, K. (2016). Quantifying radiographic knee osteoarthritis severity using deep convolutional neural networks. Paper presented at the 2016 23rd International Conference on Pattern Recognition (ICPR).
• Ashinsky, B. G., Bouhrara, M., Coletta, C. E., Lehallier, B., Urish, K. L., Lin, P. C., . . . Spencer, R. G. (2017). Predicting early symptomatic osteoarthritis in the human knee using machine learning classification of magnetic resonance images from the osteoarthritis initiative. Journal of Orthopaedic Research, 35(10), 2243-2250.
• Avramidis, G. P., Avramidou, M. P., & Papakostas, G. A. (2022). Rheumatoid Arthritis Diagnosis: Deep Learning vs. Humane. Applied Sciences, 12(1), 10.
• Becker, A. (2019). Artificial intelligence in medicine: What is it doing for us today? Health Policy and Technology, 8(2), 198-205.
• Bengio, Y., & LeCun, Y. (2007). Scaling learning algorithms towards AI. Large-scale kernel machines, 34(5), 1-41.
• Bidgood Jr, W. D., Horii, S. C., Prior, F. W., & Van Syckle, D. E. (1997). Understanding and using DICOM, the data interchange standard for biomedical imaging. Journal of the American Medical Informatics Association, 4(3), 199-212.
• Boonen, A., Brinkhuizen, T., Landewé, R., van der Heijde, D., & Severens, J. L. (2010). Impact of ankylosing spondylitis on sick leave, presenteeism and unpaid productivity, and estimation of the societal cost. Annals of the Rheumatic Diseases, 69(6), 1123-1128. doi:10.1136/ard.2009.116764
• Boonen, A., & Mau, W. (2009). The economic burden of disease: comparison between rheumatoid arthritis and ankylosing spondylitis. Clinical and experimental rheumatology, 27(4 Suppl 55), S112-117.
• Brahim, A., Jennane, R., Riad, R., Janvier, T., Khedher, L., Toumi, H., & Lespessailles, E. (2019). A decision support tool for early detection of knee OsteoArthritis using X-ray imaging and machine learning: Data from the OsteoArthritis Initiative. Computerized Medical Imaging and Graphics, 73, 11-18.
• Bressem, K. K., Vahldiek, J. L., Adams, L. C., Niehues, S. M., Haibel, H., Rodriguez, V. R., . . . Poddubnyy, D. (2020). Detecting radiographic sacroiliitis using deep learning with expert-level accuracy in axial spondyloarthritis. medRxiv, 2020.2005.2019.20105304. doi:10.1101/2020.05.19.20105304
• Brui, E., Efimtcev, A. Y., Fokin, V. A., Fernandez, R., Levchuk, A. G., Ogier, A. C., . . . Bendahan, D. (2020). Deep learning‐based fully automatic segmentation of wrist cartilage in MR images. NMR in Biomedicine, 33(8), e4320.
• Carano, R. A., Lynch, J. A., Redei, J., Ostrowitzki, S., Miaux, Y., Zaim, S., . . . Genant, H. K. (2004). Multispectral analysis of bone lesions in the hands of patients with rheumatoid arthritis. Magnetic resonance imaging, 22(4), 505-514.
• Castro-Zunti, R., Park, E. H., Choi, Y., Jin, G. Y., & Ko, S.-b. (2020). Early detection of ankylosing spondylitis using texture features and statistical machine learning, and deep learning, with some patient age analysis. Computerized Medical Imaging and Graphics, 82, 101718. doi:https://doi.org/10.1016/j.compmedimag.2020.101718
• Chaturvedi, N. (2021). Deepra: Predicting joint damage from radiographs using cnn with attention. arXiv preprint arXiv:2102.06982.
• Deng, L., Li, J., Huang, J.-T., Yao, K., Yu, D., Seide, F., . . . Williams, J. (2013). Recent advances in deep learning for speech research at Microsoft. Paper presented at the 2013 IEEE international conference on acoustics, speech and signal processing.
• Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018). Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805.
• Esses, S. J., Lu, X., Zhao, T., Shanbhogue, K., Dane, B., Bruno, M., & Chandarana, H. (2018). Automated image quality evaluation of T2‐weighted liver MRI utilizing deep learning architecture. Journal of Magnetic Resonance Imaging, 47(3), 723-728.
• Faleiros, M. C., Nogueira-Barbosa, M. H., Dalto, V. F., Júnior, J. R. F., Tenório, A. P. M., Luppino-Assad, R., . . . de Azevedo-Marques, P. M. (2020). Machine learning techniques for computer-aided classification of active inflammatory sacroiliitis in magnetic resonance imaging. Advances in Rheumatology, 60, 1-10.
• Fiorentino, M. C., Moccia, S., Cipolletta, E., Filippucci, E., & Frontoni, E. (2019). A learning approach for informative-frame selection in US rheumatology images. Paper presented at the International Conference on image analysis and processing.
• Golkov, V., Dosovitskiy, A., Sperl, J. I., Menzel, M. I., Czisch, M., Sämann, P., . . . Cremers, D. (2016). Q-space deep learning: twelve-fold shorter and model-free diffusion MRI scans. IEEE transactions on medical imaging, 35(5), 1344-1351.
• Hamet, P., & Tremblay, J. (2017). Artificial intelligence in medicine. Metabolism, 69, S36-S40.
• Hammernik, K., Klatzer, T., Kobler, E., Recht, M. P., Sodickson, D. K., Pock, T., & Knoll, F. (2018). Learning a variational network for reconstruction of accelerated MRI data. Magnetic resonance in medicine, 79(6), 3055-3071.
• Han, Q., Lu, Y., Han, J., Luo, A., Huang, L., Ding, J., . . . Liang, Q. (2021). Automatic quantification and grading of hip bone marrow oedema in ankylosing spondylitis based on deep learning. Modern Rheumatology.
• Hemalatha, R., Vijaybaskar, V., & Thamizhvani, T. (2019). Automatic localization of anatomical regions in medical ultrasound images of rheumatoid arthritis using deep learning. Proceedings of the Institution of Mechanical Engineers, Part H: Journal of Engineering in Medicine, 233(6), 657-667.
• Hirano, T., Nishide, M., Nonaka, N., Seita, J., Ebina, K., Sakurada, K., & Kumanogoh, A. (2019). Development and validation of a deep-learning model for scoring of radiographic finger joint destruction in rheumatoid arthritis. Rheumatology advances in practice, 3(2), rkz047.
• Hirvasniemi, J., Gielis, W. P., Arbabi, S., Agricola, R., van Spil, W. E., Arbabi, V., & Weinans, H. (2019). Bone texture analysis for prediction of incident radiographic hip osteoarthritis using machine learning: data from the Cohort Hip and Cohort Knee (CHECK) study. Osteoarthritis and cartilage, 27(6), 906-914.
• Kansagra, A. P., John-Paul, J. Y., Chatterjee, A. R., Lenchik, L., Chow, D. S., Prater, A. B., . . . Heilbrun, M. E. (2016). Big data and the future of radiology informatics. Academic radiology, 23(1), 30-42.
• Kayalibay, B., Jensen, G., & van der Smagt, P. (2017). CNN-based segmentation of medical imaging data. arXiv preprint arXiv:1701.03056.
• Khan, M. A. (2002). Thoughts concerning the early diagnosis of ankylosing spondylitis and related diseases. Clinical and experimental rheumatology, 20(6 Suppl 28), S6-10. Retrieved from http://europepmc.org/abstract/MED/12463439
• Kim, K.-J., & Tagkopoulos, I. (2019). Application of machine learning in rheumatic disease research. The Korean journal of internal medicine, 34(4), 708-722. doi:10.3904/kjim.2018.349
• Knight, W. (2017). The dark secret at the heart of AI'11 April 2017. In: MIT Technology Review https://www. technologyreview. com/s/604087/the-dark ….
• Kowsari, K., Brown, D. E., Heidarysafa, M., Meimandi, K. J., Gerber, M. S., & Barnes, L. E. (2017). Hdltex: Hierarchical deep learning for text classification. Paper presented at the 2017 16th IEEE international conference on machine learning and applications (ICMLA).
• Kräter, M., Abuhattum, S., Soteriou, D., Jacobi, A., Krüger, T., Guck, J., & Herbig, M. (2021). AIDeveloper: deep learning image classification in life science and beyond. Advanced science, 8(11), 2003743.
• KS, A. S. D. M. D., Selvakumar, M., Sathyamangalam, E., & Nadu, T. (2023). CLASSIFICATION OF DEEP LEARNING ALGORITHM FOR RHEUMATOID ARTHRITIS PREDICTOR.
• Kushner, D. C., & Lucey, L. L. (2005). Diagnostic radiology reporting and communication: the ACR guideline. Journal of the American College of Radiology, 2(1), 15-21.
• Lakhani, P., Prater, A. B., Hutson, R. K., Andriole, K. P., Dreyer, K. J., Morey, J., . . . Itri, J. N. (2018). Machine learning in radiology: applications beyond image interpretation. Journal of the American College of Radiology, 15(2), 350-359.
• LeCun, Y., & Bengio, Y. (1995). Convolutional networks for images, speech, and time series. The handbook of brain theory and neural networks, 3361(10), 1995.
• Lee, H., Tajmir, S., Lee, J., Zissen, M., Yeshiwas, B. A., Alkasab, T. K., . . . Do, S. (2017). Fully automated deep learning system for bone age assessment. Journal of digital imaging, 30(4), 427-441.
• Liu, F., Zhou, Z., Samsonov, A., Blankenbaker, D., Larison, W., Kanarek, A., . . . Kijowski, R. (2018). Deep learning approach for evaluating knee MR images: achieving high diagnostic performance for cartilage lesion detection. Radiology, 289(1), 160-169.
• Liu, J., Chang, W.-C., Wu, Y., & Yang, Y. (2017). Deep learning for extreme multi-label text classification. Paper presented at the Proceedings of the 40th international ACM SIGIR conference on research and development in information retrieval.
• Majumder, N., Poria, S., Gelbukh, A., & Cambria, E. (2017). Deep learning-based document modeling for personality detection from text. IEEE Intelligent Systems, 32(2), 74-79.
• Maziarz, K., Krason, A., & Wojna, Z. (2021). Deep Learning for Rheumatoid Arthritis: Joint Detection and Damage Scoring in X-rays. arXiv preprint arXiv:2104.13915.
• Mennini, F. S., Viti, R., Marcellusi, A., Sciattella, P., Viapiana, O., & Rossini, M. (2018). Economic evaluation of spondyloarthritis: economic impact of diagnostic delay in Italy. Clinicoecon Outcomes Res, 10, 45-51. doi:10.2147/ceor.S144209
• Murakami, S., Hatano, K., Tan, J., Kim, H., & Aoki, T. (2018). Automatic identification of bone erosions in rheumatoid arthritis from hand radiographs based on deep convolutional neural network. Multimedia tools and applications, 77(9), 10921-10937.
• Nanni, L., Costa, Y. M., Aguiar, R. L., Silla Jr, C. N., & Brahnam, S. (2018). Ensemble of deep learning, visual and acoustic features for music genre classification. Journal of New Music Research, 47(4), 383-397.
• Nathan M. Cross, M., MS, CIIP, University of Pennsylvania; Jason DeBerry, MD; Daniel Ortiz, MD; Justine Kemp, MD; José Morey, MD. (2017). Diagnostic Quality of Machine Learning Algorithm for Optimization of Low-Dose Computed Tomography Data Paper presented at the SIIM (Society for Imaging Informatics in Medicine ) 2017 Pittsburgi PA.
• https://cdn.ymaws.com/siim.org/resource/resmgr/siim2017/abstracts/posters-Cross.pdf Norman, B., Pedoia, V., Noworolski, A., Link, T. M., & Majumdar, S. (2019). Applying densely connected convolutional neural networks for staging osteoarthritis severity from plain radiographs. Journal of digital imaging, 32(3), 471-477.
• Orange, D. E., Agius, P., DiCarlo, E. F., Robine, N., Geiger, H., Szymonifka, J., . . . Donlin, L. T. (2018). Identification of Three Rheumatoid Arthritis Disease Subtypes by Machine Learning Integration of Synovial Histologic Features and RNA Sequencing Data. Arthritis & Rheumatology, 70(5), 690-701. doi:https://doi.org/10.1002/art.40428
• Page, M. J., McKenzie, J. E., Bossuyt, P. M., Boutron, I., Hoffmann, T. C., Mulrow, C. D., . . . Brennan, S. E. (2021). The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. Systematic reviews, 10(1), 1-11.
• Pahade, J., Couto, C., Davis, R., Patel, P., Siewert, B., & Rosen, M. (2012). Reviewing imaging examination results immediately after study completion with a radiologist: Patient preferences and assessment of feasibility in an academic department. AJR. American journal of roentgenology, 199(4), 844.
• Pedoia, V., Norman, B., Mehany, S. N., Bucknor, M. D., Link, T. M., & Majumdar, S. (2019). 3D convolutional neural networks for detection and severity staging of meniscus and PFJ cartilage morphological degenerative changes in osteoarthritis and anterior cruciate ligament subjects. Journal of Magnetic Resonance Imaging, 49(2), 400-410.
• Poddubnyy, D., Sieper, J., Akar, S., Muñoz-Fernández, S., Haibel, H., Hojnik, M., . . . Inman, R. D. (2021). Characteristics of patients with axial spondyloarthritis by geographic regions: PROOF multicountry observational study baseline results. Rheumatology. doi:10.1093/rheumatology/keab901
• Prasoon, A., Petersen, K., Igel, C., Lauze, F., Dam, E., & Nielsen, M. (2013). Deep feature learning for knee cartilage segmentation using a triplanar convolutional neural network. Paper presented at the International conference on medical image computing and computer-assisted intervention.
• Ribas, L. C., Riad, R., Jennane, R., & Bruno, O. M. (2022). A complex network based approach for knee Osteoarthritis detection: Data from the Osteoarthritis initiative. Biomedical Signal Processing and Control, 71, 103133. doi:https://doi.org/10.1016/j.bspc.2021.103133
• Ronneberger, O., Fischer, P., & Brox, T. (2015). U-net: Convolutional networks for biomedical image segmentation. Paper presented at the International Conference on Medical image computing and computer-assisted intervention.
• Rothenberg, S., Patel, J., & Herscu, M. (2016). Evaluation of a machine learning approach to protocol MRI examinations: initial experience predicting use of contrast by neuroradiologists in MRI protocols. Paper presented at the Radiology Society of North America, 102nd Scientific Assembly and Annual Meeting.
• Samuel, A. L. (2000). Some studies in machine learning using the game of checkers. IBM Journal of research and development, 44(1.2), 206-226.
• Schaefer, G., Krawczyk, B., Doshi, N. P., & Merla, A. (2013). Scleroderma capillary pattern identification using texture descriptors and ensemble classification. Paper presented at the 2013 35th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).
• Segen, J., Kulbacki, M., & Wereszczyński, K. (2015). Registration of ultrasound images for automated assessment of synovitis activity. Paper presented at the Asian Conference on Intelligent Information and Database Systems.
• Shenkman, Y., Qutteineh, B., Joskowicz, L., Szeskin, A., Yusef, A., Mayer, A., & Eshed, I. (2019). Automatic detection and diagnosis of sacroiliitis in CT scans as incidental findings. Medical Image Analysis, 57, 165-175. doi:https://doi.org/10.1016/j.media.2019.07.007
• Subhash, M. G., & Kureshi, A. K. (2023). An Efficient CNN for Hand X-Ray Classification of Rheumatoid Arthritis. Microprocessors and Microsystems, 104822. doi:https://doi.org/10.1016/j.micpro.2023.104822
• Thrall, J. H., Li, X., Li, Q., Cruz, C., Do, S., Dreyer, K., & Brink, J. (2018). Artificial Intelligence and Machine Learning in Radiology: Opportunities, Challenges, Pitfalls, and Criteria for Success. Journal of the American College of Radiology, 15(3, Part B), 504-508. doi:https://doi.org/10.1016/j.jacr.2017.12.026
• Tiulpin, A., Thevenot, J., Rahtu, E., Lehenkari, P., & Saarakkala, S. (2018). Automatic Knee Osteoarthritis Diagnosis from Plain Radiographs: A Deep Learning-Based Approach. Scientific Reports, 8(1), 1727. doi:10.1038/s41598-018-20132-7
• Tripoliti, E. E., Fotiadis, D. I., & Argyropoulou, M. (2007). Automated segmentation and quantification of inflammatory tissue of the hand in rheumatoid arthritis patients using magnetic resonance imaging data. Artificial intelligence in medicine, 40(2), 65-85.
• Ward, M. M. (2002). Functional disability predicts total costs in patients with ankylosing spondylitis. Arthritis Rheum, 46(1), 223-231. doi:10.1002/1529-0131(200201)46:1<223::Aid-art498>3.0.Co;2-#
• Wesseling, J., Dekker, J., van den Berg, W. B., Bierma-Zeinstra, S. M. A., Boers, M., Cats, H. A., . . . Bijlsma, J. W. J. (2009). CHECK (Cohort Hip and Cohort Knee): similarities and differences with the Osteoarthritis Initiative. Annals of the Rheumatic Diseases, 68(9), 1413-1419. doi:10.1136/ard.2008.096164
• Wu, M., Wu, H., Wu, L., Cui, C., Shi, S., Xu, J., . . . Dong, F. (2022). A deep learning classification of metacarpophalangeal joints synovial proliferation in rheumatoid arthritis by ultrasound images. J Clin Ultrasound, 50(2), 296-301. doi:10.1002/jcu.23143
• Xue, Y., Zhang, R., Deng, Y., Chen, K., & Jiang, T. (2017). A preliminary examination of the diagnostic value of deep learning in hip osteoarthritis. PloS one, 12(6), e0178992.
• Zhang, W., Yang, G., Lin, Y., Ji, C., & Gupta, M. M. (2018). On definition of deep learning. Paper presented at the 2018 World automation congress (WAC).
• Zharkova, V. (2007). Artificial intelligence in recognition and classification of astrophysical and medical images (Vol. 46): Springer Science & Business Media.