Improvement proposals for lesion segmentation on dermoscopic images

Year 2025, Volume: 40 Issue: 1, 251 - 264, 16.08.2024

Saadet Aytaç Arpacı , Songül Varlı

https://doi.org/10.17341/gazimmfd.1335533

Abstract

The U-Net architecture, which performs segmentation on images, has also achieved very successful results in the medical field. However, there is also a need to improve the U-Net architecture for better results. In this article, some improvement proposals are presented for the U-Net model's encoder part, and the segmentation success of the implemented architecture for segmentation of dermoscopic image lesions is evaluated. The PH2 dataset and the "International Skin Imaging Collaboration" datasets (ISIC-2016 and ISIC-2017) were used for the research. The traditional data augmentation method was applied to the selected PH2 dataset samples. The results of the proposed model (EnecaU-Net) and the U-Net model obtained with the PH2 dataset were compared. Furthermore, in this article, the mix data augmentation method, which has an influence on the model's segmentation success, is examined for lesion segmentation on dermoscopic images. This investigation was made with the ISIC-2016 dataset, and its experimental results were compared with the same amount of the ISIC-2017 dataset that didn't apply data augmentation operations. Although, during the evaluation phase, Dice and Jaccard (IoU) metrics were used primarily to measure the success of the model, specificity, sensitivity, and accuracy criteria were also used. According to our results, the segmentation success of the EnecaU-Net model applied for lesion segmentation on dermoscopic images is high, and the applied mix data augmentation method improves the segmentation success of the EnecaU-Net model. The average test results achieved by the proposed model are 88.05% and 80.30% for ISIC-2016 and 83.09% and 74.54% for ISIC-2017 in terms of the Dice and Jaccard values, respectively.

Keywords

Convolutional neural network, mix data augmentation, segmentation, dermoscopic lesion, U-Net

Dermoskopik görüntülerde lezyon bölütleme için iyileştirme önerileri

Abstract

Görüntülerde bölütleme görevini gerçekleştiren U-Net mimarisi tıp alanında da oldukça başarılı sonuçlar elde etmiştir. Fakat daha iyi sonuçlar için mimarinin iyileştirilmesine de gerek vardır. Bu makalede, U-Net modelinin kodlayıcı bölümü için bazı iyileştirme önerileri sunulmaktadır ve uygulanan mimarinin dermoskopik görüntülerde lezyon bölütleme işlemi için bölütleme başarısı değerlendirilmektedir. Araştırma, PH2 ve “Uluslararası Cilt Görüntüleme İşbirliği” (ISIC-2016 ve ISIC-2017) veri kümeleri ile yapılmıştır. Geleneksel veri artırma yöntemi, seçilen PH2 veri kümesi örneklerine uygulanmıştır. Önerilen model (EnecaU-Net) ve U-Net modelinin PH2 veri kümesi ile elde edilen sonuçları karşılaştırılmıştır. Ayrıca bu makalede dermoskopik görüntülerde lezyon bölütleme işlemi için modelin bölütleme başarısı üzerinde etki gösteren karıştırma veri artırma yöntemi de incelenmiştir. Bu inceleme, ISIC-2016 veri kümesi ile yapılmıştır ve veri artırma işlemi uygulanmayan aynı miktardaki ISIC-2017 veri kümesi ile yapılan inceleme sonuçları karşılaştırılmıştır. Değerlendirme aşamasında model başarısının ölçümü için öncelikle Dice ve Jaccard (IoU) ölçütlerini kullanmış olmakla beraber özgüllük, duyarlılık ve doğruluk ölçütlerinden de faydalanılmıştır. Elde ettiğimiz sonuçlara göre, dermoskopik görüntülerde lezyon bölütleme işlemi için uygulanan EnecaU-Net modelinin bölütleme başarısı yüksektir ve uygulanan karıştırma veri artırma yöntemi, EnecaU-Net modelinin bölütleme başarısını iyileştirmektedir. Önerilen modelin ulaştığı ortalama test sonuçları, Dice ve Jaccard değerleri açısından sırasıyla ISIC-2016 için %88,05 ve %80,30 ve ISIC-2017 için %83,09 ve %74,54’tür.

Keywords

Evrişimsel sinir ağı, karıştırma veri artırma, bölütleme, dermoskopik lezyon, U-Net

References

• 1. Ronneberger O., Fischer P., Brox T., U-Net: Convolutional networks for biomedical image segmentation, Medical Image Computing and Computer-Assisted Intervention, Munich-Germany, 234-241, 2015.
• 2. Wang Q., Wu B., Zhu P., Li P., Zuo W., Hu Q., ECA-Net: Efficient channel attention for deep convolutional neural networks, IEEE/CVF 2020 Conference on Computer Vision and Pattern Recognition (CVPR), Seattle- WA-USA, 11531-11539, 2020.
• 3. Yu J., Yang D., Zhao H., FFANet: Feature fusion attention network to medical image segmentation, Biomedical Signal Processing and Control, 69, 102912, 2021.
• 4. Zhang Z., Jiang Y., Qiao H., Wang M., Yan W., Chen J., SIL-Net: A Semi-Isotropic L-shaped network for dermoscopic image segmentation, Computers in Biology and Medicine, 150, 106146, 2022.
• 5. Zhang J., Pan W., Wang B., Chen Q., Cheng Y., Multi-scale aggregation networks with flexible receptive fields for melanoma segmentation, Biomedical Signal Processing and Control, 78, 103950, 2022.
• 6. Guo Z., Zhao L., Yuan J., Yu H., MSANet: Multiscale aggregation network integrating spatial and channel information for lung nodule detection, IEEE Journal of Biomedical and Health Informatics, 26 (6), 2547-2558, 2022.
• 7. Shu X., Gu Y., Zhang X., Hu C., Cheng K., FCRB U-Net: A novel fully connected residual block U-Net for fetal cerebellum ultrasound image segmentation, Computers in Biology and Medicine, 148,105693, 2022.
• 8. Li L., Verma M., Nakashima Y., Nagahara H., Kawasaki R., IterNet: Retinal image segmentation utilizing structural redundancy in vessel networks, In: 2020 IEEE Winter Conference on Applications of Computer Vision (WACV), Snowmass- CO- USA, 3645-3654, 2020.
• 9. Xu L., Tetteh G., Mustafa M., Lipkova J., Zhao Y., Bieth M., Christ P., Piraud M., Menze B., Shi K., W-Net for whole-body bone lesion detection on 68Ga-Pentixafor PET/CT imaging of multiple myeloma patients, 5th International Workshop on Computational Methods for Molecular Imaging (CMMI), Québec-Canada, 23-30, 2017.
• 10. Chen W., Zhang Y., He J., Qiao Y., Chen Y., Shi H., Wu E.X., Tang X., Prostate segmentation using 2D bridged U-net, 2019 International Joint Conference on Neural Networks (IJCNN), Budapest- Hungary, 1-7, 2019.
• 11. Das S., Deka A., Iwahori Y., Bhuyan M.K., Iwamoto T., Ueda J., Contour-aware residual W-Net for nuclei segmentation, 23rd International Conference on Knowledge-Based and Intelligent Information & Engineering Systems, Budapest-Hungary, 1479–1488, 2019.
• 12. Singh K.R., Sharma A., Singh G.K., W-Net: Novel deep supervision for deep learning-based cardiac magnetic resonance imaging segmentation, IETE Journal of Research, 1-18, 2022.
• 13. Mohan S., Bhattacharya S., Ghosh S., Attention W-Net: Improved skip connections for better representations, 26th International Conference on Pattern Recognition (ICPR), Montréal Québec- Canada, 1-6, 2022.
• 14. Khanna A., Londhe N.D., Gupta S., Semwal A., A deep residual U-Net convolutional neural network for automated lung segmentation in computed tomography images, Biocybernetics and Biomedical Engineering, 40 (3), 1314-1327, 2020.
• 15. Arpacı S.A., Varlı S., EncU-Net: A modified U-Net for dermoscopic image segmentation, 29th Signal Processing and Communications Applications Conference (SIU), Istanbul-Turkey, 1-4, 2021.
• 16. Arpacı S.A., Varlı S., LUPU-Net: A new improvement proposal for encoder-decoder architecture, International Advanced Researches and Engineering Journal, 5 (3), 352-361, 2021.
• 17. Ünlü E.I., Çınar A., Segmentation of benign and malign lesions on skin images using U-Net, 2021 International Conference on Innovation and Intelligence for Informatics, Computing, and Technologies (3ICT), Zallaq-Bahrain, 165-169, 2021.
• 18. Khouloud S., Ahlem M., Fadel T., Amel S., W-net and inception residual network for skin lesion segmentation and classification, Applied Intelligence, 52, 3976–3994, 2022.
• 19. Zou P., Wu J.S., SwinE-UNet3+: swin transformer encoder network for medical image segmentation, Progress in Artificial Intelligence, 12, 99–105, 2023.
• 20. Kaur R., Ranade S.K., Improving accuracy of convolutional neural network-based skin lesion segmentation using group normalization and combined loss function, International Journal of Information Technology, 15, 2827–2835, 2023.
• 21. He K., Zhang X., Ren S., Sun J., Deep residual learning for image recognition, IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas- USA, 770-778, 2016.
• 22. Zafar K., Gilani S.O., Waris A., Ahmed A., Jamil M., Khan M.N., Sohail Kashif A., Skin lesion segmentation from dermoscopic images using convolutional neural network, Sensors, 20 (6),1601, 2020.
• 23. Iranpoor R., Mahboob A.S., Shahbandegan S., Baniasadi N., Skin lesion segmentation using convolutional neural networks with improved U-Net architecture, 2020 6th Iranian Conference on Signal Processing and Intelligent Systems (ICSPIS), Mashhad-Iran, 1-5, 2020.
• 24. Badshah N., Ahmad A., ResBCU-Net: Deep learning approach for segmentation of skin images, Biomedical Signal Processing and Control, 71, 103137, 2022.
• 25. Kumar A., Hamarneh G., Drew M.S., Illumination-based transformations improve skin lesion segmentation in dermoscopic images, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Seattle- WA-USA, 3132-3141, 2020.
• 26. Adegun A.A., Viriri S., Ogundokun R.O., Deep learning approach for medical image analysis, Computational Intelligence and Neuroscience, 2021, 1-9, 2021.
• 27. Alhudhaif A., Ocal H., Barisci N., Atacak İ., Nour M., Polat K., A novel approach to skin lesion segmentation: Multipath fusion model with fusion loss, Computational and Mathematical Methods in Medicine, 2022, 1-12, 2022.
• 28. Beeche C., Singh J.P., Leader J.K., Gezer N.S., Oruwari A.P., Dansingani K.K., Chhablani J., Pu J., Super U-Net: A modularized generalizable architecture, Pattern Recognition, 128, 1-12, 2022.
• 29. Rehman A., Butt M.A., Zaman M., Attention Res-UNet: Attention residual UNet with focal tversky loss for skin lesion segmentation, International Journal of Decision Support System Technology (IJDSST), 15 (1), 1-17, 2023.
• 30. Sun J., Xi W., Bai G., Liu X., Yu F., Zhang C., ACFNet: An adaptive context fusion network for skin lesion segmentation, International Joint Conference on Neural Networks (IJCNN), Padua-Italy, 1-8, 2022.
• 31. Yu Z., Yu L., Zheng W., Wang S., EIU-Net: Enhanced feature extraction and improved skip connections in U-Net for skin lesion segmentation, Computers in Biology and Medicine, 162, 1-10, 2023.
• 32. Fan C., Yang L., Lin H., Qiu Y., DFE-Net: Dual-branch feature extraction network for Enhanced segmentation in skin lesion, Biomedical Signal Processing and Control, 81, 1-12, 2023.
• 33. Liu L, Wang G., Wu Y., Wang H., Li Y., PCCA-Model: an attention module for medical image segmentation, Biomedical Optics Express, 14 (4), 1428-1444, 2023.
• 34. Hu J., Shen L., Sun G., Squeeze-and-excitation networks, IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City-UT-USA, 7132-7141, 2018.
• 35. Yang L., Fan C., Lin H., Qiu Y., Rema-Net: An efficient multi-attention convolutional neural network for rapid skin lesion segmentation, Computers in Biology and Medicine, 159, 1-13, 2023.
• 36. Le P.T., Pham B., Chang C., Hsu Y. Tai T., Li Y., Wang J., Anti-aliasing attention U-net model for skin lesion segmentation, Diagnostics, 13 (8),1460, 2023.
• 37. Kingma D.P., Ba J.L., ADAM: A method for stochastic optimization. https://arxiv.org/pdf/1412.6980.pdf. Güncelleme tarihi Ocak 30, 2017. Erişim tarihi Haziran 6, 2023.
• 38. Tensorflow platform. https://www.tensorflow.org. Erişim tarihi Haziran 6, 2023.
• 39. Keras platform. https://keras.io. Erişim tarihi Haziran 6, 2023.
• 40. Mendonça T., Ferreira P.M., Marques J.S., Marcal A.R.S., Rozeira J., PH2 - a dermoscopic image database for research and benchmarking, In: 35th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), Osaka- Japan, 5437-5440, 2013.
• 41. PH2 veri kümesi. https://www.fc.up.pt/addi/ph2%20database.html. Erişim tarihi Haziran 6, 2023.
• 42. Bloice M.D., Stocker C., Holzinger A., Augmentor: An image augmentation library for machine learning. https://arxiv.org/abs/1708.04680. Erişim tarihi Haziran 6, 2023.
• 43. Gutman D., Codella N.C. F., Celebi E., Helba B., Marchetti M., Mishra N., Halpern A., Skin lesion analysis toward melanoma detection: A challenge at the International Symposium on Biomedical Imaging (ISBI) 2016, hosted by the International Skin Imaging Collaboration (ISIC). https://arxiv.org/abs/1605.01397. Erişim tarihi Haziran 6, 2023.
• 44. ISIC-2016 veri kümesi. https://challenge.isic-archive.com/data/#2016. Erişim tarihi Haziran 6, 2023.
• 45. Zhang H., Cissé M., Dauphin Y., Lopez-Paz D., mixup: Beyond empirical risk minimization, 6th International Conference on Learning Representations (ICLR), Vancouver- BC-Canada, 1-13, 2018.
• 46. Codella N., Gutman D., Celebi M.E., Helba B., Marchetti M.A., Dusza S., Kalloo A., Liopyris K., Mishra N., Kittler H., Halpern A., Skin lesion analysis toward melanoma detection: A challenge at the 2017 International Symposium on Biomedical Imaging (ISBI), Hosted by the International Skin Imaging Collaboration (ISIC). https://arxiv.org/pdf/1710.05006.pdf. Erişim tarihi Haziran 6, 2023.
• 47. ISIC-2017 veri kümesi. https://challenge.isic-archive.com/data/#2017. Erişim tarihi Haziran 6, 2023.
• 48. Divyanth L.G., Ahmad A., Saraswat D., A two-stage deep-learning based segmentation model for crop disease quantification based on corn field imagery, Smart Agricultural Technology, 3, 1-12, 2023.
• 49. Zhou T., Canu S., Vera P., Ruan S., A dual supervision guided attentional network for multimodal MR brain tumor segmentation, Proceedings of 2021 International Conference on Medical Imaging and Computer-Aided Diagnosis (MICAD), Birmingham-United Kingdom, 1-9, 2021.
• 50. Vahadane A., B A., Majumdar S., Dual encoder attention U-net for nuclei segmentation, 43rd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC), Mexico, 3205-3208, 2021. 51. Zhao B., Chen X., Li Z., Yu Z., Yao S., Yan L., Wang Y., Liu Z., Liang C., Han C., Triple U-net: Hematoxylin-aware nuclei segmentation with progressive dense feature aggregation, Medical Image Analysis, 65, 1-11, 2020.
• 52. Ramadan R., Aly S., CU-Net: A new improved multi-input color U-Net model for skin lesion semantic segmentation, IEEE Access, 10, 15539-15564, 2022.
• 53. Zhao C., Shuai R., Ma L., Liu W., Wu M., Segmentation of skin lesions image based on U-Net++. Multimedia Tools and Applications, 81, 8691–8717, 2022.
• 54. Anand V., Gupta S., Koundal D., Nayak S.R., Barsocchi P., Bhoi A.K., Modified U-Net architecture for segmentation of skin lesion, Sensors, 22 (3), 867, 2022.
• 55. Zhang G., Wang S., Dense and shuffle attention U-Net for automatic skin lesion segmentation, Int J Imaging Syst Technol, 32 (6), 2066- 2079, 2022.
• 56. Song Z., Luo W., Shi Q., Res-CDD-Net: A network with multi-scale attention and optimized decoding path for skin lesion segmentation, Electronics, 11 (17), 2672, 2022.
• 57. Araújo R.L., Araújo F.H.D., Silva R.R.V., Automatic segmentation of melanoma skin cancer using transfer learning and fine-tuning, Multimedia Systems, 28, 1239–1250, 2022.
• 58. Gu R., Wang L., Zhang L., DE-Net: A deep edge network with boundary information for automatic skin lesion segmentation, Neurocomputing, 468, 71-84, 2022.
• 59. Aghdam E.K., Azad R., Zarvani M., Merhof D., Attention swin U-Net: Cross-contextual attention mechanism for skin lesion segmentation. https://arxiv.org/abs/2210.16898. Yayın tarihi Ekim 30, 2022. Erişim tarihi Haziran 6, 2023.
• 60. Hafhouf B., Zitouni A., Megherbi A.C., Sbaa S., An Improved and Robust Encoder–Decoder for Skin Lesion Segmentation, Arabian Journal for Science and Engineering, 47, 9861–9875, 2022.
• 61. Oliveira A., Pereira S., Silva C.A., Augmenting data when training a CNN for retinal vessel segmentation: How to warp?, 2017 IEEE 5th Portuguese Meeting on Bioengineering (ENBENG), Coimbra-Portugal, 1-4, 2017.
• 62. Ribeiro V., Avila S., Valle E., Less is more: Sample selection and label conditioning improve skin lesion segmentation, IEEE/CVF 2020 Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Seattle- WA-USA, 3182-3191, 2020.
• 63. Akyel C., Arıcı N., LinkNet-B7: Noise removal and lesion segmentation in images of skin cancer, Mathematics, 10 (5), 736, 2022.
• 64. Thapar P., Rakhra M., Cazzato G., Hossain S., A novel hybrid deep learning approach for skin lesion segmentation and classification, Journal of Healthcare Engineering, 2022, 1-21, 2022.
• 65. Arpacı S.A., Varlı S., Retinal vessel segmentation with differentiated U-Net network, 28th Signal Processing and Communications Applications Conference, Gaziantep-Turkey, 1-4, 2020.
• 66. Arpacı S.A., Varlı S., Retinal vessel segmentation with the mixup data augmentation method, Journal of Health Institutes of Türkiye, 5 (1), 41-50, 2022.
• 67. Arpacı S.A., Varlı S., Semantic segmentation with the mixup data augmentation method, 30th Signal Processing and Communications Applications Conference (SIU), Safranbolu-Turkey, 1-4, 2022.
• 68. Nishio M., Noguchi S., Fujimoto K., Automatic pancreas segmentation using coarse-scaled 2D model of deep learning: Usefulness of data augmentation and deep U-Net, Applied Sciences, 10 (10), 3360, 2020.
• 69. Noguchi S., Nishio M., Yakami M., Nakagomi K., Togashi K., Bone segmentation on whole-body CT using convolutional neural network with novel data augmentation techniques, Computers in Biology and Medicine, 121, 2020.
• 70. Konur U., Semi-automatic heuristic segmentation of fetal skull images, Journal of the Faculty of Engineering and Architecture of Gazi University, 38 (2), 679–692, 2022.
• 71. Eker A.G., Pehlivanoğlu M.K., İnce İ., Duru N., Deep learning and transfer learning based brain tumor segmentation, 8th International Conference on Computer Science and Engineering (UBMK), Burdur-Turkiye, 163-168, 2023.
• 72. Çekiç İ., Çavdar K., Detection of the cracks in metal sheets using convolutional neural network (CNN), Journal of the Faculty of Engineering and Architecture of Gazi University, 38 (1), 153–162, 2022.
• 73. Türkmen M., Orman, Z., İnsan omurgasına ait bilgisayarlı tomografi görüntülerinin iyileştirilmesi ve omur segmentasyonu, Avrupa Bilim ve Teknoloji Dergisi, 52, 95-103, 2023.