Comparative analysis of encoder variants in deep learning-based semantic segmentation of concrete cracks

Year 2024, Volume: 15 Issue: 3, 581 - 593, 30.09.2024

Hasan Polat , Serhat Alpergin , Mehmet Siraç Özerdem

https://doi.org/10.24012/dumf.1465724

Abstract

Following natural disasters such as earthquakes, floods, and fires, significant damages manifest in both buildings and urban infrastructure. Cracks are widely recognized as the predominant indicators of damage or deterioration in concrete structures. Hence, the early and accurate detection of crack defects is crucial to ensure structural safety and longer service life. Deep learning architectures, which have made a significant breakthrough in computer vision applications in recent years, have begun to be widely used in the automatic detection and segmentation of concrete cracks. In particular, deep learning-based segmentation architectures, typically comprising an encoder and a decoder part, play a crucial role in conducting thorough structural health analyses by precisely detecting cracks along with their spatial boundaries. However, encoder block limitations such as the small receptive field of convolution kernels, information losses caused by the pooling operation, and insufficient local feature processing can hinder segmentation performance. This study examines the efficacy of various backbone architectures (ResNet-18, ResNet-50, MobileNetV2, Xception, and Inception-ResNet) as employed in the encoder block within the DeepLabV3+ framework, proposed for the segmentation of cracks on concrete surfaces. The effectiveness of low-level and high-level features provided by different backbone architectures in the encoder part was evaluated on open-access DeepCrack and CrackForest datasets. The results revealed that the MobileNetV2 architecture was the most successful network in terms of learnable parameters and segmentation performance for both data sets. The MobileNetV2 encoder-based segmentation framework achieved 0.81 and 0.70 Dice similarity coefficient (DSC) for both datasets, respectively, using approximately 6.7 million learnable weights.

Keywords

Semantic Segmentation, Deep Learning, Image Processing, Crack Detection, Crack Segmentation

Beton çatlakların derin öğrenme tabanlı semantik segmentasyonunda kodlayıcı değişkenlerinin karşılaştırmalı analizi

Abstract

Depremler, seller ve yangınlar gibi doğal afetler akabinde yapılarda ve kentsel altyapıda ciddi hasarlar meydana gelmektedir. Çatlaklar, beton yapılarda meydana gelen hasarların veya bozulmaların en yaygın belirtileri olarak kabul edilmektedir. Dolayısıyla, çatlak kusurlarının erken ve doğru bir şekilde tespit edilmesi, bu tür yapıların güvenliklerinin sağlanması ve hizmet süreleri açısından önem arz etmektedir. Son yıllarda bilgisayarlı görü uygulamalarında önemli bir atılım sergileyen derin öğrenme mimarileri, beton çatlaklarının otomatik olarak tespit ve segmente edilmesinde yaygın olarak kullanılmaya başlanmıştır. Özellikle, genel olarak bir kodlayıcı ve bir kod çözücü bloktan oluşan derin öğrenme tabanlı segmentasyon mimarileri çatlakları uzamsal sınırları ile tespit ederek, kapsamlı yapı sağlığı analizlerini mümkün kılmaktadır. Ancak, evrişimsel filtrede küçük alıcı alan, pooling işleminin neden olduğu bilgi kayıpları ve yetersiz yerel özellik işlenmesi gibi kodlayıcı blok sınırlandırmaları segmentasyon performansını sekteye uğratmaktadır. Bu çalışmada, beton yüzeylerindeki çatlakların segmentasyonu için önerilen DeepLabV3+ mimarisinde kodlayıcı blok için farklı omurga mimarilerinin (ResNet-18, ResNet-50, MobileNetV2, Xception ve Inception-ResNet) etkinlikleri analiz edilmiştir. Farklı omurga mimariler ile sağlanan alçak ve yüksek seviyeli özelliklerin etkinliklerinin test edilmesi için erişime açık Deepcrack ve CrackForest veri setleri kullanılmıştır. Bulgular her iki veri seti için de MobileNetV2 mimarisinin eğitilebilir parametre ve segmentasyon perfromansı açısından en başarılı ağ olduğunu göstermiştir. MobileNetV2 kodlayıcı tabanlı segmentasyon çerçevesi, yaklaşık 6.7 milyon eğitilebilir ağırlık kullanarak her iki veri seti için sırasıyla 0.81 ve 0.70 Dice benzerlik katsayısı (DSC) başarımı elde etmiştir.

Keywords

Semantik Segmentasyon, Derin Öğrenme, Görüntü İşleme, Çatlak Tespiti, Çatlak Segmenasyonu, Yapı Denetimi

References

• [1] Q. An et al., “Segmentation of Concrete Cracks by Using Fractal Dimension and UHK-Net,” Fractal Fract., vol. 6, no. 2, pp. 1–18, 2022, doi: 10.3390/fractalfract6020095.
• [2] L. Song, H. Sun, J. Liu, Z. Yu, and C. Cui, “Automatic segmentation and quantification of global cracks in concrete structures based on deep learning,” Meas. J. Int. Meas. Confed., vol. 199, no. June, 2022, doi: 10.1016/j.measurement.2022.111550.
• [3] X. Han et al., “Structural damage-causing concrete cracking detection based on a deep-learning method,” Constr. Build. Mater., vol. 337, no. 196, 2022, doi: 10.1016/j.conbuildmat.2022.127562.
• [4] Y. Bai, H. Sezen, and A. Yilmaz, “End-to-end deep learning methods for automated damage detection in extreme events at various scales,” Proc. - Int. Conf. Pattern Recognit., no. c, pp. 5736–5743, 2020, doi: 10.1109/ICPR48806.2021.9413041.
• [5] W. Wang, C. Su, G. Han, and H. Zhang, “A lightweight crack segmentation network based on knowledge distillation,” vol. 76, no. May, 2023.
• [6] L. Yang, H. Huang, S. Kong, and Y. Liu, “A deep segmentation network for crack detection with progressive and hierarchical context fusion,” J. Build. Eng., vol. 75, no. May, 2023, doi: 10.1016/j.jobe.2023.106886.
• [7] J. König, M. D. Jenkins, M. Mannion, P. Barrie, and G. Morison, “Optimized deep encoder-decoder methods for crack segmentation,” Digit. Signal Process. A Rev. J., vol. 108, 2021, doi: 10.1016/j.dsp.2020.102907.
• [8] A. Mahgoub, A. Talab, Z. Huang, F. Xi, and L. DUJE (Dicle University Journal of Engineering) 15:3 (2024) Sayfa 581-593 592 Haiming, “Detection crack in image using Otsu method and multiple filtering in image processing techniques,” Opt. - Int. J. Light Electron Opt., pp. 1– 4, 2015, doi: 10.1016/j.ijleo.2015.09.147.
• [9] P. Chun, S. Izumi, and T. Yamane, “Automatic detection method of cracks from concrete surface imagery using two-step light gradient boosting machine,” pp. 61–72, 2021, doi: 10.1111/mice.12564.
• [10] Q. Zou, Y. Cao, Q. Li, Q. Mao, and S. Wang, “CrackTree : Automatic crack detection from pavement images,” Pattern Recognit. Lett., vol. 33, no. 3, pp. 227–238, 2012, doi: 10.1016/j.patrec.2011.11.004.
• [11] Y. Liu, J. Yao, X. Lu, R. Xie, and L. Li, “DeepCrack: A deep hierarchical feature learning architecture for crack segmentation,” Neurocomputing, vol. 338, pp. 139–153, 2019, doi: 10.1016/j.neucom.2019.01.036.
• [12] R. Pu, G. Ren, H. Li, W. Jiang, J. Zhang, and H. Qin, “Autonomous Concrete Crack Semantic Segmentation Using Deep Fully Convolutional Encoder–Decoder Network in Concrete Structures Inspection,” Buildings, vol. 12, no. 11, 2022, doi: 10.3390/buildings12112019.
• [13] K. Makantasis, E. Protopapadakis, A. Doulamis, N. Doulamis, and C. Loupos, “Deep Convolutional Neural Networks for efficient vision based tunnel inspection,” Proc. - 2015 IEEE 11th Int. Conf. Intell. Comput. Commun. Process. ICCP 2015, pp. 335– 342, 2015, doi: 10.1109/ICCP.2015.7312681.
• [14] F. Bahreini and A. Hammad, “Dynamic graph CNN based semantic segmentation of concrete defects and as-inspected modeling,” Autom. Constr., vol. 159, no. November 2023, p. 105282, 2024, doi: 10.1016/j.autcon.2024.105282.
• [15] H. Li, H. Zhang, H. Zhu, K. Gao, H. Liang, and J. Yang, “Automatic crack detection on concrete and asphalt surfaces using semantic segmentation network with hierarchical Transformer,” Eng. Struct., vol. 307, no. February, p. 117903, 2024, doi: 10.1016/j.engstruct.2024.117903.
• [16] W. Qayyum, R. Ehtisham, A. Bahrami, C. Camp, J. Mir, and A. Ahmad, “Assessment of Convolutional Neural Network Pre-Trained Models for Detection and Orientation of Cracks,” pp. 1–16, 2023.
• [17] Y. J. Cha, W. Choi, G. Suh, S. Mahmoudkhani, and O. Büyüköztürk, “Autonomous Structural Visual Inspection Using Region-Based Deep Learning for Detecting Multiple Damage Types,” Comput. Civ. Infrastruct. Eng., vol. 33, no. 9, pp. 731–747, 2018, doi: 10.1111/mice.12334.
• [18] F. Panella, A. Lipani, and J. Boehm, “Semantic segmentation of cracks: Data challenges and architecture,” Autom. Constr., vol. 135, no. December 2021, p. 104110, 2022, doi: 10.1016/j.autcon.2021.104110.
• [19] M. Jamshidi, M. El-Badry, and N. Nourian, “Improving Concrete Crack Segmentation Networks through CutMix Data Synthesis and Temporal Data Fusion,” Sensors, vol. 23, no. 1, 2023, doi: 10.3390/s23010504.
• [20] O. Ronneberger, P. Fischer, and T. Brox, “U-net: Convolutional networks for biomedical image segmentation,” Lect. Notes Comput. Sci. (including Subser. Lect. Notes Artif. Intell. Lect. Notes Bioinformatics), vol. 9351, pp. 234–241, 2015, doi: 10.1007/978-3-319-24574-4_28/COVER/.
• [21] V. Badrinarayanan, A. Kendall, and R. Cipolla, “\href{https://arxiv.org/pdf/1511.00561.pdf}{Segnet : A deep convolutional encoder-decoder architecture for image segmentation},” IEEE Trans. Pattern Anal. Mach. Intell., vol. 39, no. 12, pp. 2481–2495, 2017.
• [22] E. Shelhamer, J. Long, and T. Darrell, “Fully Convolutional Networks for Semantic Segmentation,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 39, no. 4, pp. 640–651, Apr. 2017, doi: 10.1109/TPAMI.2016.2572683.
• [23] L. C. Chen, Y. Zhu, G. Papandreou, F. Schroff, and H. Adam, “Encoder-decoder with atrous separable convolution for semantic image segmentation,” Lect. Notes Comput. Sci. (including Subser. Lect. Notes Artif. Intell. Lect. Notes Bioinformatics), vol. 11211 LNCS, pp. 833–851, 2018, doi: 10.1007/978-3-030- 01234-2_49.
• [24] Y. Zhang, X. Gao, and H. Zhang, “Deep LearningBased Semantic Segmentation Methods for Pavement Cracks,” 2023.
• [25] T. Lee, J. H. Kim, S. J. Lee, S. K. Ryu, and B. C. Joo, “Improvement of Concrete Crack Segmentation Performance Using Stacking Ensemble Learning,” Appl. Sci., vol. 13, no. 4, 2023, doi: 10.3390/app13042367.
• [26] Z. Al-Huda, B. Peng, R. N. A. Algburi, M. A. Alantari, R. AL-Jarazi, and D. Zhai, “A hybrid deep learning pavement crack semantic segmentation,” Eng. Appl. Artif. Intell., vol. 122, no. November 2022, p. 106142, 2023, doi: 10.1016/j.engappai.2023.106142.
• [27] A. N. Soni, “Crack Detection in buildings using convolutional neural Network,” no. September, 2020.
• [28] A. Liu, W. Hua, J. Xu, Z. Yang, and J. Fu, “Concrete crack segmentation based on multi-dimensional structure information fusion-based network,” Constr. Build. Mater., vol. 414, no. August 2023, 2024, doi: 10.1016/j.conbuildmat.2024.134982.
• [29] H. Lang, Y. Yuan, J. Chen, S. Ding, J. J. Lu, and Y. Zhang, “Augmented Concrete Crack Segmentation: Learning Complete Representation to Defend Background Interference in Concrete Pavements,” IEEE Trans. Instrum. Meas., vol. 73, pp. 1–13, 2024, doi: 10.1109/TIM.2024.3378205.
• [30] Y. Shi, L. Cui, Z. Qi, F. Meng, and Z. Chen, “Automatic road crack detection using random structured forests,” IEEE Trans. Intell. Transp. Syst., vol. 17, no. 12, pp. 3434–3445, 2016, doi: 10.1109/TITS.2016.2552248.
• [31] B. Baheti, S. Innani, S. Gajre, and S. Talbar, “Semantic scene segmentation in unstructured environment with modified DeepLabV3+,” Pattern Recognit. Lett., vol. 138, pp. 223–229, 2020, doi: DUJE (Dicle University Journal of Engineering) 15:3 (2024) Sayfa 581-593 593 10.1016/j.patrec.2020.07.029.
• [32] T. Ahmad, V. Gharehbaghi, J. Li, C. Bennett, and R. Lequesne, “Crack segmentation in the wild using convolutional neural networks and bootstrapping,” Earthq. Eng. Resil., vol. 2, no. 3, pp. 348–363, 2023, doi: 10.1002/eer2.52.
• [33] Jia Deng, Wei Dong, R. Socher, Li-Jia Li, Kai Li, and Li Fei-Fei, “ImageNet: A large-scale hierarchical image database,” pp. 248–255, 2009, doi: 10.1109/cvprw.2009.5206848.
• [34] K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit., vol. 2016- Decem, pp. 770–778, 2016, doi: 10.1109/CVPR.2016.90.
• [35] B. Baheti, S. Gajre, and S. Talbar, “Semantic Scene Understanding in Unstructured Environment with Deep Convolutional Neural Network,” IEEE Reg. 10 Annu. Int. Conf. Proceedings/TENCON, vol. 2019- Octob, pp. 790–795, 2019, doi: 10.1109/TENCON.2019.8929376.
• [36] F. Chollet, “Xception: Deep learning with depthwise separable convolutions,” Proc. - 30th IEEE Conf. Comput. Vis. Pattern Recognition, CVPR 2017, vol. 2017-Janua, pp. 1800–1807, 2017, doi: 10.1109/CVPR.2017.195.
• [37] R. E. Philip, A. D. Andrushia, A. Nammalvar, B. G. A. Gurupatham, and K. Roy, “A Comparative Study on Crack Detection in Concrete Walls Using Transfer Learning Techniques,” J. Compos. Sci., vol. 7, no. 4, 2023, doi: 10.3390/jcs7040169.
• [38] A. G. Howard et al., “MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications,” Apr. 2017, [Online]. Available: http://arxiv.org/abs/1704.04861.
• [39] M. Sandler, A. Howard, M. Zhu, A. Zhmoginov, and L. C. Chen, “MobileNetV2: Inverted Residuals and Linear Bottlenecks,” Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit., pp. 4510–4520, 2018, doi: 10.1109/CVPR.2018.00474.
• [40] C. Szegedy, V. Vanhoucke, S. Ioffe, J. Shlens, and Z. Wojna, “Rethinking the Inception Architecture for Computer Vision,” Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit., vol. 2016-Decem, pp. 2818–2826, 2016, doi: 10.1109/CVPR.2016.308.
• [41] K. He, X. Zhang, S. Ren, and J. Sun, “Spatial pyramid pooling in deep convolutional networks for visual recognition,” Lect. Notes Comput. Sci. (including Subser. Lect. Notes Artif. Intell. Lect. Notes Bioinformatics), vol. 8691 LNCS, no. PART 3, pp. 346–361, 2014, doi: 10.1007/978-3-319-10578-9_23.
• [42] D. Tabernik, M. Šuc, and D. Skočaj, “Automated detection and segmentation of cracks in concrete surfaces using joined segmentation and classification deep neural network,” Constr. Build. Mater., vol. 408, no. September, p. 133582, 2023, doi: 10.1016/j.conbuildmat.2023.133582.
• [43] D. P. Kingma and J. L. Ba, “Adam: A method for stochastic optimization,” 3rd Int. Conf. Learn. Represent. ICLR 2015 - Conf. Track Proc., pp. 1–15, 2015