Comparison of U-Net Based Models for Human Embryo Segmentation

Year 2022, Volume: 15 Issue: 1, 35 - 44, 31.01.2022

Nefise Uysal , Tahir Koray Yozgatlı , Ecem Nur Yıldızcan , Emre Kar , Murat Gezer , Ercan Baştu

https://doi.org/10.17671/gazibtd.949430

Cited By: 2

Abstract

The quality of human embryos produced during in vitro fertilization is conventionally graded by clinical embryologists and this process is time-consuming and prone to human error. Artificial intelligence methods may be used to grade images captured by time-lapse microscopy (TLM). Segmentation of embryos from the background of TLM images is an essential step for embryo quality assessment as the background of the embryo has various artifacts which may mislead the grading algorithms. In this study, we performed a comparative analysis of automated day-5 human embryo (blastocyst) image segmentation methods based on deep learning. Four fully convolutional deep models, including U-Net and its three variants, were created using the combination of two gradient descent-based optimizers and two-loss functions and compared to our proposed model. The experimental results on the test set confirmed that our customized Dilated Inception U-Net model with Adam optimizer and Dice loss outperformed other U-Net variants with Dice coefficient, Jaccard index, accuracy, and precision of 98.68%, 97.52%, 99.20%, and 98.52%, respectively.

Keywords

U-Net, deep learning, convolutional neural network, in vitro fertilization (IVF), human embryo, segmentation

Supporting Institution

TÜBİTAK

Project Number

2190328

Thanks

This study was completed with the support within the scope of 1512 Program, carried out by the Scientific and Technological Research Council of Turkey (TUBITAK). The authors would like to thank to TUBITAK for this support.

İnsan Embriyo Segmentasyonu için U-Net Tabanlı Modellerin Karşılaştırılması

Abstract

Tüp bebek tedavisi sırasında üretilen insan embriyolarının kalitesi, geleneksel olarak klinik embriyologlar tarafından derecelendirilir ve bu süreç zaman alıcı olup insan hatasına açıktır. Hızlandırılmış mikroskopi (TLM) yöntemi ile alınan görüntüleri derecelendirmek için yapay zeka yöntemleri kullanılabilir. TLM görüntülerinde embriyonun arka plandan segmentasyonu, arka planın derecelendirme algoritmalarını yanlış yönlendirebilecek çeşitli artefaktlara sahip olması nedeniyle embriyo kalite değerlendirmesi için önemli bir adımdır. Bu çalışmada, derin öğrenmeye dayalı otomatikleştirilmiş 5. gün insan embriyosu (blastosist) görüntü segmentasyon yöntemlerinin karşılaştırmalı bir analizi yapılmıştır. U-Net ve üç varyantından oluşan dört tam evrişimli derin model, iki gradyan iniş tabanlı optimizasyon algoritmasının ve iki kayıp fonksiyonunun kombinasyonu kullanılarak oluşturulmuş ve önerilen modelimiz ile karşılaştırılmıştır. Test setindeki deneysel sonuçlar, optimizasyon fonksiyonu olarak Adam ve kayıp fonksiyonu olarak ise Dice kullanan özelleştirilmiş Dilated Inception U-Net modelinin, sırasıyla %98.68, %97.52, %99.20 ve %98.52'lik Dice katsayısı, Jaccard benzerlik katsayısı, doğruluk ve kesinlik ile diğer U-Net tabanlı modellerden daha iyi performans gösterdiğini doğrulamıştır.

Keywords

U-Net, derin öğrenme, evrişimli sinir ağları, in vitro fertilizasyon (IVF), insan embriyosu, segmentasyon

Project Number

2190328

References

• K. S. Tamilselvan and G. Murugesan, “Image Segmentation”, Medical and Biological Image Analysis, Editor: R.Koprowski, IntechOpen, London, UK, 2018. P. Aggarwal, V. Renu, S. Bhadoria, and C. Dethe, “Role of Segmentation in Medical Imaging: A Comparative Study”, International Journal of Computer Applications, 29(1), 54–61, 2011.
• M. Vander Borght and C. Wyns, “Fertility and infertility: Definition and epidemiology”, Clinical Biochemistry, 62, 2–10, 2018.
• G. E. Crawford and W. L. Ledger, “In vitro fertilisation/intracytoplasmic sperm injection beyond 2020”, BJOG: An International Journal of Obstetrics & Gynaecology, 126(2), 237-243, 2019.
• D. Castelló, Y. Motato, N. Basile, J. Remohí, M. Espejo-Catena, and M. Meseguer, “How much have we learned from time-lapse in clinical IVF?”, Molecular Human Reproduction, 22(10), 719–727, 2016.
• A. M. Lee, M. T. Connell, J. M. Csokmay, and A. K. Styer, “Elective single embryo transfer- the power of one”, Contraception and Reproductive Medicine, 1, 11, 2016.
• D. M. Kissin, A. D. Kulkarni, V. A. Kushnir, and D. J. Jamieson, “Number of Embryos Transferred After In Vitro Fertilization and Good Perinatal Outcome”, Obstetrics and Gynecology, 123(2 Pt 1), 239–247, 2014.
• R. J. Heitmann, M. J. Hill, K. S. Richter, A. H. DeCherney, and E. A. Widra, “The simplified SART embryo scoring system is highly correlated to implantation and live birth in single blastocyst transfers”, Journal of Assisted Reproduction and Genetics, 30(4), 563–567, 2013.
• J. L. Collins, B. van Knippenberg, K. Ding, and A. V.Kofman, “Time-Lapse Microscopy”, Cell Culture, Editor: Radwa Ali Mehanna, IntechOpen, London, UK, 2018.
• K. Kirkegaard, A. Ahlström, H. J. Ingerslev, and T. Hardarson, “Choosing the best embryo by time lapse versus standard morphology”, Fertility and Sterility, 103(2), 323–332, 2015.
• L. Sundvall, H. J. Ingerslev, U. Breth Knudsen, and K. Kirkegaard, “Inter- and intra-observer variability of time-lapse annotations”, Human Reproduction, 28(12), 3215–3221, 2013.
• D. J. Kaser and C. Racowsky, “Clinical outcomes following selection of human preimplantation embryos with time-lapse monitoring: a systematic review”, Human Reproduction Update, 20(5), 617–631, 2014.
• E. S. Filho, J. A. Noble, M. Poli, T. Griffiths, G. Emerson, and D. Wells, “A method for semi-automatic grading of human blastocyst microscope images”, Human Reproduction, 27(9), 2641–2648, 2012. C. Manna, L. Nanni, A. Lumini, and S. Pappalardo, “Artificial intelligence techniques for embryo and oocyte classification”, Reproductive Biomedicine Online, 26(1), 42–49, 2013.
• A. A. Septiandri, A. Jamal, P. A. Iffanolida, O. Riayati, and B. Wiweko, “Human Blastocyst Classification after In Vitro Fertilization Using Deep Learning”, 7th International Conference on Advance Informatics: Concepts, Theory and Applications (ICAICTA), Online Virtual Conference, 1–4, 2020.
• A. Karlsson, N. C. Overgaard, and A. Heyden, “Automatic segmentation of zona pellucida in HMC images of human embryos”, Proceedings of the 17th International Conference on Pattern Recognition (ICPR), Cambridge, UK, 8380971, 518-521, 2004.
• A. Karlsson, N. Chr. Overgaard, and A. Heyden, “A Two-Step Area Based Method for Automatic Tight Segmentation of Zona Pellucida in HMC Images of Human Embryos”, Scale Space and PDE Methods in Computer Vision, Berlin, Heidelberg, 503–514, 2005.
• D. A. Morales, E. Bengoetxea, and P. Larrañaga, “Automatic Segmentation of Zona Pellucida in Human Embryo Images Applying an Active Contour Model”, Proceedings of the 12th Medical Image Understanding and Analysis (MIUA), Dundee, UK, 209-213, 2008.
• E. S. Filho, J. A. Noble, and D. Wells, “A Review on Automatic Analysis of Human Embryo Microscope Images”, The Open Biomedical Engineering Journal, 4(1), 170–177, 2010.
• P. Saeedi, D. Yee, J. Au, and J. Havelock, “Automatic Identification of Human Blastocyst Components via Texture”, IEEE Transactions on Biomedical Engineering, 64(12), 2968–2978, 2017.
• A. Singh, J. Au, P. Saeedi, and J. Havelock, “Automatic Segmentation of Trophectoderm in Microscopic Images of Human Blastocysts”, IEEE Transactions on Biomedical Engineering, 62(1), 382–393, 2015.
• R. M. Rad, P. Saeedi, J. Au, and J. Havelock, “Trophectoderm segmentation in human embryo images via inceptioned U-Net”, Medical Image Analysis, 62(4), 101612, 2020.
• A. Krizhevsky, I. Sutskever, and G. E. Hinton, “ImageNet classification with deep convolutional neural networks”, Proceedings of the 25th International Conference on Neural Information Processing Systems, Red Hook, NY, USA, 1097–1105, 2012.
• J. Long, E. Shelhamer, and T. Darrell, “Fully convolutional networks for semantic segmentation”, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, USA, 3431–3440, 2015.
• O. Ronneberger, P. Fischer, and T. Brox, “U-Net: Convolutional Networks for Biomedical Image Segmentation”, Medical Image Computing and Computer-Assisted Intervention – MICCAI 2015, Munih, Germany, 234–241, 2015.
• S. Kheradmand, A. Singh, P. Saeedi, J. Au, and J. Havelock, “Inner cell mass segmentation in human HMC embryo images using fully convolutional network”, 2017 IEEE International Conference on Image Processing (ICIP), Beijing, China, 1752–1756, 2017.
• R. M. Rad, P. Saeedi, J. Au, and J. Havelock, “Multi-Resolutional Ensemble of Stacked Dilated U-Net for Inner Cell Mass Segmentation in Human Embryonic Images”, 25th IEEE International Conference on Image Processing (ICIP), Athens, Greece, 3518–3522, 2018.
• M. Y. Harun, T. Huang, and A. T. Ohta, “Inner Cell Mass and Trophectoderm Segmentation in Human Blastocyst Images using Deep Neural Network”, IEEE 13th International Conference on Nano/Molecular Medicine & Engineering (NANOMED), Gwangju, Korea (South), 214–219, 2019.
• C. Szegedy et al., “Going deeper with convolutions”, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, USA, 1–9, 2015.
• M. A. Kızrak and B. Bolat, Derin Öğrenme ile Kalabalık Analizi Üzerine Detaylı Bir Araştırma, Bilişim Teknolojileri Dergisi, 11(3), 263–286, 2018.
• Z. Zhang, Q. Liu, and Y. Wang, “Road Extraction by Deep Residual U-Net”, IEEE Geoscience Remote Sensing Letters, 15(5), 749–753, 2018.
• W. Shi, F. Jiang, and D. Zhao, “Single image super-resolution with dilated convolution based multi-scale information learning inception module”, 2017 IEEE International Conference on Image Processing (ICIP), 977–981, 2017.
• G. Klambauer, T. Unterthiner, A. Mayr, and S. Hochreiter, “Self-Normalizing Neural Networks”, Proceedings of the 25th International Conference on Neural Information Processing Systems, Red Hook, NY, USA, 972–981, 2017.
• Y. Wu and K. He, “Group Normalization”, International Journal of Computer Vision, 128(3), 742–755, 2020.
• D. P. Kingma and J. Ba, “Adam: A Method for Stochastic Optimization”, Proceedings of the 3rd International Conference on Learning Representations (ICLR 2015), San Diego, 2015.
• T. Tieleman and G. Hinton, “Lecture 6.5-rmsprop: Divide the gradient by a running average of its recent magnitude”, COURSERA: Neural Networks for Machine Learning, 4(2), 26–31, 2012.
• F. Milletari, N. Navab, and S.-A. Ahmadi, “V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation”, 2016 Fourth International Conference on 3D Vision (3DV), California, USA, 565–571, 2016.
• S. S. M. Salehi, D. Erdogmus, and A. Gholipour, “Tversky Loss Function for Image Segmentation Using 3D Fully Convolutional Deep Networks”, Machine Learning in Medical Imaging: 8th International Workshop, MLMI 2017, QC, Canada, 379–387, 2017.
• A. A. Taha and A. Hanbury, “Metrics for evaluating 3D medical image segmentation: analysis, selection, and tool”, BMC Medical Imaging, 15(29), 2015.
• M. Y. Harun et al., “Image Segmentation of Zona-Ablated Human Blastocysts”, 2019 IEEE 13th International Conference on Nano/Molecular Medicine Engineering (NANOMED), Gwangju, South Korea, 208–213, 2019.