Use of photogrammetry in the development of digital twins and visualization with augmented reality

Yıl 2023, Cilt: 12 Sayı: 4, 1372 - 1384, 15.10.2023

Abdurahman Yasin Yiğit , Murat Uysal

https://doi.org/10.28948/ngumuh.1313019

Cited By: 2

Öz

The most important concept in digital twin creation is how well the object represents reality and how updatable it is. At this point, photogrammetry is one of the methods that best represent reality as it is a method of creating 3D models from images. With photogrammetry, multiple images of an object can be taken from different angles to create a 3D model of the object, which can then be used to create a digital twin. The focus of this paper is to formulate the basic requirements for a representation of 3D photogrammetric data generated from image data to function as a digital twin and to show how these basic requirements can be implemented in a prototype digital twin. For these purpose, digital twins of three objects of different sizes and complex structures were created by photogrammetry method and SfM algorithm in Context Capture software. Finally, the digital twin was visualized and presented to the user as augmented reality in an easy-to-access mobile or WEB based.

Anahtar Kelimeler

Digital twin, Photogrammetry, 3D model, Augmented reality, WEB-based visualization

Dijital ikizlerin geliştirilmesinde fotogrametrinin kullanımı ve artırılmış gerçeklik ile görselleştirilmesi

Öz

Dijital ikiz oluşturmada en önemli kavram, objenin gerçekliği ne kadar iyi temsil ettiği ve güncellenebilir olmasıdır. Fotogrametri yöntemi bu noktada fotoğraflardan 3B modeller oluşturma yöntemi olduğu için gerçekliği en iyi temsil eden yöntemlerden biridir. Fotogrametri yöntemiyle bir nesnenin farklı açılardan birden fazla fotoğrafı çekilerek nesnenin 3B modeli oluşturulabilir ve bu modeller daha sonra dijital ikiz oluşturmak için kullanılabilir. Çalışmanın ana odak noktası fotoğraf verilerinden üretilen 3B fotogrametrik verilerin dijital ikiz olarak işlev görmesi için bir temsilin temel gereksinimlerini formüle etmek ve bu temel gereksinimlerin bir prototip dijital ikizde nasıl uygulanabileceğini göstermektir. Bu amaç doğrultusunda farklı boyutlarda ve karmaşık yapıdaki üç adet objenin Context Capture yazılımında fotogrametri yöntemi ve SfM algoritması ile dijital ikizi oluşturulmuştur. Son olarak üretilen dijital ikizin görselleştirilmesi ve kullanıcıya mobil veya WEB ortamında erişiminin kolay bir şekilde artırılmış gerçeklik olarak sunumu gerçekleştirilmiştir.

Anahtar Kelimeler

Dijital ikiz, Fotogrametri, 3B model, Artırılmış gerçeklik, WEB tabanlı görselleştirme

Kaynakça

• M. W. Grieves, Virtually intelligent product systems: Digital and physical twins. Complex Systems Engineering: Theory and Practice. American Institute of Aeronautics and Astronautics. 2019. https://doi.org/10.2514/5.9781624105654.0175.0200.
• F. Tao, H. Zhang, A. Liu and A. Y. Nee, Digital twin in industry: State-of-the-art. IEEE Transactions on industrial informatics, 15 (4), 2405-2415, 2018. https://doi.org/10.1109/TII.2018.2873186.
• X. Zheng, J. Lu, and D. Kiritsis, The emergence of cognitive digital twin: vision, challenges and opportunities. International Journal of Production Research, 60 (24), 7610-7632, 2022.https://doi.org/v 10.1080/00207543.2021.2014591.
• F. Tao, N. Anwer, A. Liu, L. Wang, A. Y. Nee, L. Li, and M. Zhang, Digital twin towards smart manufacturing and industry 4.0. Journal of manufacturing systems, 58, 1-2, 2021.https://doi.org/ 10.1016/j.jmsy.2020.12.005.
• K. Y. H. Lim, P. Zheng and C. H. A. Chen, State-of-the-art survey of Digital Twin: techniques, engineering product lifecycle management and business innovation perspectives. Journal of Intelligent Manufacturing, 31, 1313-1337, 2020. https://doi.org/10.1007/s10845-019-01512-w.
• B. Carrión-Ruiz, S. Blanco-Pons, A. Weigert, S. Fai and J. L. Lerma, Merging photogrammetry and augmented reality: The Canadian Library of Parliament. International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, 42 (2/W11), 367-371, 2019. https://doi.org/10.5194/isprs-archives-XLII-2-W11-367-2019
• R. F. Grasso, F. Andresciani, C. Altomare, G. Pacella, G. Castiello, M. Carassiti and B. Beomonte Zobel, Lung Thermal Ablation: Comparison between an Augmented Reality Computed Tomography (CT) 3D Navigation System (SIRIO) and Standard CT-Guided Technique. Biology, 10 (7), 646, 2021. https://doi.org/10.3390/biology10070646.
• ISO/DIS 23247-1., Automation systems and integration–Digital Twin framework for manufacturing–Part 1: Overview and general principles, 2020.
• C. Wang, Y. K. Cho and C. Kim, Automatic BIM component extraction from point clouds of existing buildings for sustainability applications. Automation in Construction, 56, 1–13, 2015. https://doi.org/10.1016/ j.autcon.2015.04.001.
• Y. Yin, P. Zheng, C. Li and L. Wang, A state-of-the-art survey on Augmented Reality-assisted Digital Twin for futuristic human-centric industry transformation. Robotics and Computer-Integrated Manufacturing, 81, 102515, 2023. https://doi.org/10.1016/ j.rcim.2022.102515.
• E. J. Tuegel, A. R. Ingraffea, T. G. Eason and S. M. Spottswood, Reengineering aircraft structural life prediction using a digital twin. International Journal of Aerospace Engineering, 2011. https://doi.org/10.1155/ 2011/154798.
• J. Wilhelm, C. Petzoldt, T. Beinke and M. Freitag, Review of digital twin-based interaction in smart manufacturing: Enabling cyber-physical systems for human-machine interaction. International Journal of Computer Integrated Manufacturing, 34 (10), 1031-1048, 2021. https://doi.org/10.1080/ 0951192X.2021.1963482.
• G. Wiegand, C. Mai, Y. Liu and H. Hußmann, Early take-over preparation in stereoscopic 3d. In Adjunct proceedings of the 10th international conference on automotive user interfaces and interactive vehicular applications, 142-146, 2018, September.
• N. Kikuchi, T. Fukuda and N. Yabuki, Future landscape visualization using a city digital twin: Integration of augmented reality and drones with implementation of 3D model-based occlusion handling. Journal of Computational Design and Engineering, 9(2), 837-856, 2022. https://doi.org/10.1093/ jcde/qwac032.
• M. Yakar ve Y. Doğan, Mersin Silifke Mezgit Kale Anıt Mezarı fotogrametrik rölöve alımı ve üç boyutlu modelleme çalışması. Geomatik, 2 (1), 11-17, 2017. https://doi.org/10.29128/geomatik.296763.
• A. Kabadayı ve A. Erdoğan, Application of terrestrial photogrammetry method in cultural heritage studies: A case study of Seyfeddin Karasungur. Mersin Photogrammetry Journal, 4(2), 62-67, 2022. https://doi.org/10.53093/mephoj.1200146.
• F. Pulat, M. Yakar ve A. Ulvi, Three-dimensional modeling of the Kubbe-i Hasiye Shrine with terrestrial photogrammetric method. Cultural Heritage and Science, 3(1), 6-11, 2022.
• N. Polat, M. Önal, Y. Kaya, A. Memduhoğlu, N. Kaya, M. Ulukavak, S. Mutlu ve S. Mutlu, Harran Ören Yeri kazisinda bulunan kabartma yazilarin üç boyutlu olarak modellenmesi. Bitlis Eren Üniversitesi Fen Bilimleri Dergisi, 10 (2), 594-601, 2021.https://doi.org/ 10.17798/bitlisfen.881781.
• O. Mırdan ve M. Yakar, Tarihi eserlerin insansız hava aracı ile modellenmesinde karşılaşılan sorunlar. Geomatik, 2 (3), 118-125, 2017. https://doi.org/ 10.29128/geomatik.306914.
• H. İ. Şenol, F. B. Ernst ve S. Akdağ, Kentsel dönüşüm alanlarının geotasarım yöntemi ile planlanması: Eyyübiye örneği. Harran Üniversitesi Mühendislik Dergisi, 3 (3), 63-69, 2018.
• A. Şasi ve M. Yakar, Photogrammetric modelling of hasbey dar'ülhuffaz (masjid) using an unmanned aerial vehicle. International Journal of Engineering and Geosciences, 3 (1), 6-11, 2018. https://doi.org/ 10.26833/ijeg.328919.
• A. Erdoğan, A. Kabadayı ve E. S. Akın, Kültürel Mirasın Fotogrametrik Yöntemle 3B Modellenmesi: Karabıyık Köprüsü Örneği. Türkiye İnsansız Hava Araçları Dergisi, 3 (1), 23-27, 2021. https://doi.org/10.51534/tiha.911147
• S. M. Sepasgozar, Digital twin and web-based virtual gaming technologies for online education: A case of construction management and engineering. Applied Sciences, 10 (13), 4678, 2020. https://doi.org/10.3390/ app10134678.
• M. Mohammadi, M. Rashidi, V. Mousavi, A. Karami, Y. Yu, and B. Samali. Quality evaluation of digital twins generated based on UAV photogrammetry and TLS: Bridge case study. Remote Sensing, 13 (17), 3499, 2021.https://doi.org/10.3390/rs13173499.
• X. Kong, and R. G. Hucks, Preserving our heritage: A photogrammetry-based digital twin framework for monitoring deteriorations of historic structures. Automation in Construction, 152, 104928, 2023. https://doi.org/10.1016/j.autcon.2023.104928.
• K. Themistocleous, E. Evagorou, C. Mettas and D. Hadjimitsis. The use of digital twin models to document cultural heritage monuments. In Earth Resources and Environmental Remote Sensing/GIS Applications XIII, 12268, 55-64, 2022.https://doi.org/ 10.1117/12.2636332.
• P. N. Zakharov, A. V. Zhdanov, D. N. Lapaev, P. V. Strelkov and S. O. Maslov, The practice of using digital twins and augmented reality technologies for visualization of innovative products and technologies of enterprises in the region. Growth Poles of the Global Economy: Emergence, Changes and Future Perspectives, 1325-1333, 2020. https://doi.org/ 10.1007/978-3-030-15160-7_135.
• I. Verner, M. Reitman, D. Cuperman, T. Yan, E. Finkelstein and T. Romm, Exposing robot learning to students in augmented reality experience. In Smart Industry & Smart Education: Proceedings of the 15th International Conference on Remote Engineering and Virtual Instrumentation 15, 610-619. Springer International Publishing, 2019.
• G. Koutitas, J. Jabez, C. Grohman, C. Radhakrishna, V. Siddaraju and S. Jadon, Demo/poster abstract: XReality research lab—Augmented reality meets Internet of Things. In IEEE INFOCOM 2018-IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS), 1-2, 2018, April.
• Z. Wang, K. Han and P. Tiwari, Digital twin-assisted cooperative driving at non-signalized intersections. IEEE Transactions on Intelligent Vehicles, 7 (2), 198-209, 2021. https://doi.org/10.1109/TIV.2021.3100465.
• P. Wisely, Digital twinning within a novel human-in-the-loop verification method for HUD safety-critical approach and landing. In Virtual, Augmented, and Mixed Reality (XR) Technology for Multi-Domain Operations II, 11759, 124-135, 2021, April.
• P. A. Cruz Franco, A. Rueda Márquez de la Plata, and E. Gómez Bernal. Protocols for the graphic and constructive diffusion of digital twins of the architectural heritage that guarantee universal accessibility through AR and VR. Applied Sciences, 12(17), 8785, 2022. https://doi.org/10.3390/ app12178785.
• K. Kraus, Photogrammetry: geometry from images and laser scans. de Gruyter, 2007. https://doi.org/10.1515/ 9783110892871.47.
• I. Marzolff, J. Ries, S. E. W. Aber and J. S. Aber, Small-format aerial photography: principles, techniques and geoscience applications, 2019. https://doi.org/10.1016/C2016-0-03506-4.
• M. E. Oruç, Küçük objelerin modellenmesinde videogrametri ve fotogrametri yöntemlerinin karşılaştırılması üzerine bir çalışma. Türkiye Fotogrametri Dergisi, 3 (2), 62-68, 2021. https://doi.org/10.53030/tufod.1019385.
• S. N. G. Hamal ve A. Ulvi, Su altı fotogrametri yöntemi ve kullanım alanı üzerine bir literatür araştırması. Türkiye Fotogrametri Dergisi, 2(2), 60-71, 2020.
• M. J. Westoby, J. Brasington, N. F. Glasser, M. J. Hambrey and J. M. Reynolds, ‘Structure-fromMotion’ photogrammetry: A low-cost, effective tool for geoscience applications. Geomorphology (Amsterdam, Netherlands), 179, 300–314, 2012. https://doi.org/ 10.1016/j.geomorph.2012.08.021.
• K. Anderson, M. J. Westoby and M. R. James, Low-budget topographic surveying comes of age: Structure from motion photogrammetry in geography and the geosciences. Progress in Physical Geography: Earth and Environment, 43 (2), 163–173, 2019. https://doi.org/10.1177/0309133319837454.
• D. Fidan, M. E. Oruç, S. N. G. Hamal and Ş. Fidan, Tersine Mühendislik Uygulamalarında Yersel Lazer Tarayıcıların Kullanım Olanaklarının Araştırılması; Klasik Otomobiller Örneği. Türkiye Lidar Dergisi, 4 (1), 1-10, 2022.
• M. W. Smith, J. L. Carrivick, and D. J. Quincey, Structure from motion photogrammetry in physical geography. Progress in Physical Geography, 40 (2), 247-275, 2016. https://doi.org/10.1177% 2F0309133315615805.
• H. İ. Şenol ve Y. Kaya, İnternet tabanlı veri kullanımıyla yerleşim alanlarının modellenmesi: çiftlikköy kampüsü örneği. Türkiye Fotogrametri Dergisi, 1 (1), 11-16, 2019.
• S. N. G. Hamal, B. Sarı ve A. Ulvi, Using of hybrid data acquisition techniques for cultural heritage a case study of Pompeiopolis. Türkiye İnsansız Hava Araçları Dergisi, 2 (2), 55-60, 2020.
• S. D. Uzun, S. N. G. Hamal ve Ş. Fidan, Elde taşınabilir lazer tarayıcılar ile insan yüzünün modellenerek güzellik ve bakım sektöründe kullanımının değerlendirilmesi. Türkiye Lidar Dergisi, 4 (1), 17-20, 2022. https://doi.org/10.51946/melid.1131186.
• T. Luhmann, S. Robson, S. Kyle and J. Boehm, Close-Range Photogrammetry and 3D Imaging (3rd edition). Walter de Gruyter GmbH & Co KG, 2019.
• B. Sarı, S. N. G. Hamal ve A. Ulvi, Documentation of complex structure using Unmanned Aerial Vehicle (UAV) photogrammetry method and Terrestrial Laser Scanner (TLS). Türkiye Lidar Dergisi, 2 (2), 48-54, 2020.
• Y. Kaya, H. İ. Şenol ve N. Polat, Three-dimensional modeling and drawings of stone column motifs in Harran Ruins. Mersin Photogrammetry Journal, 3 (2), 48-52, 2021. https://doi.org/10.53093/ mephoj.1012937.
• A. Memduhoglu, H. İ. Şenol, S. Akdağ ve M. Ulukavak, 3D Map Experience for Youth with Virtual/Augmented Reality Applications. Harran Üniversitesi Mühendislik Dergisi, 5(3), 175-182, 2020.https://doi.org/10.46578/humder.771954.
• M. Yakar, H. M. Yılmaz ve Ö. Mutluoǧlu, Close range photogrammetry and robotic total station in volume calculation. International Journal of the Physical Sciences, 5 (2), 86–96, 2010.
• N. Polat ve Y. Kaya, Investigation of the performance of different pixel-based classification methods in land use/land cover (LULC) determination. Türkiye İnsansız Hava Araçları Dergisi, 3 (1), 1- 6, 2021. https://doi.org/10.51534/tiha.829656.
• H. M. Yılmaz, M. Yakar, S. A. Gulec ve O. Dulgerler, Importance of digital close-range photogrammetry in documentation of cultural heritage. Journal of Cultural Heritage, 8 (4), 428-433, 2007. https://doi.org/10.1016/ j.culher.2007.07.004.
• M. Yakar, Using close range photogrammetry to measure the position of inaccessible geological features. Experimental Techniques, 35 (1), 54-59, 2011.https://doi.org/10.1111/j.17471567.2009.00583.x.
• F. Remondino and S. Campana, 3D recording and modelling in archaeology and cultural heritage. BAR international series, 2598, 111-127, 2014.
• C. Sweeney, Theia vision library, 2016. URL http://www.theia-sfm.org/sfm.html.
• R. G. Boboc, F. Gîrbacia and E. V. Butilă, The application of augmented reality in the automotive industry: A systematic literature review. Applied Sciences, 10 (12), 4259, 2020. https://doi.org/ 10.3390/app10124259.
• U. Gürel, Artırılmış gerçeklik yardımı ile öğrenme deneyimi. Eskişehir Türk Dünyası Uygulama ve Araştırma Merkezi Bilişim Dergisi, 2 (1), 42-45, 2021.
• H. Thwaites, Digital Heritage: What Happens When We Digitize Everything? In Visual Heritage in the Digital Age; Ch’ng, E., Gaffney, V., Chapman, H., Eds.; Springer London: London, UK, 327–348, 2013.
• D. W. F. Van Krevelen and R. A. Poelman, Survey of augmented reality technologies, applications and limitations. International journal of virtual reality, 9 (2), 1-20, 2010.
• A. Atanasyan and J. Rossmann, An architecture for ar-based human-machine interaction with application to an autonomous mobile robot platform. In ISR 2020; 52th International Symposium on Robotics. 1-6, 2020, December.
• S. Discher, R. Richter and J. Döllner, Concepts and techniques for web-based visualization and processing of massive 3D point clouds with semantics. Graphical Models, 104, 101036, 2019. https://doi.org/10.1016/ j.gmod.2019.101036.
• A. Prouzeau, Y. Wang, B. Ens, W. Willett and T. Dwyer, Corsican twin: Authoring in situ augmented reality visualisations in virtual reality. In Proceedings of the international conference on advanced visual interfaces, 1-9, 2020, September.
• B. Demirezen, Artırılmış gerçeklik ve sanal gerçeklik teknolojisinin turizm sektöründe kullanılabilirliği üzerine bir literatür taraması. Uluslararası Global Turizm Araştırmaları Dergisi, 3 (1), 1-26, 2019.
• F. Bilici, Pazarlamada artırılmış gerçeklik ve karekod teknolojileri: tüketicilerin artırılmış gerçeklik teknoloji algılamaları üzerine bir alan araştırması (Doktora tezi). Bursa Uludağ Üniversitesi, Türkiye, 2015.
• F. N. Koç, Dijital pazarlamanın metaverse fenomenine sunduğu fırsatlar ve sınırlılıklar: Tekstil sektörü incelemesi Master's thesis, İstanbul Gelişim Üniversitesi Lisansüstü Eğitim Enstitüsü, 2023.
• İ. Demirbağ, Üç boyutlu sanal dünyalar. Açıköğretim Uygulamaları ve Araştırmaları Dergisi, 6 (4), 97-112, 2020.