Video Verilerinde Bulunan Tehlikeli Nesnelerin Derin Öğrenme Yöntemleri ile Tespiti Üzerine Derleme

Year 2023, Volume: 16 Issue: 1, 60 - 80, 29.06.2023

Ayşe Berika Varol Malkoçoğlu , Rüya Şamlı

https://doi.org/10.54525/tbbmd.1184322

Abstract

Bilgisayarla görme tekniklerinden biri olan nesne saptaması son yıllarda hem akademik hem de ticarî potansiyeli sayesinde büyük ilgi görmektedir. Günümüzde teknolojinin gelişimi ile birlikte güvenlik ya da kişisel amaçlarla çekilen video görüntülerinin artması ve donanım elemanlarının gelişmesi, ihtiyaç duyulan kaynaklara erişimi kolaylaştırmış dolayısıyla nesne saptama sistemlerinin gelişimini hızlandırmıştır. Bu alanda yaya saptaması, yüz tanıma gibi bazı klasikleşmiş konularda çok sayıda çalışma bulunmaktadır. Fakat bu çalışmada farklı nesne gruplarının getirdiği zorlukları gözlemlemek adına tehlikeli nesneler üzerine yapılan ve güvenlik güçlerine yardımcı sistemlerin tasarlanmasına katkı sağlayan çalışmalar araştırılıp derlenmiştir. Çalışmalarda kullanılan nesne saptama yöntemleri geleneksel yöntemler ve derin öğrenme tabanlı modern yöntemler olarak iki kısımda incelenmiş olup avantajları ve dezavantajları tartışılmıştır. Ayrıca literatürdeki eksiklikler belirlenip, gelecekteki çalışmalar için araştırmacılara yönergeler sunulmuştur.

Keywords

Nesne tespiti, nesne algılama, tehlikeli nesneler, derin öğrenme

Review on Detection of Dangerous Objects in Video Data using Deep Learning Methods

Abstract

Object detection, which is one of the computer vision techniques, has been very interested in both academic and commercial potential in recent years. Today, the development of technology, combined with the increased video images for security or personal purposes, and the development of hardware elements, made it easier to access the resources needed, thereby accelerating the development of object detection systems. There are many studies in some classics such as pedestrian detection, face recognition etc. in this area. However, this studies on dangerous objects and contributing to the design of safety-aid systems have been researched and compiled to observe the challenges of different groups of objects in the study. The methods of object detection used in the studies have been studied in two parts as traditional methods and deep learning modern methods, discussing the advantages and disadvantages. In addition, deficiencies in the literature have been identified and guidelines have been provided to researchers for future studies.

Keywords

Object detection, object recognition, dangerous objects, deep learning

References

• Umut vakfı, http://www.umut.org.tr/ (22.08.2022)
• UNIDIR, https://unidir.org/ (22.08.2022)
• United Nation, https://www.un.org/disarmament/ (22.08.2022)
• Piza, E. L., Welsh, B. C., Farrington, D. P., & Thomas, A. L., “CCTV surveillance for crime prevention: A 40‐year systematic review with meta‐analysis”, Criminology & Public Policy, 18(1):135-159, (2019)
• Cohen, N., Gattuso, J. & MacLennan-Brown, K., “CCTV operational requirements manual 2009”, St. Albans: Home Office Scientific Development Branch, (2009)
• Tickner, A. H., & Poulton, E. C., “Monitoring up to 16 synthetic television pictures showing a great deal of movement”, Ergonomics, 16(4):381-401, (1973)
• Darker, I., Gale, A., Ward, L., & Blechko, A., “Can CCTV reliably detect gun crime?”, In 2007 41st Annual IEEE International Carnahan Conference on Security Technology, 264-271, (2007)
• Zou, Z., Shi, Z., Guo, Y., & Ye, J., “Object detection in 20 years: A survey”, arXiv preprint arXiv:1905.05055, (2019)
• Zhao, Z. Q., Zheng, P., Xu, S. T., & Wu, X., “Object detection with deep learning: A review”, IEEE transactions on neural networks and learning systems, 30(11):3212-3232, (2019)
• Xiao, Y., Tian, Z., Yu, J., Zhang, Y., Liu, S., Du, S., & Lan, X., “A review of object detection based on deep learning”, Multimedia Tools and Applications, 79(33):23729-23791, (2020)
• [Brunetti, A., Buongiorno, D., Trotta, G. F., & Bevilacqua, V., “Computer vision and deep learning techniques for pedestrian detection and tracking: A survey”, Neurocomputing, 300:17-33, (2018)
• Wu, X., Kumar, V., Ross Quinlan, J. et al., “Top 10 algorithms in data mining”, Knowledge and information systems, 14:1–37, (2008)
• Viola, P., & Jones, M.,” Rapid object detection using a boosted cascade of simple features”, In Proceedings of the 2001 IEEE computer society conference on computer vision and pattern recognition. CVPR 2001, 1: I-I, (2001)
• Chen, C., Seff, A., Kornhauser, A., & Xiao, J., “Deepdriving: Learning affordance for direct perception in autonomous driving”, In Proceedings of the IEEE international conference on computer vision, 2722-2730, (2015)
• Chen, Y., Zhao, D., Lv, L., & Zhang, Q., “Multi-task learning for dangerous object detection in autonomous driving”, Information Sciences, 432:559-571, (2018)
• Chen, X., Kundu, K., Zhang, Z., Ma, H., Fidler, S., & Urtasun, R., “Monocular 3d object detection for autonomous driving”, In Proceedings of the IEEE conference on computer vision and pattern recognition, 2147-2156, (2016)
• Ren, S., He, K., Girshick, R., & Sun, J., “Faster r-cnn: Towards real-time object detection with region proposal networks”, Advances in neural information processing systems, 28:91-99, (2015)
• Brunetti, A., Buongiorno, D., Trotta, G. F., & Bevilacqua, V., “Computer vision and deep learning techniques for pedestrian detection and tracking: A survey” Neurocomputing, 300:17-33, (2018)
• Yang, B., Huang, C., & Nevatia, R., “Learning affinities and dependencies for multi-target tracking using a CRF model”, In CVPR 2011, 1233-1240, (2011)
• Wojke, N., Bewley, A., & Paulus, D., “Simple online and realtime tracking with a deep association metric”, In 2017 IEEE international conference on image processing (ICIP), 3645-3649, (2017)
• Yang, Z., & Nevatia, R., “A multi-scale cascade fully convolutional network face detector”, In 2016 23rd International Conference on Pattern Recognition (ICPR), 633-638, (2016)
• Coşkun, M., Uçar, A., Yildirim, Ö., & Demir, Y., “Face recognition based on convolutional neural network”, In 2017 International Conference on Modern Electrical and Energy Systems (MEES), 376-379, (2017)
• Kamencay, P., Benco, M., Mizdos, T., & Radil, R., “A new method for face recognition using convolutional neural network”, Advances in Electrical and Electronic Engineering, 15(4):663-672, (2017)
• Salari, A., Djavadifar, A., Liu, X. R., & Najjaran, H., “Object recognition datasets and challenges: A review”, Neurocomputing, (2022)
• Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., ... & Fei-Fei, L., “Imagenet large scale visual recognition challenge”, International journal of computer vision, 115(3):211-252, (2015)
• Karagiannakos S., “Localization and Object Detection with Deep Learning”, Towards Data Science, (2019)
• Stanford University, “Lecture 11: Detection and Segmentation”, Stanford University, (2019)
• Gürbüz, M. E., & Gangal, A., “Döndürülmüş Kayan Pencereler Kullanarak İyleştirilmiş Hibrid Nesne Tespit Yöntemi “, Eleco 2014 Elektrik – Elektronik – Bilgisayar ve Biyomedikal Mühendisliği Sempozyumu, 573-577, (2014)
• Lienhart, R., & Maydt, J., “An extended set of haar-like features for rapid object detection”, In Proceedings. international conference on image processing, 1: I-I, (2002)
• Papageorgiou, C. P., Oren, M., & Poggio, T., “A general framework for object detection”, In Sixth International Conference on Computer Visio, 555-562, (1998)
• Lowe, D. G., “Object recognition from local scale-invariant features”, In Proceedings of the seventh IEEE international conference on computer vision, 2:1150-1157, (1999)
• Dalal, N., & Triggs, B., “Histograms of oriented gradients for human detection”, In 2005 IEEE computer society conference on computer vision and pattern recognition (CVPR'05), 1: 886-893, (2005)
• Emrullah, A. C. A. R., & Özerdem, M. S., “Tarımsal imge dokularından HOG algoritması ile öznitelik çıkarımı ve öznitelik tabanlı toprak neminin tahmini” Anatolian Science-Bilgisayar Bilimleri Dergisi, 1(1):1-7, (2016)
• Felzenszwalb, P. F., Girshick, R. B., McAllester, D., & Ramanan, D., “Object detection with discriminatively trained part-based models”, IEEE transactions on pattern analysis and machine intelligence, 32(9):1627-1645, (2010)
• Berwick R. & Idiot V., “An Idiot’s guide to Support vector machines (SVMs)”, MIT, (2022)
• Vapnik, V. N., “The Nature of Statistical Learning Theory (Second edition)”, Springer Science & Business Media, 314, New York, (1995)
• Hinton, G. E., “Deep Belief Works”, Scholarpedia, 4:5, (2009)
• Krizhevsky, A., Sutskever, I., & Hinton, G. E., “Imagenet classification with deep convolutional neural networks”, In Advances in neural information processing systems. In Advances in Neural İnformation Processing Systems 25 (NIPS 2012), 25(1):1097-1105, (2012)
• Girshick, R., Donahue, J., Darrell, T., & Malik, J., “Rich feature hierarchies for accurate object detection and semantic segmentation”, In Proceedings of the IEEE conference on computer vision and pattern recognition, 580-587, (2014)
• Sermanet P., Eigen D., Zhang X., Mathieu M., Fergus R., and LeCun Y., “Overfeat: Integrated recognition, localization and detection using convolutional networks”, 2nd International Conference on Learning Representations, (2014)
• Varol Malkoçoğlu, A. B., “Akut lenfoblastik lösemi hücrelerinin derin öğrenme yöntemleri ile sınıflandırılması”, Yüksek lisans tezi, Ondokuz Mayıs Üniversitesi, (2020)
• Popescu, A., & Coatrıeux, G., “Large-Scale Object Detection for Social Media User’s Visual Privacy Protection”, Master Thesis, Paris-Saclay University, (2020)
• Du, L., Zhang, R., & Wang, X., “Overview of two-stage object detection algorithms”, In Journal of Physics: Conference Series, 1544(1):1-6, (2020)
• Uijlings, J. R., Van De Sande, K. E., Gevers, T., & Smeulders, A. W., “Selective search for object recognition”, International journal of computer vision, 104(2):154-171, (2013)
• Mohan S., “6 Different Types of Object Detection Algorithms in Nutshell”, Machine Learning Knowledge, (2020)
• Girshick, R., Fast R-CNN. In Proceedings of the IEEE international conference on computer vision, 1440-1448, (2015)
• He, K., Zhang, X., Ren, S., & Sun, J., “Spatial pyramid pooling in deep convolutional networks for visual recognition”, IEEE transactions on pattern analysis and machine intelligence, 37(9):1904-1916, (2015)
• Erdem K., “Understanding Region of Interest (RoI Pooling)”, (2020)
• Ren, S., He, K., Girshick, R., & Sun, J, “Faster R-CNN: Towards real-time object detection with region proposal networks”, Advances in neural information processing systems, 28, (2015)
• Dai, J., Li, Y., He, K., & Sun, J., “R-FCN: Object detection via region-based fully convolutional networks”, Advances in neural information processing systems, 29, (2016)
• He, K., Gkioxari, G., Dollár, P., & Girshick, R., “Mask R-CNN.” In Proceedings of the IEEE international conference on computer vision, 2961-2969, (2017)
• Bozdağ, Z., “Histopatolojik Görüntülerde Tümör Bölütlenmesi”, Doktora Tezi, İnönü Üniversitesi, (2021)
• Redmon, J., Divvala, S., Girshick, R., & Farhadi, A., “You only look once: Unified, real-time object detection”, In Proceedings of the IEEE conference on computer vision and pattern recognition, 779-788, (2016)
• Redmon, J., & Farhadi, A., “YOLO9000: better, faster, stronger.” In Proceedings of the IEEE conference on computer vision and pattern recognition, 7263-7271, (2017)
• Redmon, J., & Farhadi, A., “Yolov3: An incremental improvement”, Cornell University, (2018)
• Lin, T. Y., Dollár, P., Girshick, R., He, K., Hariharan, B., & Belongie, S., “Feature pyramid networks for object detectio”, In Proceedings of the IEEE conference on computer vision and pattern recognition, 2117-2125, (2017)
• Bochkovskiy, A., Wang, C. Y., & Liao, H. Y. M., “Yolov4: Optimal speed and accuracy of object detection”, arXiv e-prints, arXiv-2004, (2020)
• Liu, S., Qi, L., Qin, H., Shi, J., & Jia, J., “Path aggregation network for instance segmentation”, In Proceedings of the IEEE conference on computer vision and pattern recognition, 8759-8768, (2018)
• Aly, G. H., Marey, M. A. E. R., El-Sayed Âmin, S., & Tolba, M. F.,” YOLO V3 and YOLO V4 for masses detection in mammograms with resnet and inception for masses classification”, In International Conference on Advanced Machine Learning Technologies and Applications, 145-153, (2021)
• Sozzi, M., Cantalamessa, S., Cogato, A., Kayad, A., & Marinello, F.,” Automatic bunch detection in white grape varieties using YOLOv3, YOLOv4, and YOLOv5 deep learning algorithms”, Agronomy, 12(2):319, (2022)
• Rahman, E. U., Zhang, Y., Ahmad, S., Ahmad, H. I., & Jobaer, S., “Autonomous vision-based primary distribution systems porcelain insulators inspection using UAVs”, Sensors, 21(3):974, (2021)
• Ge, Z., Liu, S., Wang, F., Li, Z., & Sun, J., “Yolox: Exceeding yolo series in 2021”, arXiv preprint, (2021)
• Nepal, U., & Eslamiat, H., “Comparing YOLOv3, YOLOv4 and YOLOv5 for autonomous landing spot detection in faulty UAVs”, Sensors, 22(2):464, (2022)
• Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S., Fu, C. Y., & Berg, A. C., “Ssd: Single shot multibox detector”, In European conference on computer vision, 21-37, (2016)
• Szegedy, C., Reed, S., Erhan, D., Anguelov, D., & Ioffe, S., “Scalable, high-quality object detection”, arXiv preprint, arXiv:1412.1441, (2014)
• Uner, M. K., Ramac, L. C., Varshney, P. K., & Alford, M. G., “Concealed weapon detection: an image fusion approach”, In Investigative image processing, 2942:123-132. (1997)
• Sheen, D. M., McMakin, D. L., & Hall, T. E., “Three-dimensional millimeter-wave imaging for concealed weapon detection”, IEEE Transactions on microwave theory and techniques, 49(9):1581-1592, (2001)
• Xue, Z., Blum, R. S., & Li, Y. “Fusion of visual and IR images for concealed weapon detection”, In Proceedings of the Fifth International Conference on Information Fusion. FUSION 2002, 2:1198-1205, (2002)
• Blum, R., Xue, Z., Liu, Z., & Forsyth, D. S., “Multisensor concealed weapon detection by using a multiresolution mosaic approach” In IEEE 60th Vehicular Technology Conference, 2004. 7:4597-4601, (2004)
• Upadhyay, E. M., & Rana, N. K., “Exposure fusion for concealed weapon detection”, In 2014 2nd International Conference on Devices, Circuits and Systems (ICDCS), 1-6, (2014)
• O'reilly, D., Bowring, N., & Harmer, S., “Signal processing techniques for concealed weapon detection by use of neural networks”, In 2012 IEEE 27th Convention of Electrical and Electronics Engineers in Israel, 1-4, (2012)
• Darker, I. T., Gale, A. G., & Blechko, A., “CCTV as an automated sensor for firearms detection: Human-derived performance as a precursor to automatic recognition”, In Unmanned/Unattended Sensors and Sensor Networks V, 7112:208-219, (2008)
• Grega, M., Łach, S., & Sieradzki, R., “Automated recognition of firearms in surveillance video”, In 2013 IEEE International Multi-Disciplinary Conference on Cognitive Methods in Situation Awareness and Decision Support (CogSIMA), 45-50, (2013).
• Tiwari, R. K., & Verma, G. K., “A computer vision based framework for visual gun detection using harris interest point detector”, Procedia Computer Science, 54:703-712, (2015).
• Tiwari, R. K., & Verma, G. K., “A computer vision based framework for visual gun detection using SURF”, In 2015 International Conference on Electrical, Electronics, Signals, Communication and Optimization (EESCO), 1-5, (2015).
• Grega, M., Matiolański, A., Guzik, P., & Leszczuk, M., “Automated detection of firearms and knives in a CCTV image”, Sensors, 16(1):47, (2016).
• Vajhala, R., Maddineni, R., & Yeruva, P. R., “Weapon detection in surveillance camera images”, Electrical Engineering, (2016).
• Buckchash, H., & Raman, B., “A robust object detector: application to detection of visual knives”, In 2017 IEEE International Conference on Multimedia & Expo Workshops (ICMEW), 633-638, (2017).
• Lai, J., & Maples, S., “Developing a real-time gun detection classifier”, Course: CS231n, Stanford University, (2017).
• Verma, G. K., & Dhillon, A., “A handheld gun detection using faster r-cnn deep learning”, In Proceedings of the 7th international conference on computer and communication technology, 84-88, (2017).
• Olmos, R., Tabik, S., & Herrera, F. “Automatic handgun detection alarm in videos using deep learning”, Neurocomputing, 275:66-72, (2018).
• Akcay, S., Kundegorski, M. E., Willcocks, C. G., & Breckon, T. P., “Using deep convolutional neural network architectures for object classification and detection within x-ray baggage security imagery”, IEEE transactions on information forensics and security, 13(9):2203-2215, (2018).
• Singleton, M., Taylor, B., Taylor, J., & Liu, Q. “Gun identification using tensorflow”, In International Conference on Machine Learning and Intelligent Communications, 3-12, (2018).
• Gelana, F., & Yadav, A., Firearm detection from surveillance cameras using image processing and machine learning techniques, In Smart innovations in communication and computational sciences, 25-34, (2019).
• Romero, D., & Salamea, C., “Convolutional models for the detection of firearms in surveillance videos”, Applied Sciences, 9(15):2965, (2019).
• Fernandez-Carrobles, M., Deniz, O., & Maroto, F., “Gun and knife detection based on faster R-CNN for video surveillance”, In Iberian conference on pattern recognition and image analysis, 441-452, (2019).
• Iqbal, J., Munir, M. A., Mahmood, A., Ali, A. R., & Ali, M., “Orientation aware object detection with application to firearms”, arXiv preprint arXiv:1904.10032, 22. (2019).
• de Azevedo Kanehisa, R. F., & de Almeida Neto, A., “Firearm Detection using Convolutional Neural Networks”, 11th International Conference on Agents and Artificial Intelligence (ICAART), 2: 707-714, (2019)
• González, J. L. S., Zaccaro, C., Álvarez-García, J. A., Morillo, L. M. S., & Caparrini, F. S., “Real-time gun detection in CCTV: An open problem”, Neural networks, 132:297-308, (2020)
• Pérez-Hernández, F., Tabik, S., Lamas, A., Olmos, R., Fujita, H., & Herrera, F., “Object detection binary classifiers methodology based on deep learning to identify small objects handled similarly: Application in video surveillance”, Knowledge-Based Systems, 194, (2020)
• Raturi, G., Rani, P., Madan, S., & Dosanjh, S., “ADoCW: An automated method for detection of concealed weapon”, In 2019 Fifth International Conference on Image Information Processing (ICIIP), 181-186, (2019)
• Noor, W. E. I. B. W., & Isa, N. M., “Object Detection: Harmful Weapons Detection using YOLOv4”, In 2021 IEEE Symposium on Wireless Technology & Applications (ISWTA), 63-70, (2021)
• Narejo, S., Pandey, B., Rodriguez, C., & Anjum, M. R., “Weapon detection using YOLO V3 for smart surveillance system”, Mathematical Problems in Engineering, (2021)
• Hashmi, T. S. S., Haq, N. U., Fraz, M. M., & Shahzad, M., “Application of Deep Learning for Weapons Detection in Surveillance Videos”, In 2021 International Conference on Digital Futures and Transformative Technologies (ICoDT2), 1-6, (2021)
• Kayalvizhi, R., Malarvizhi, S., Choudhury, S. D., Topkar, A., & Vijayakumar, P., “Detection of sharp objects using deep neural network based object detection algorithm”, In 2020 4th International Conference on Computer, Communication and Signal Processing (ICCCSP), 1-5, (2020)
• Bhatti, M. T., Khan, M. G., Aslam, M., & Fiaz, M. J., “Weapon detection in real-time cctv videos using deep learning”, IEEE Access, 9:34366-34382, (2021)
• Salido, J., Lomas, V., Ruiz-Santaquiteria, J., & Deniz, O., “Automatic handgun detection with deep learning in video surveillance images”, Applied Sciences, 11(13):6085, (2021)
• Iqbal, M. J., Iqbal, M. M., Ahmad, I., Alassafi, M. O., Alfakeeh, A. S., & Alhomoud, A. “Real-Time Surveillance Using Deep Learning”, Security and Communication Networks, (2021)
• Sivakumar, P., “Real Time Crime Detection Using Deep Learning Algorithm”, In 2021 International Conference on System, Computation, Automation and Networking (ICSCAN), 1-5, (2021)
• Kaya, V., Tuncer, S., & Baran, A., “Detection and classification of different weapon types using deep learning”, Applied Sciences, 11(16):7535, (2021)
• Bushra, S. N., Shobana, G., Maheswari, K. U., & Subramanian, N., “Smart Video Survillance Based Weapon Identification Using Yolov5”, In 2022 International Conference on Electronic Systems and Intelligent Computing (ICESIC), 351-357, (2022)