Feature-Based Performance Comparison of Machine Learning Algorithms for Phishing Detection through Uniform Resource Locator

Year 2022, Volume: 25 Issue: 3, 1261 - 1270, 01.10.2022

Taki Savaş , Serkan Savaş

https://doi.org/10.2339/politeknik.1035286

Cited By: 4

Abstract

Recently, phishing attacks are very common. Such attacks are carried out with the aim of obtaining personal information of individuals or defrauding individuals. There are multiple types of phishing attacks. One of these types is the common attacks carried out through the uniform resource locator (URL). The purpose of this study is to classify whether URL addresses are malicious or not using different machine learning algorithms. Eight different machine learning algorithms including support vector machines, random forest, Gaussian Naive Bayes, logistic regression, k-nearest neighbor, decision trees, multilayer perceptrons and XGBoost algorithms were used in the study. Data were obtained from USOM, Alexa, and Phishtank to be used for training and testing purposes. Feature extraction was performed by applying various data pre-processing steps to these data. As a result of the research, the accuracy of 99.8% in more than one model has been achieved, and the success of machine learning algorithms in this area has been proven.

Keywords

Cybersecurity, phishing, machine learning, domain, cyber-attack detection

Tekdüzen Kaynak Bulucu Yoluyla Kimlik Avı Tespiti için Makine Öğrenmesi Algoritmalarının Özellik Tabanlı Performans Karşılaştırması

Abstract

Günümüzde kimlik avı (oltalama/phishing) saldırılarına çok sık rastlanmaktadır. Bu tür saldırılar insanların kişisel bilgilerini ele geçirmek ya da insanları dolandırmak amacıyla gerçekleştirilmektedir. Kimlik avı saldırılarının birden fazla türü bulunmaktadır. Bu türlerden birisi de tekdüzen kaynak bulucu (uniform resource locater – URL) yoluyla gerçekleştirilen ve yaygın olarak rastlanılan saldırılardır. Bu çalışmanın amacı, URL adreslerinin farklı makine öğrenmesi algoritmaları kullanarak zararlı olup olmadığını sınıflandırmaktır. Çalışmada destek vektör makineleri, rastgele orman, Gauss Naive Bayes, lojistik regresyon, k-en yakın komşu, karar ağaçları, çok katmanlı algılayıcılar ve XGBoost algoritmaları olmak üzere sekiz farklı makine öğrenmesi algoritması kullanılmıştır. Eğitim ve test amaçlı kullanılmak üzere USOM, Alexa ve Phishtank üzerinden veriler elde edilmiştir. Bu verilere çeşitli veri ön-işleme adımları uygulanarak özellik çıkarımı gerçekleştirilmiştir. Araştırma sonucunda birden fazla modelde %99.8 doğruluk oranına ulaşılarak, makine öğrenmesi algoritmalarının bu alandaki başarımı kanıtlanmıştır.

Keywords

Siber güvenlik, kimlik avı, makine öğrenmesi, internet alan adı, siber saldırı tespiti

References

• [1] Savaş, S. and Topaloğlu, N., “Data analysis through social media according to the classified crime”, Turkish Journal of Electrical Engineering & Computer Sciences, 27(1): 407-420, (2019).
• [2] FBI, “Internet Crime Report”, (2020).
• [3] Korkmaz, A. and Büyükgöze, S., “Sahte Web Sitelerinin Sınıflandırma Algoritmaları İle Tespit Edilmesi”, Avrupa Bilim ve Teknoloji Dergisi, (16): 826-833, (2019).
• [4] Sönmez, Ü., “Bilişim Sistemleri Aracılığıyla Dolandırıcılık Suçu”, Dicle Üniversitesi Adalet Meslek Yüksekokulu Dicle Adalet Dergisi, 1(2): 47-68, (2017).
• [5] Bassett, G., et al., “Data Breach Investigations Report (DBIR 2021)”, (2021).
• [6] Rosenthal, M. Must-Know Phishing Statistics: Updated 2021. 2021 [cited 2021; Available from: https://www.tessian.com/blog/phishing-statistics-2020/.
• [7] McCarthy, J., et al., “A proposal for the Dartmouth summer conference on artificial intelligence”, Dartmouth Workshop, (1955).
• [8] Jain, A.K. and Gupta, B. B., “PHISH-SAFE: URL Features-Based Phishing Detection System Using Machine Learning. in Cyber Security”, Singapore: Springer Singapore, (2018).
• [9] Kadı, C., “Zararlı Web Sayfalarının Tespiti ve Sınıflandırılması için Yeni Bir Sistem Önerisi”, Yüksek Lisans Tezi, Fen Bilimleri Enstitüsü, Gazi University: Ankara, (2018).
• [10] Sanglerdsinlapachai, N. and Rungsawang, A., “Using domain top-page similarity feature in machine learning-based web phishing detection”, 2010 Third International Conference on Knowledge Discovery and Data Mining, IEEE, (2010).
• [11] Shirazi, H., Bezawada, B., and Ray, I., “"Kn0w Thy Doma1n Name": Unbiased Phishing Detection Using Domain Name Based Features”, in Proceedings of the 23nd ACM on Symposium on Access Control Models and Technologies, Association for Computing Machinery: Indianapolis, Indiana, USA, 69–75, (2018).
• [12] Jain, A.K. and Gupta, B. B., “A machine learning based approach for phishing detection using hyperlinks information”, Journal of Ambient Intelligence and Humanized Computing, 10(5): 2015-2028, (2019).
• [13] Zhang, D., et al., “A domain-feature enhanced classification model for the detection of Chinese phishing e-Business websites”, Information & Management, 51(7): 845-853, (2014).
• [14] Alkhozae, M.G. and Batarfi, O. A., “Phishing websites detection based on phishing characteristics in the webpage source code”, International Journal of Information and Communication Technology Research, 1(6), (2011).
• [15] Hong, J., et al., “Phishing url detection with lexical features and blacklisted domains”, in Adaptive Autonomous Secure Cyber System, Springer, 253-267, (2020).
• [16] Christou, O., et al. “Phishing url detection through top-level domain analysis: A descriptive approach”, in 6th ICISSP, arXiv (2020).
• [17] Moghimi, M. and Varjani, A. Y., “New rule-based phishing detection method”, Expert Systems with Applications, 53: 231-242, (2016).
• [18] Sahingoz, O.K., et al., “Machine learning based phishing detection from URLs”, Expert Systems with Applications, 117: 345-357, (2019).
• [19] Koşan, M.A., Yıldız, O., and Karacan, H., “Kimlik avı web sitelerinin tespitinde makine öğrenmesi algoritmalarının karşılaştırmalı analizi”, Pamukkale Üniversitesi Mühendislik Bilimleri Dergisi, 24(2): 276-282, (2018).
• [20] Buyrukoğlu, S. “Improvement of Machine Learning Models’ Performances based on Ensemble Learning for the detection of Alzheimer Disease”, in 2021 6th International Conference on Computer Science and Engineering (UBMK), IEEE, (2021).
• [21] Dalmaz, H., Erdal, E., and Ünver, H. M., “Machine Learning Approaches in Detecting Network Attacks”, in 2021 6th International Conference on Computer Science and Engineering (UBMK), IEEE, (2021).
• [22] Pan, Y. and Ding, X., “Anomaly based web phishing page detection”, in 2006 22nd Annual Computer Security Applications Conference (ACSAC'06), IEEE, (2006).
• [23] Uçar, E., İncetaş, M., and Ucar, M., “A Deep learning approach for detection of malicious URLs”, in 6th International Management Information Systems Conference, (2019).
• [24] Awadh, K. and Akbaş, A., “Intrusion Detection Model Based on TF. IDF and C4. 5 Algorithms”, Politeknik Dergisi, 24(4): 1691-1698, (2021).
• [25] Calp, M. H., “The role of artificial intelligence within the scope of digital transformation in enterprises, in Advanced MIS and digital transformation for increased creativity and innovation in business”, IGI Global, 122-146, (2020).
• [26] Güler, O. and Yücedağ, İ., “Mesleki ortaöğretim öğrencilerinin alan seçimi problemine bulanık mantık temelli yaklaşım”, Hacettepe Üniversitesi Eğitim Fakültesi Dergisi, 32(1): 111-122, (2017).
• [27] Çetin, G. and Karakış, R., “A wiki application for artificial neural network course in engineering education”, in 2012 15th International Conference on Interactive Collaborative Learning (ICL), (2012).
• [28] Akbaş, A., “Machine Learning based Heart Failure Risk Analysis in Python”, in Programming Solutions for Engineering Problems, A. Akbaş, S. Buyrukoğlu, and A. Gökçe, Editors, Nobel Akademik Yayıncılık: Ankara. 89-110, (2021).
• [29] Yılmaz, Y. and Buyrukoğlu, S, “Hybrid Machine Learning Model Coupled with School Closure For Forecasting COVID-19 Cases in the Most Affected Countries”, Hittite Journal of Science and Engineering, 8(2): 123-131, (2021).
• [30] Kaynar, O., et al., “Makine öğrenmesi yöntemleriyle müşteri kaybı analizi”, Cumhuriyet Üniversitesi İktisadi ve İdari Bilimler Dergisi, 18(1): 1-14, (2017).
• [31] Calp, M. H., “İşletmeler için Personel Yemek Talep Miktarının Yapay Sinir Ağları Kullanılarak Tahmin Edilmesi”, Politeknik Dergisi, 22(3):675-686, (2019).
• [32] Cortes, C. and Vapnik, V., “Support-vector networks”, Machine Learning, 20(3):273-297, (1995).
• [33] Ho, T. K, “Random decision forests”, in Proceedings of 3rd International Conference on Document Analysis and Recognition, (1995).
• [34] Ho, T.K., “Recognition of handwritten digits by combining independent learning vector quantizations”, in Proceedings of 2nd International Conference on Document Analysis and Recognition (ICDAR'93), IEEE, (1993).
• [35] Patil, T.R. and Sherekar, S. S., “Performance analysis of naive bayes and J48 classification algorithm for data classification”, Journal of Computer Science and Applications, 6(2): 256-261 (2013).
• [36] Savaş, S., “Karotis Arter Intima Media Kalınlığının Derin Öğrenme ile Sınıflandırılması”, Doktora Tezi, Fen Bilimleri Enstitüsü, Gazi University: Ankara, (2019).
• [37] Fix, E. and Hodges, J. L., “Discriminatory analysis. Nonparametric discrimination: Consistency properties”, International Statistical Review/Revue Internationale de Statistique, 57(3): 238-247, (1989).
• [38] Tekerek A., “Support Vector Machine Based Spam SMS Detection”, Politeknik Dergisi, 22(3): 779-784, (2019).
• [39] Kırmızıgül Çalışkan, S. and Soğukpınar, İ., “KxKNN: K-Means ve K En Yakin Komşu Yöntemleri İle Ağlarda Nüfuz Tespiti” EMO Yayınları, 120-24, (2008).
• [40] SPSS. “AnswerTree Algorithm Summary”, [cited 2021, from: https://s2.smu.edu/~mhd/8331f03/AT.pdf, (1999).
• [41] Rosenblatt, F., “The perceptron: a probabilistic model for information storage and organization in the brain”, Psychological review, 65(6): 386, (1958).
• [42] Bulut, F., “Çok Katmanlı Algılayıcılar ile Doğru Meslek Tercihi”, Anadolu University Journal of Science and Technology A-Applied Sciences and Engineering, 17(1): 97-109, (2016).
• [43] Chen, T. and Guestrin, C., “Xgboost: A scalable tree boosting system”, in Proceedings of the 22nd acm sigkdd international conference on knowledge discovery and data mining, (2016).
• [44] Buyrukoğlu, S., “New hybrid data mining model for prediction of Salmonella presence in agricultural waters based on ensemble feature selection and machine learning algorithms”, Journal of Food Safety, 41(4): e12903, (2021).
• [45] Al-Helli, S. and Akbaş, A., “Guided Feature Selection and Dimensionality Reduction Method for IDS Improvement in DDoS Attacks”, in International Conference on Engineering Technologies (ICENTE'20), Konya: Selçuk University, (2020).
• [46] Mohammad, R.M., Thabtah, F., and McCluskey, L., “Phishing websites features. School of Computing and Engineering”, University of Huddersfield, (2015).
• [47] USOM. “Zararlı Bağlantılar”, Available from: https://www.usom.gov.tr/adres. (2021).
• [48] Alexa. “Site Info”, Available from: https://www.alexa.com/siteinfo. (2021).
• [49] PhishTank. “Join the fight against phishing”, Available from: http://data.phishtank.com/data/online-valid.csv. (2021).
• [50] Savaş, S., Topaloğlu, N., Kazcı, Ö., and Koşar, P. N., “Classification of Carotid Artery Intima Media Thickness Ultrasound Images with Deep Learning”, Journal of Medical Systems, 43(8): 273, (2019).
• [51] Savaş, S., Topaloğlu, N., Kazcı, Ö., and Koşar, P. N., “Performance Comparison of Carotid Artery Intima Media Thickness Classification by Deep Learning Methods”, in International Congress on Human-Computer Interaction, Optimization, and Robotic Applications, SETSCI Conference Proceedings: Urgup, Nevşehir, Turkey. 125-131, (2019). doi: https://doi.org/10.36287/setsci.4.5.025
• [52] Arslan, R. S., “Kötücül Web Sayfalarının Tespitinde Doc2Vec Modeli ve Makine Öğrenmesi Yaklaşımı” Avrupa Bilim ve Teknoloji Dergisi, (27): 792-801, (2021).
• [53] Almseidin, M., et al., “Phishing detection based on machine learning and feature selection methods”. International Association of Online Engineering, (2019).
• [54] Özker, U., “İçerik tabanlı oltalama saldırısı tespit sistemi”, Yüksek Lisans Tezi, Lisansüstü Eğitim Enstitüsü, İstanbul Kültür Üniversitesi, (2021).
• [55] İncir, R., “Derin öğrenme yöntemi kullanarak web tabanlı kimlik avı saldırılarının sınıflandırılması”, Yüksek Lisans Tezi, Fen Bilimleri Enstitüsü, Fırat Üniversitesi, (2020).
• [56] Abu-Nimeh, S., et al., “A comparison of machine learning techniques for phishing detection”, in Proceedings of the anti-phishing working groups 2nd annual eCrime researchers summit, Association for Computing Machinery: Pittsburgh, Pennsylvania, USA. 60–69, (2007).
• [57] Chiew, K.L., et al., “A new hybrid ensemble feature selection framework for machine learning-based phishing detection system”, Information Sciences, 484: 153-166, (2019).
• [58] Kalaycı, T. E., “Kimlik hırsızı web sitelerinin sınıflandırılması için makine öğrenmesi yöntemlerinin karşılaştırılması”, Pamukkale Üniversitesi Mühendislik Bilimleri Dergisi, 24(5): 870-878, (2018).