DERİN ÖĞRENMEDE KULLANILAN OPTİMİZASYON ALGORİTMALARININ UYGULANMASI VE KIYASLANMASI

Year 2020, Volume: 5 Issue: 2, 90 - 98, 01.12.2020

Ebubekir Seyyarer , Faruk Ayata , Taner Uçkan , Ali Karci

Abstract

Deep learning (multi layer artificial neural network) is a sub-branch of machine learning that enables learning from large data sets with its layered architecture. Increased processor speeds and storage, inexpensive computer hardware and machine learning gains are factors that boost deep learning development. Gradient Descent based optimization algorithms(Stochastic gradient descent (SGD), Momentum, Adam, Adagrad, RMSProp and Adadelta) used to minimize error in deep learning are also part of this success.
In this study, deep learning and optimization algorithms were applied to the caltech 101 data set, which is an international data set. Relu was used as activation function and cross entropy was preferred as loss function. Images are resized to 64x64. Each time the program is run, a random six category image is taken and 100 iterations are executed. Different results were obtained with optimization algorithms and these results were analyzed. The success rates in classification were observed as SGD: 64.5%, Momentum: 85.56%, Adam: 92.31%, Adagrad: 71.25%, RMSProp: 40.26% and Adadelta: 86.88%.

Keywords

Optimization Algorithms, Deep Learning, Artifical Neural Networks

References

• Akca, M.F., (2020). https://medium.com/deep-learning-turkiye/gradient-descent-nedir-3ec6afcb9900. Erişim Tarihi: 12.03.2020.
• Alpaydın, E., (2018). “Yapay Öğrenme”. Boğaziçi Üniversitesi Yayınevi, İstanbul.
• Anadolu, B., (2019). Dijital Hikaye Anlatıcılığı Bağlamında Yapay Zekanın Sinemaya Etkisi: Sunspring ve It’s No Game Filmlerinin Analizi. Erciyes İletişim Dergisi, (1), 39-56.
• Bosch, A., Zisserman, A., & Munoz, X., (2007). Image classification using random forests and ferns. In 2007 IEEE 11th international conference on computer vision (pp. 1-8). Ieee.
• Cebeci, H., (2019). “Yapay Zeka ve Derin Öğrenme A-Z™: Tensorflow”. Erişim Tarihi: 17.11.2019.
• Dogo, E. M., Afolabi, O. J., Nwulu, N. I., Twala, B. and Aigbavboa, C. O., (2018). “A Comparative Analysis of Gradient Descent-Based Optimization Algorithms on Convolutional Neural Networks.” 2018 International Conference on Computational Techniques, Electronics and Mechanical Systems (CTEMS). IEEE.
• Fei-Fei, L., Fergus, R. ve Perona, P., (2006). One-Shot learning of object categories. IEEE Trans. Pattern Recognition and Machine Intelligence. In press. vol. 28, no. 4, pp. 594-611, April 2006, doi: 10.1109/TPAMI.79.
• Fortuner B., (2019). https://github.com/bfortuner/ml-glossary/blob/master/docs/loss_functions.rst. Erişim Tarihi: (07.05.2019)
• Frome, A., Singer, Y., & Malik, J., (2007). Image retrieval and classification using local distance functions. In Advances in neural information processing systems (pp. 417-424).
• Karim R., (2018). https://towardsdatascience.com/10-gradient-descent-optimisation-algorithms-86989510b5e9 . Erişim Tarihi: 05.05.2019.
• Kurt, İ., (2019). Makine öğrenmesi yöntemleri kullanılarak sesin müzikal öznitelikleri ile parkinson hastalığının tespiti (Master's thesis, Trakya Üniversitesi Fen Bilimleri Enstitüsü).
• Rajakumaran, G., Venkataraman, N. & Mukkamala, R.R., (2020). “Denial of Service Attack Prediction Using Gradient Descent Algorithm.” SN Computer Science 1.: 45.
• Ruder S., (2016). https://ruder.io/optimizing-gradient-descent/. Erişim Tarihi: 02.02.2020.
• Seyyarer, A., Aydın, T., (2017). Değişmez Momentler Kullanarak İçerik Tabanlı Görüntü Erişim Sistemi ve İmge Sınıflandırma Yöntemlerinin Karşılaştırılması. Anatolian Science - Bilgisayar Bilimleri Dergisi, 2 (1), 1-9. Retrieved from http://dergipark.org.tr/bbd/issue/30846/333815.
• Vani, S., ve Rao, T.V.M.,( 2019). “An Experimental Approach towards the Performance Assessment of Various Optimizers on Convolutional Neural Network.” 2019 3rd International Conference on Trends in Electronics and Informatics (ICOEI). IEEE.
• Yazan, E., ve Talu, M.F., (2017). “Comparison of the stochastic gradient descent based optimization techniques.” Artificial Intelligence and Data Processing Symposium (IDAP), 2017 International. IEEE,
• Yılmaz, A.,( 2017). “Yapay Zeka”, Kodlab Yayınevi, İstanbul.
• Zeiler, M.D., (2012). “ADADELTA: an adaptive learning rate method”. arXiv preprint arXiv:1212.5701.