Research Article
BibTex RIS Cite

ÇKKV Yöntemleri İçin Normalizasyon Tekniği Seçimi: Finansal Veri Türlerindeki Değişikliklere Uyum Sağlayabilecek Esnek ve Konjonktürel Bir Çözüm

Year 2023, Volume: 5 Issue: Özel Sayı, 148 - 164, 17.12.2023

Abstract

MCDM metodolojisini çok kriterli bir problemde alternatiflerden en uygun olanı seçmek ve sıralama yapmak amacıyla kullanmak mantıklıdır. İlk karar matrisindeki birbirinden farklı amaçlara sahip kriterler genelde farklı birimlerden oluşan bir yapıya sahip olduğundan bu verileri birimsiz bir boyutta normalize ederek homojenize etmek gerekir. Öte yandan masum bir dönüştürücü gibi görünen normalizasyon tekniklerinin herhangi bir MCDM yönteminin nihai sıralamasını etkileyebilme potansiyeli önemli bir sorundur. Nitekim bu teknikler genel sıralamayı ve en iyi alternatifin belirlenmesini etkileyebilmektedir. Aslında herhangi bir MCDM yönteminin amacının en iyi olan bir alternatifi önermek olduğu göz önüne alındığında rast gele seçilen bir normalizasyon tekniği karar verici için ciddi bir kalite maliyeti oluşturabilir. Ne var ki normalizasyon yöntemlerinin seçimi için henüz literatürde kesin bir mutabakat yoktur. Bu çalışmada yenilikçi bir bakış açısıyla normalizasyon yöntemlerinin gerçek yaşamı yakalama derecesi ya da üçüncü bir tarafla ilişkisi açısından değerlendirilmesi önerilmektedir. Bu çalışmada ÇKKV yönteminin (normalizasyon sonrası denklemi sabit tutularak) farklı normalizasyon yöntemlerinin üretilen sonuçları nasıl etkiledikleri incelenmiştir. Farklı finansal veri setlerinde test edilen yaklaşımın bulgularına göre genel itibariyle dönemsel olarak en başarılı olan teknik farklıdır. Dolayısıyla veri yapısına bağlı olarak en iyi normalizasyon tekniğinin seçimi, statik değil dinamik bir bakış açısıyla değerlendirilmelidir. Ayrıca bu çalışma, klasik normalizasyon yöntemlerinin yanı sıra sıralama (ranking) temelli dönüştürme fonksiyonunun da kullanılabileceğini net olarak göstermiştir.

References

  • Alkan, T., & Durduran, S. S. (2020). Konut seçimi sürecinin AHP temelli TOPSIS yöntemi ile analizi. Necmettin Erbakan Üniversitesi Fen ve Mühendislik Bilimleri Dergisi, 2(2), 12-21.
  • Aytekin, A. (2021). Comparative Analysis of Normalizatıon Techniques in the Context of MCDM Problems. Decision Making: Applications in Management and Engineering, 4(2), 1-25.
  • Baghla, S., and Bansal, S. (2014). Effect of normalization techniques in VIKOR method for network selection in heterogeneous networks. in 2014 IEEE International Conference on Computational Intelligence and Computing Research (Coimbatore: IEEE), 1–6.
  • Baydaş, M., & Elma, O. E. (2021). An objectıve criteria proposal for the comparison of MCDM and weighting methods in financial performance measurement: An application in Borsa Istanbul. Decision Making: Applications in Management and Engineering, 4(2), 257-279.
  • Baydaş, M., Eren, T., Stević, Ž., Starčević, V., & Parlakkaya, R. (2023). Proposal for an objective binary benchmarking framework that validates each other for comparing MCDM methods through data analytics. PeerJ Computer Science, 9, e1350.
  • Baydaş, M., & Pamučar, D. (2022). Determining objective characteristics of MCDM methods under uncertainty: an exploration study with financial data. Mathematics, 10(7), 1115.
  • Baydaş, M., & Tevfik, Eren. (2021). Finansal performans ölçümünde ÇKKV yöntem seçimi problemine objektif bir yaklaşım: Borsa İstanbul’da bir uygulama. Eskişehir Osmangazi Üniversitesi İktisadi ve İdari Bilimler Dergisi, 16(3), 664-687.
  • Biswas, S., & Pamucar, D. (2021). Combinative Distance based Assessment (CODAS) Framework using Logarithmic Normalization for Multi-Criteria Decision Making. Serbian Journal of Management, 16(2), 321-340.
  • Brigham, E. F., & Houston, J. F. (2019). Fundamentals of financial management 15th.
  • Chakraborty, S., and Yeh, C.-H. (2009). “A simulation comparison of normalization procedures for TOPSIS,” in 2009 International Conference on Computers & Industrial Engineering (IEEE). University of Technology of Troyes.
  • Chatterjee, P., & Chakraborty, S. (2014). Investigating the Effect of Normalization Norms in Flexible Manufacturing Sytem Selection Using Multi-Criteria Decision-Making Methods. Journal of Engineering Science and Technology Review, 7(3), 141-150.
  • Chen, P. (2019). Effects of Normalization on the Entropy-Based TOPSIS Method. Expert Systems With Applications, 136, 33-41.
  • Çelen, A. (2014). Comparative analysis of normalization procedures in TOPSIS method: with an application to Turkish deposit banking market. Informatica 25, 185–208. doi: 10.15388/Informatica.2014.10
  • Ersoy, N. (2022). The Influence of Statistical Normalization Techniques on Performance Ranking Results: The Application of MCDM Method Proposed by Biswas and Saha. International Journal of Business Analytics (IJBAN), 9(5), 1-21.
  • Ghorabaee, K. M., Zavadskas, E. K., Turskis, Z., & Antucheviciene, J. (2016). A new combinative distance-based assessment (CODAS) method for multi-criteria decision-making. Economic Computation & Economic Cybernetics Studies & Research, 50(3).
  • Jafaryeganeh, H., Ventura, M., & Soares, C. (2020). Effect of Normalization Techniques in Multi-criteria Decision Making Methods for the Design of Ship Internal Layout from A Pareto Optimal Set. Structural and Multidisciplinary Optimization, 62(3), 1849-1863.
  • Jahan, A., & Edwards, K. (2015). A State-of-the-Art Survey on the Influence of Normalization Techniques in Ranking: Improving the Materials Selection Process in Engineering Design. Materials and Design, 65, 335-342.
  • Kızıl, E. (2019). Borsada İşlem Gören Şirketlerin Finansal Performansları İle Borsa Performansları Arasındaki İlişki: BİST Taş, Toprak Endeksindeki Çimento Firmaları Üzerine Bir Uygulama. Necmettin Erbakan Üniversitesi Siyasal Bilgiler Fakültesi Dergisi, 1(1), 51-67.
  • Kosareva, N., Aleksandras, K., & Zavadskas, E. (2018). Statistical Analysis of MCDM Data Normalization Methods Using Monte Carlo Approach. The Case of Ternary Estimates Matrix. Economic Computation and Economic Cybernetics Studies and Research / Academy of Economic Studies, 52, 159-175.
  • Lakshmi, T., & Venkatesan, V. (2014). A Comparison of Various Normalization in Techniques for Order Performance by Similarity to Ideal Solution (TOPSIS). International Journal of Computing Algorithm, 03(03), 255-259.
  • Mathew, M., Sahu, S., and Upadhyay, A. K. (2017). Effect of normalization techniques in robot selection using weighted aggregated sum product assessment. Int. J. Innov. Res. Adv. Stud 4, 59–63. Available online at: https://www.ijiras.com/2017/Vol_4-Issue_2/paper_12.pdf
  • Mhlanga, S., & Lall, M. (2022). Influence of Normalization Techniques on Multi-criteria Decision-making Method. Journal of Physics: Conference Series, 2224, 1-13.
  • Milani, A. S., Shanian, A., Madoliat, R., and Nemes, J. A. (2005). The effect of normalization norms in multiple attribute decision making models: a case study in gear material selection. Struct. Multidiscipl. Optimiz. 29, 312–318. doi: 10.1007/s00158-004-0473-1
  • Özdağoğlu, A. (2013a). Farklı Normalizasyon Tekniklerinin TOPSIS'te Karar Verme Sürecine Etkisi. Ege Academic Review, 13(2), 245-257.
  • Özdağoğlu, A. (2013b). Çok Ölçütlü Karar Verme Modellerinde Normalizasyon Tekniklerinin Sonuçlara Etkisi: COPRAS Örneği. Eskişehir Osmangazi Üniversitesi İİBF Dergisi, 8(2), 229-252.
  • Özdağoğlu, A. (2014). Normalizasyon Yöntemlerinin Çok Ölçütlü Karar Verme Sürecine Etkisi - MOORA Yöntemi İncelemesi. Ege Academic Review, 14(2), 283-294.
  • Palczewski, K., and Sałabun, W. (2019). Influence of various normalization methods in PROMETHEE II: an empirical study on the selection of the airport location. Proc. Comput. Sci. 159, 2051–2060. doi: 10.1016/j.procs.2019.09.378
  • Pavlicic, D. (2001). Normalization affects the results of MADM methods, Yugoslav Journal of Operations Research, 11(2): 251–265.
  • Polska, O., Kudermetov, R., Alsayaydeh, J. A. J., & Shkarupylo, V. (2021). QoS-aware Web-services ranking: normalization techniques comparative analysis for LSP method. ARPN Journal of Engineering and Applied Sciences, 16(2), 248-254.
  • Satıcı, S. (2021). Farklı Normalizasyon Tekniklerinin Çok Kriterli Karar Verme Yöntemlerine Etkisi: WASPAS Örneği. The Journal of Business, Economic and Management Research(2), 350-361.
  • Sayar, M. A., SELVİ, H. Z., & BUĞDAYCI, İ. (2019). Suruç Çadırkent Alanının Analitik Hiyerarşi Yöntemiyle Belirlenmesi. Necmettin Erbakan Üniversitesi Fen ve Mühendislik Bilimleri Dergisi, 1(1), 20-31.
  • Stewart, B. (2013). Best-practice EVA: the definitive guide to measuring and maximizing shareholder value. John Wiley & Sons.
  • Vafaei, N., Ribeiro, R. A., and Camarinha-Matos, L. M. (2015). Importance of data normalization in decision making: case study with TOPSIS method. İn ICDSST 2015 Proceedings–The 1st International Conference on Decision Support Systems Technologies, An EWG-DSS Conference (Belgrade).
  • Vafaei, N., Ribeiro, R. A., and Camarinha-Matos, L. M. (2016). Normalization techniques for multi-criteria decision making: analytical hierarchy process case study. in Doctoral Conference on Computing, Electrical and Industrial Systems (Costa de Caparica).
  • Vafaei, N., Ribeiro, R. A., and Camarinha-Matos, L. M. (2018). Selection of normalization technique for weighted average multi-criteria decision making. İn Doctoral Conference on Computing, Electrical and Industrial Systems (Costa de Caparica).
  • Vafaei, N., Ribeiro, R. A., and Camarinha-Matos, L. M. (2020). Selecting normalization techniques for the analytical hierarchy process. in Doctoral Conference on Computing, Electrical and Industrial Systems (Costa de Caparica). Vafaei, N., Ribeiro, R., & Camarinha-Matos, L. (2022). Assessing Normalization Techniques for Simple Additive Weighting Method. Procedia Computer Science, 199, 1229-1236.
  • Wang, Z., Parhi, S. S., Rangaiah, G. P., & Jana, A. K. (2020). Analysis of weighting and selection methods for pareto-optimal solutions of multiobjective optimization in chemical engineering applications. Industrial & Engineering Chemistry Research, 59(33), 14850-14867.
  • Więckowski, J., & Sałabun, W. (2020). How the Normalization of the Decision Matrix Influences the Results in the VIKOR Method? Procedia Computer Science, 176, 2222-2231.
  • Yaakob, A., Gegov, A., & Rahman, S. F. A. (2018). Fuzzy networks with rule base aggregation for selection of alternatives. Fuzzy Sets and Systems, 341, 123-144.
  • Yalcin, N., Bayrakdaroglu, A., & Kahraman, C. (2012). Application of fuzzy multi-criteria decision making methods for financial performance evaluation of Turkish manufacturing industries. Expert systems with applications, 39(1), 350-364.
  • Yang, W. C., Chon, S. H., Choe, C. M., & Yang, J. Y. (2021). Materials selection method using TOPSIS with some popular normalization methods. Engineering Research Express, 3(1), 015020.
  • Zolfani, S., Yazdani, M., Pamucar, D., & Zarate, P. (2020). A VIKOR And TOPSIS Focused Reanalysis of the MADM Methods Based on Logaiıthmic Normalization. Facta Universitatis Series: Mechanical Engineeringis, 18(3), 341-355.

Normalization Technique Selection for MCDM Methods: A Flexible and Conjunctural Solution that can Adapt to Changes in Financial Data Types

Year 2023, Volume: 5 Issue: Özel Sayı, 148 - 164, 17.12.2023

Abstract

It makes sense to use the MCDM methodology to select and rank alternatives for a multi-criteria problem. As it is known, since it is not possible to use criteria consisting of different units in a common calculation, it is necessary to convert them into a unitless dimension. Many alternative normalization techniques have been proposed in the past for this conversion process. On the other hand, normalization techniques that appear to be accurate fair transformers have the potential to affect the final ranking of any MCDM method, and this is a significant problem. As a matter of fact, these alternative techniques can change the best alternative and overall ranking for an MCDM. Therefore, an unconsciously chosen normalization technique may reduce the quality of the findings. However, it cannot be said that there is a consensus on the choice of normalization methods. Previous studies have unanimously stated that normalization methods can affect MCDM results. In this study, from an innovative perspective, the effect of normalization methods on the results is evaluated with a third party, an external constant factor. In other words, we focus on how the normalization technique affects the relationship of MCDM with an external factor. Thus, we want to achieve a fair assessment by choosing a reference point. According to the findings of the approach tested in different financial data sets, the most successful technique may change periodically. Therefore, the selection of the best normalization technique depending on the data structure should be evaluated from a dynamic rather than static perspective.

References

  • Alkan, T., & Durduran, S. S. (2020). Konut seçimi sürecinin AHP temelli TOPSIS yöntemi ile analizi. Necmettin Erbakan Üniversitesi Fen ve Mühendislik Bilimleri Dergisi, 2(2), 12-21.
  • Aytekin, A. (2021). Comparative Analysis of Normalizatıon Techniques in the Context of MCDM Problems. Decision Making: Applications in Management and Engineering, 4(2), 1-25.
  • Baghla, S., and Bansal, S. (2014). Effect of normalization techniques in VIKOR method for network selection in heterogeneous networks. in 2014 IEEE International Conference on Computational Intelligence and Computing Research (Coimbatore: IEEE), 1–6.
  • Baydaş, M., & Elma, O. E. (2021). An objectıve criteria proposal for the comparison of MCDM and weighting methods in financial performance measurement: An application in Borsa Istanbul. Decision Making: Applications in Management and Engineering, 4(2), 257-279.
  • Baydaş, M., Eren, T., Stević, Ž., Starčević, V., & Parlakkaya, R. (2023). Proposal for an objective binary benchmarking framework that validates each other for comparing MCDM methods through data analytics. PeerJ Computer Science, 9, e1350.
  • Baydaş, M., & Pamučar, D. (2022). Determining objective characteristics of MCDM methods under uncertainty: an exploration study with financial data. Mathematics, 10(7), 1115.
  • Baydaş, M., & Tevfik, Eren. (2021). Finansal performans ölçümünde ÇKKV yöntem seçimi problemine objektif bir yaklaşım: Borsa İstanbul’da bir uygulama. Eskişehir Osmangazi Üniversitesi İktisadi ve İdari Bilimler Dergisi, 16(3), 664-687.
  • Biswas, S., & Pamucar, D. (2021). Combinative Distance based Assessment (CODAS) Framework using Logarithmic Normalization for Multi-Criteria Decision Making. Serbian Journal of Management, 16(2), 321-340.
  • Brigham, E. F., & Houston, J. F. (2019). Fundamentals of financial management 15th.
  • Chakraborty, S., and Yeh, C.-H. (2009). “A simulation comparison of normalization procedures for TOPSIS,” in 2009 International Conference on Computers & Industrial Engineering (IEEE). University of Technology of Troyes.
  • Chatterjee, P., & Chakraborty, S. (2014). Investigating the Effect of Normalization Norms in Flexible Manufacturing Sytem Selection Using Multi-Criteria Decision-Making Methods. Journal of Engineering Science and Technology Review, 7(3), 141-150.
  • Chen, P. (2019). Effects of Normalization on the Entropy-Based TOPSIS Method. Expert Systems With Applications, 136, 33-41.
  • Çelen, A. (2014). Comparative analysis of normalization procedures in TOPSIS method: with an application to Turkish deposit banking market. Informatica 25, 185–208. doi: 10.15388/Informatica.2014.10
  • Ersoy, N. (2022). The Influence of Statistical Normalization Techniques on Performance Ranking Results: The Application of MCDM Method Proposed by Biswas and Saha. International Journal of Business Analytics (IJBAN), 9(5), 1-21.
  • Ghorabaee, K. M., Zavadskas, E. K., Turskis, Z., & Antucheviciene, J. (2016). A new combinative distance-based assessment (CODAS) method for multi-criteria decision-making. Economic Computation & Economic Cybernetics Studies & Research, 50(3).
  • Jafaryeganeh, H., Ventura, M., & Soares, C. (2020). Effect of Normalization Techniques in Multi-criteria Decision Making Methods for the Design of Ship Internal Layout from A Pareto Optimal Set. Structural and Multidisciplinary Optimization, 62(3), 1849-1863.
  • Jahan, A., & Edwards, K. (2015). A State-of-the-Art Survey on the Influence of Normalization Techniques in Ranking: Improving the Materials Selection Process in Engineering Design. Materials and Design, 65, 335-342.
  • Kızıl, E. (2019). Borsada İşlem Gören Şirketlerin Finansal Performansları İle Borsa Performansları Arasındaki İlişki: BİST Taş, Toprak Endeksindeki Çimento Firmaları Üzerine Bir Uygulama. Necmettin Erbakan Üniversitesi Siyasal Bilgiler Fakültesi Dergisi, 1(1), 51-67.
  • Kosareva, N., Aleksandras, K., & Zavadskas, E. (2018). Statistical Analysis of MCDM Data Normalization Methods Using Monte Carlo Approach. The Case of Ternary Estimates Matrix. Economic Computation and Economic Cybernetics Studies and Research / Academy of Economic Studies, 52, 159-175.
  • Lakshmi, T., & Venkatesan, V. (2014). A Comparison of Various Normalization in Techniques for Order Performance by Similarity to Ideal Solution (TOPSIS). International Journal of Computing Algorithm, 03(03), 255-259.
  • Mathew, M., Sahu, S., and Upadhyay, A. K. (2017). Effect of normalization techniques in robot selection using weighted aggregated sum product assessment. Int. J. Innov. Res. Adv. Stud 4, 59–63. Available online at: https://www.ijiras.com/2017/Vol_4-Issue_2/paper_12.pdf
  • Mhlanga, S., & Lall, M. (2022). Influence of Normalization Techniques on Multi-criteria Decision-making Method. Journal of Physics: Conference Series, 2224, 1-13.
  • Milani, A. S., Shanian, A., Madoliat, R., and Nemes, J. A. (2005). The effect of normalization norms in multiple attribute decision making models: a case study in gear material selection. Struct. Multidiscipl. Optimiz. 29, 312–318. doi: 10.1007/s00158-004-0473-1
  • Özdağoğlu, A. (2013a). Farklı Normalizasyon Tekniklerinin TOPSIS'te Karar Verme Sürecine Etkisi. Ege Academic Review, 13(2), 245-257.
  • Özdağoğlu, A. (2013b). Çok Ölçütlü Karar Verme Modellerinde Normalizasyon Tekniklerinin Sonuçlara Etkisi: COPRAS Örneği. Eskişehir Osmangazi Üniversitesi İİBF Dergisi, 8(2), 229-252.
  • Özdağoğlu, A. (2014). Normalizasyon Yöntemlerinin Çok Ölçütlü Karar Verme Sürecine Etkisi - MOORA Yöntemi İncelemesi. Ege Academic Review, 14(2), 283-294.
  • Palczewski, K., and Sałabun, W. (2019). Influence of various normalization methods in PROMETHEE II: an empirical study on the selection of the airport location. Proc. Comput. Sci. 159, 2051–2060. doi: 10.1016/j.procs.2019.09.378
  • Pavlicic, D. (2001). Normalization affects the results of MADM methods, Yugoslav Journal of Operations Research, 11(2): 251–265.
  • Polska, O., Kudermetov, R., Alsayaydeh, J. A. J., & Shkarupylo, V. (2021). QoS-aware Web-services ranking: normalization techniques comparative analysis for LSP method. ARPN Journal of Engineering and Applied Sciences, 16(2), 248-254.
  • Satıcı, S. (2021). Farklı Normalizasyon Tekniklerinin Çok Kriterli Karar Verme Yöntemlerine Etkisi: WASPAS Örneği. The Journal of Business, Economic and Management Research(2), 350-361.
  • Sayar, M. A., SELVİ, H. Z., & BUĞDAYCI, İ. (2019). Suruç Çadırkent Alanının Analitik Hiyerarşi Yöntemiyle Belirlenmesi. Necmettin Erbakan Üniversitesi Fen ve Mühendislik Bilimleri Dergisi, 1(1), 20-31.
  • Stewart, B. (2013). Best-practice EVA: the definitive guide to measuring and maximizing shareholder value. John Wiley & Sons.
  • Vafaei, N., Ribeiro, R. A., and Camarinha-Matos, L. M. (2015). Importance of data normalization in decision making: case study with TOPSIS method. İn ICDSST 2015 Proceedings–The 1st International Conference on Decision Support Systems Technologies, An EWG-DSS Conference (Belgrade).
  • Vafaei, N., Ribeiro, R. A., and Camarinha-Matos, L. M. (2016). Normalization techniques for multi-criteria decision making: analytical hierarchy process case study. in Doctoral Conference on Computing, Electrical and Industrial Systems (Costa de Caparica).
  • Vafaei, N., Ribeiro, R. A., and Camarinha-Matos, L. M. (2018). Selection of normalization technique for weighted average multi-criteria decision making. İn Doctoral Conference on Computing, Electrical and Industrial Systems (Costa de Caparica).
  • Vafaei, N., Ribeiro, R. A., and Camarinha-Matos, L. M. (2020). Selecting normalization techniques for the analytical hierarchy process. in Doctoral Conference on Computing, Electrical and Industrial Systems (Costa de Caparica). Vafaei, N., Ribeiro, R., & Camarinha-Matos, L. (2022). Assessing Normalization Techniques for Simple Additive Weighting Method. Procedia Computer Science, 199, 1229-1236.
  • Wang, Z., Parhi, S. S., Rangaiah, G. P., & Jana, A. K. (2020). Analysis of weighting and selection methods for pareto-optimal solutions of multiobjective optimization in chemical engineering applications. Industrial & Engineering Chemistry Research, 59(33), 14850-14867.
  • Więckowski, J., & Sałabun, W. (2020). How the Normalization of the Decision Matrix Influences the Results in the VIKOR Method? Procedia Computer Science, 176, 2222-2231.
  • Yaakob, A., Gegov, A., & Rahman, S. F. A. (2018). Fuzzy networks with rule base aggregation for selection of alternatives. Fuzzy Sets and Systems, 341, 123-144.
  • Yalcin, N., Bayrakdaroglu, A., & Kahraman, C. (2012). Application of fuzzy multi-criteria decision making methods for financial performance evaluation of Turkish manufacturing industries. Expert systems with applications, 39(1), 350-364.
  • Yang, W. C., Chon, S. H., Choe, C. M., & Yang, J. Y. (2021). Materials selection method using TOPSIS with some popular normalization methods. Engineering Research Express, 3(1), 015020.
  • Zolfani, S., Yazdani, M., Pamucar, D., & Zarate, P. (2020). A VIKOR And TOPSIS Focused Reanalysis of the MADM Methods Based on Logaiıthmic Normalization. Facta Universitatis Series: Mechanical Engineeringis, 18(3), 341-355.
There are 42 citations in total.

Details

Primary Language English
Subjects Finance
Journal Section Research Articles
Authors

Mahmut Baydaş 0000-0001-6195-667X

Tevfik Eren 0000-0002-6674-602X

Mustafa İyibildiren 0000-0002-5076-1348

Publication Date December 17, 2023
Submission Date October 30, 2023
Acceptance Date November 30, 2023
Published in Issue Year 2023 Volume: 5 Issue: Özel Sayı

Cite

APA Baydaş, M., Eren, T., & İyibildiren, M. (2023). Normalization Technique Selection for MCDM Methods: A Flexible and Conjunctural Solution that can Adapt to Changes in Financial Data Types. Necmettin Erbakan Üniversitesi Siyasal Bilgiler Fakültesi Dergisi, 5(Özel Sayı), 148-164.

Journal of Necmettin Erbakan University Faculty of Political Sciences is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License (CC BY NC).