Classification of Customer Complaints Using BERTopic Topic Modelling Technique

Year 2022, , 66 - 79, 31.12.2022

Kutan Koruyan

https://doi.org/10.47899/ijss.1167719

Cited By: 3

Abstract

The analysis of customer complaints enables companies to amend mistakes, protect brand value and attract new customers. Utilizing machine learning techniques for data classification and prediction provide decision makers with time and cost benefits, particularly with increased complaint data size. Therefore, this study employed BERTopic topic modelling technique, a contemporary approach, to examine customer complaint distribution with respect to distinct topics, by product and in general. In the study, the complaints submitted to a consumer electronics retailer in 2020 were adopted and classified. The monthly variation of the complaints was also investigated with dynamic topic modelling. The results showed that the complaints concentrated more heavily on shipping, television, mobile phone, laptop, earphones, tablets, store clerks and order cancellation topics.

Keywords

Customer Complaints, Topic Modelling, BERTopic, Dynamic Topic Modelling

BERTopic Konu Modelleme Tekniği Kullanılarak Müşteri Şikayetlerinin Sınıflandırılması

Abstract

Müşteri şikâyetlerinin analizi işletmeler açısından geçmişte yaptıkları hataları düzeltme, marka değerini koruma ve yeni müşteriler edinmeleri açısından önemli bir kavramdır. Özellikle şikâyet verisinin büyüklüğü arttıkça verinin sınıflandırılması ve tahminlenmesi için makine öğrenmesi tekniklerinden yararlanmak zaman ve maliyet açısından karar vericilere avantaj sağlamaktadır. Bu yüzden çalışmada, müşteri şikayetlerinin ürün bazında ve genel anlamda hangi farklı konularda dağılım gösterdiğinin bulunması amacıyla güncel bir yaklaşım olan BERTopic konu modelleme tekniğinden yararlanılmıştır. Buna yönelik olarak da veri seti olarak 2020 yılına ait bir tüketici elektroniği perakende şirketine yapılan şikayetler kullanılmış ve sınıflandırılmıştır. Bunun yanında, şikayetlerin aylık olarak zaman içindeki değişimi de dinamik konu modelleme kullanılarak incelenmiştir. Sonuçlara göre en fazla şikâyet kargolama, televizyon, cep telefonu, dizüstü bilgisayar, kulaklık, tablet, mağaza çalışanları, sipariş iptali konularında yoğunlaşmıştır.

Keywords

Müşteri Şikayetleri, Konu Modelleme, BERTopic, Dinamik Konu Modelleme

References

• Abuzayed, A., & Al-Khalifa, H. (2021). BERT for Arabic topic modeling: An experimental study on BERTopic technique. Procedia Computer Science, 189, 191–194. https://doi.org/10.1016/j.procs.2021.05.096
• Akbıyık, A., & Arı, O. (2022). Lojistik regresyon ile faydalı müşteri yorumlarını tahminlime. Journal of Research in Business, 7, IMISC 2021 Special Issue, 15–32. https://doi.org/10.54452/jrb.1024602
• Alabay, M. N. (2012). Müşteri şikâyetleri yönetimi. Uluslararası Yönetim İktisat ve İşletme Dergisi, 8(16), 137–157.
• Alhaj, F., Al-Haj, A., Sharieh, A., & Jabri, R. (2022). Improving Arabic cognitive distortion classification in Twitter using BERTopic. International Journal of Advanced Computer Science and Applications, 13(1), 854–860. https://doi.org/10.14569/ijacsa.2022.0130199
• Altıntaş, V., Albayrak, M., & Topal, K. (2021). Kanser hastalığı ile ilgili paylaşımlar için Dirichlet ayrımı ile gizli konu modelleme. Gazi Üniversitesi Mühendislik-Mimarlık Fakültesi Dergisi, 36(4), 2183–2196 https://doi.org/10.17341/gazimmfd.734730
• Angelov, D. (2020). Top2Vec: Distributed representations of topics. arXiv. https://doi.org/10.48550/ARXIV.2008.09470
• Aşkun, O. (2015). Şikâyet iletilerinin örgütsel öğrenme üzerine etkisi. Marmara Üniversitesi İktisadi ve İdari Bilimler Dergisi, 24(1), 221–243.
• Baird, A., Xia, Y., & Cheng, Y. (2022). Consumer perceptions of telehealth for mental health or substance abuse: A Twitter-based topic modeling analysis. JAMIA Open, 5(2), 1–8. https://doi.org/10.1093/jamiaopen/ooac028
• Bal, V. (2014). Online satış girişimcilerinin karşılaştıkları müşteri şikâyetlerinin analizi. AİBÜ Sosyal Bilimler Enstitüsü Dergisi, 14(1), 59–74.
• Bayrak, A. T., Türker, B. B., Yıldız, E., & Özbek, E. E. (2021). Complaint detection and classification of customer reviews. SIU, 29th Signal Processing and Communications Applications Conference, 9-11 June 2021, İstanbul, Türkiye, pp. 1–4. https://doi.org/10.1109/siu53274.2021.9478016
• Bayram, U. (2022). Revealing the reflections of the pandemic by investigating COVID-19 related news articles using machine learning and network analysis. Bilişim Teknolojileri Dergisi, 15(2), 209–220. https://doi.org/10.17671/gazibtd.949599
• Blei, D. M., Ng, A. Y., & Jordan, M. I. (2003). Latent dirichlet allocation. Journal of Machine Learning Research, 3, 993–1022.
• Bozyiğit, F., Doğan, O., & Kılınç, D. (2022). Categorization of customer complaints in food industry using machine learning approaches. Journal of Intelligent Systems: Theory and Applications, 5(1), 85–91. https://doi.org/10.38016/jista.954098
• Campello, R. J., Moulavi, D. & Sander, J. (2013). Density-based clustering based on hierarchical density estimates. PAKDD 2013, 17th Pacific-Asia Conference, April 2013, Gold Coast, Australia, pp. 160–172.
• Chong, M., & Chen, H. (2021). Racist framing through stigmatized naming: A topical and geo‐locational analysis of #chinavirus and #chinesevirus on Twitter. 84th Annual Meeting of the Association for Information Science & Technology, 29 October – 3 November 2021, Salt Lake City, USA, pp. 70–79. https://doi.org/10.1002/pra2.437
• Çağlar Çetinkaya, N. (2020). Hizmet kalitelerine ilişkin müşteri şikayetleri: Bir içerik analizi. Selçuk 2. Uluslararası Sosyal Bilimler Kongresi, 7 Haziran 2020, Konya, pp. 39–54.
• Deerwester, S., Dumais, S. T., Furnas, G. W., Landauer, T. K., & Harshman, R. (1990). Indexing by latent semantic analysis. Journal of the American Society for Information Science, 41(6), 391–407. https://doi.org/10.1002/(sici)1097-4571(199009)41:6<391::aid-asi1>3.0.co;2-9
• Demirel, Y. (2017). Müşteri ilişkileri yönetimi: Teori, uygulama, ölçüm, 3. Baskı. Ankara: Seçkin Yayıncılık.
• Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018). BERT: Pre-training of deep bidirectional transformers for language understanding (Version 2). arXiv. https://doi.org/10.48550/ARXIV.1810.04805
• Du, T., Umar, P., Rajtmajer, S., & Squicciarini, A. (2022). The contribution of verified accounts to self-disclosure in COVID-related Twitter conversations. Sixteenth International AAAI Conference on Web and Social Media, 6-9 June 2022, Atlanta, Georgia, USA, pp. 1393-1397.
• Dwivedi, M., Shibu, T. P., & Venkatesh, U. (2007). Social software practices on the internet. International Journal of Contemporary Hospitality Management, 19(5), 415–426. https://doi.org/10.1108/09596110710757570
• Ebeling, R., Córdova Sáenz, C. A., Nobre, J., & Becker, K. (2021). The effect of political polarization on social distance stances in the Brazilian COVID-19 scenario. Journal of Information and Data Management, 12(1). https://doi.org/10.5753/jidm.2021.1889
• Egger, R., & Yu, J. (2022). A topic modeling comparison between LDA, NMF, Top2Vec, and BERTopic to demystify Twitter posts. Frontiers in Sociology, 7, 886498. https://doi.org/10.3389/fsoc.2022.886498
• Elazar, Y., Kassner, N., Ravfogel, S., Ravichander, A., Hovy, E., Schütze, H., & Goldberg, Y. (2021). Measuring and improving consistency in pretrained language models. Transactions of the Association for Computational Linguistics, 9, 1012–1031. https://doi.org/10.1162/tacl_a_00410
• Faed, A. (2010). Handling e-complaints in customer complaint management system using FMEA as a qualitative system, IMS 2010, 6th International Conference on Advanced Information Management and Service, 30 November - 02 December 2010, Seoul, Korea, pp. 205–209.
• Filieri, R., Lin, Z., Li, Y., Lu, X., & Yang, X. (2022). Customer emotions in service robot encounters: A hybrid machine-human intelligence approach. Journal of Service Research, 109467052211039. https://doi.org/10.1177/10946705221103937
• Galitsky, B. (2020). Artificial intelligence for customer relationship management, Springer International Publishing, Cham. https://doi.org/10.1007/978-3-030-52167-7
• Garding, S., & Bruns, A. (2015). Complaint management and channel choice, Springer International Publishing, Cham. https://doi.org/10.1007/978-3-319-18179-0
• Ghazzawi, A., & Alharbi, B. (2019). Analysis of customer complaints data using data mining techniques. Procedia Computer Science, 163, 62–69. https://doi.org/10.1016/j.procs.2019.12.087
• Goodman, J. (1992). Leveraging the customer database to your competitive advantage, Journal of Direct Marketing, 55, 26–27.
• Greedharry, M., Seewoogobin, V., & Gooda Sahib-Kaudeer, N. (2019). A smart mobile application for complaints in Mauritius. Advances in Intelligent Systems and Computing, Springer Singapore, Singapore. https://doi.org/10.1007/978-981-13-3338-5_32
• Grootendorst, M. (2021). BERTopic. https://maartengr.github.io/BERTopic/api/bertopic.html
• Grootendorst, M. (2022). BERTopic: Neural topic modeling with a class-based TF-IDF procedure. arXiv. https://doi.org/10.48550/ARXIV.2203.05794
• Gropp, C., Herzog, A., Safro, I., Wilson, P. W., & Apon, A. W. (2016). Scalable dynamic topic modeling with clustered latent dirichlet allocation (CLDA) (Version 3). arXiv. https://doi.org/10.48550/ARXIV.1610.07703
• Gupta, M., Singh, A., Jain, R., Saxena, A., & Ahmed, S. (2021). Multi-class railway complaints categorization using Neural Networks: RailNeural. Journal of Rail Transport Planning and Management, 20, 100265. https://doi.org/10.1016/j.jrtpm.2021.100265
• HaCohen-Kerner, Y., Dilmon, R., Hone, M., & Ben-Basan, M. A. (2019). Automatic classification of complaint letters according to service provider categories. Information Processing and Management, 56(6), 102102. https://doi.org/10.1016/j.ipm.2019.102102
• Halstead, D., & Dröge, C. (1991). Consumer attitudes toward complaining and the prediction of multiple complaint responses. Advances in Consumer Research, 18(1), 210–216.
• Hendry, D., Darari, F., Nurfadillah, R., Khanna, G., Sun, M., Condylis, P. C., & Taufik, N. (2021). Topic modeling for customer service chats. ICACSIS, International Conference on Advanced Computer Science and Information Systems, 23-25 October 2021, Depok, Indonesia, pp. 1–6. https://doi.org/10.1109/icacsis53237.2021.9631322
• Hofmann, T. (1999). Probabilistic latent semantic indexing. SIGIR ’99, 22nd annual international ACM SIGIR conference on Research and development in information retrieval, August 1999, New York, USA, pp. 50–57. https://doi.org/10.1145/312624.312649
• Homburg, C., & Fürst, A., 2007. See no evil, hear no evil, speak no evil: a study of defensive organizational behavior towards customer complaints. Journal of the Academy of Marketing Science. 35(4), 523–536. https://doi.org/10.1007/s11747-006-0009-x
• İlhan Omurca, S., Ekinci, E., Yakupoğlu, E., Arslan, E., & Çapar, B. (2021). Automatic detection of the topics in customer complaints with artificial intelligence. Balkan Journal of Electrical and Computer Engineering, 9(3), 268–277. https://doi.org/10.17694/bajece.832274
• Johnston, R. (2001). Linking complaint management to profit. International Journal of Service Industry Management, 12(1), 60-69. https://doi.org/10.1108/09564230110382772
• Karami, A., & Pendergraft, N. M. (2018). Computational analysis of insurance complaints: GEICO case study. arXiv. https://doi.org/10.48550/ARXIV.1806.09736
• Kirilenko, A. P., Stepchenkova, S. O., & Dai, X. (2021). Automated topic modeling of tourist reviews: Does the Anna Karenina principle apply?. Tourism Management, 83, 104241. https://doi.org/10.1016/j.tourman.2020.104241
• Lee, D. D., & Seung, H. S. (2000). Algorithms for non-negative matrix factorization. NIPS 2001, Advances in neural information processing systems, 3-8 December 2001, Vancouver, Canada.
• McInnes, L., Healy, J., & Melville, J. (2018). UMAP: Uniform manifold approximation and projection for dimension reduction (Version 3). arXiv. https://doi.org/10.48550/ARXIV.1802.03426
• Nikolenko, S. I., Koltcov, S., & Koltsova, O. (2017). Topic modelling for qualitative studies. Journal of Information Science, 43(1), 88–102. https://doi.org/10.1177/0165551515617393
• Oğuzlar, A. (2007). Analitik hiyerarşi süreci ile müşteri şikayetlerinin analizi. Akdeniz İİBF Dergisi, 7(14), 122–134.
• Oly Ndubisi, N., & Yin Ling, T. (2006). Complaint behaviour of Malaysian consumers. Management Research News, 29(1/2), 65–76. https://doi.org/10.1108/01409170610645457
• Özçınar, H., & Öztürk, T. (2022). Eğitim bilimleri çalışmalarında kullanılan ağ yaklaşımının kavramsal haritalanması. Pamukkale University Journal of Education. https://doi.org/10.9779/pauefd.1087757
• Reimers, N., & Gurevych, I. (2019). Sentence-BERT: Sentence embeddings using siamese BERT-Networks (Version 1). arXiv. https://doi.org/10.48550/ARXIV.1908.10084
• Richins, M. L. (1983). Negative word-of-mouth by dissatisfied consumers: a pilot study. Journal of Marketing, 47(1), 68–78. https://doi.org/10.2307/3203428
• Salmi, S., Mérelle, S., Gilissen, R., van der Mei, R., & Bhulai, S. (2022). Detecting changes in help seeker conversations on a suicide prevention helpline during the COVID− 19 pandemic: in-depth analysis using encoder representations from transformers. BMC Public Health, 22(530). https://doi.org/10.1186/s12889-022-12926-2
• Sánchez-Franco, M. J., & Rey‐Moreno, M. (2021). Do travelers’ reviews depend on the destination? An analysis in coastal and urban peer‐to‐peer lodgings. Psychology and Marketing, 39(2), 441–459. https://doi.org/10.1002/mar.21608
• Sann, R., Lai, P-C., Liaw, S-Y., & Chen, C-T. (2021). Modelling online complaining behaviour in the hospitality industry: An application of data mining algorithms. APacCHRIE 2021 Conference, 2-4 June 2021, Singapore, pp. 699–702.
• Sarı, F. Ö., Alikılıç, Ö., & Onat, F. (2013). E-Complaining: Analysis of lodging customers’ e-complaints from a Turkish internet website. ICIBET-2013, International Conference on Information, Business and Education Technology, 14-15 March 2013, Beijing, China, pp. 561–565. https://doi.org/10.2991/icibet.2013.183
• Tanrısever, C. (2018). Paket tur satın alan müşterilerin şikâyet analizi. Turizm Akademik Dergisi, 5(1), 114–123.
• Taşar, D. E., Ozan, Ş., Özdil, U., Akça, M. F., Ölmez, O., Gülüm, S., Kutal, S., & Belhan, C. (2021). Kısa konuşma cümlelerinin dönüştürücü yöntemleriyle otomatik etiketlenmesi. ASYU, 2021 Akıllı Sistemlerde Yenilikler ve Uygulamaları Konferansı, 6-8 Ekim 2021, Elazığ, Türkiye. https://doi.org/10.1109/asyu52992.2021.9598957
• Yakut Aymankuy, Ş. (2011). Yerli turistlerin internet ortamındaki şikayetlerinin satınalma kararlarına etkileri. Balıkesir Üniversitesi Sosyal Bilimler Enstitüsü Dergisi, 14(25), 218–238.
• Yang, Y., Xu, D.-L., Yang, J.-B., & Chen, Y.-W. (2018). An evidential reasoning-based decision support system for handling customer complaints in mobile telecommunications. Knowledge-Based Systems, 162, 202–210. https://doi.org/10.1016/j.knosys.2018.09.029
• Zhunis, A., Lima, G., Song, H., Han, J., & Cha, M. (2022). Emotion bubbles: Emotional composition of online discourse before and after the COVID-19 outbreak. WWW '22, ACM Web Conference, 25-29 April 2022, Virtual Event, Lyon, France, pp. 2603–2613. https://doi.org/10.1145/3485447.3512132