ÖN EĞİTİMLİ DİL MODELLERİYLE DUYGU ANALİZİ

Year 2023, , 46 - 53, 26.12.2023

Ömer Yiğit Yürütücü , Şeniz Demir

https://doi.org/10.47769/izufbed.1312032

Abstract

Duygu analizi, çeşitli platformlarda bir konu hakkında düşünce, duygu ya da tutumu irdelemek, analiz etmek ve yorumlamak amacıyla kullanılan yöntemlerden biridir. Farklı konulardaki metinlerin öznel içeriklerine göre sınıflandırılabildiği duygu analizinde makine öğrenmesi ve derin öğrenme modellerinden sıklıkla faydalanılmaktadır.
Bu çalışmada, önceden eğitilmiş dil modellerinden yararlanılarak Covid-19 tweet metinleri üzerinde duygu analizi yapılmıştır. Naive Bayes sınıflandırıcıya ek olarak BERT, RoBERTa ve BERTweet dil modelleri kullanılarak farklı sınıflandırıcılar eğitilmiş ve tweet veri kümesi üzerinde elde edilen sonuçlar kıyaslanmıştır. Bildiride aktarılan çalışmanın ileride bu alanda yürütülecek araştırmalara bir zemin oluşturacağı öngörülmektedir.

Keywords

Doğal Dil İşleme, Duygu Analizi, BERT, RoBERTa, BERTweet

References

• Abid, F., Alam, M. Yasir, M. & Li, C. (2019). Sentiment analysis through recurrent variants latterly on convolutional neural network of Twitter, Future Generation Computer Systems. 95, 292-308. Doi: https://doi.org/10.1016/j.future.2018.12.018
• Anttiroiko, A. V. & Savolainen, R. (2011). Towards Library 2.0: The Adoption of Web 2.0 Technologies in Public libraries. Libri,61(2), 87-99.
• Azzouza, N.; Akli-Astouati, K. & İbrahim, R. (2020). TwitterBERT: Framework for Twitter Sentiment Analysis Based on Pre-Trained Language Model Representations. F. Saeed et al. (Eds.): IRICT 2019, AISC 1073, 428–437. https://doi.org/10.1007/978-3-030-33582-3_41
• Baker, W. (2021). Using Large Pre-Trained Language Models to Track Emotions of Cancer Patients on Twitter. Computer Science and Compute Engineering Undergraduate Honors Theses Retrieved from https://scholarworks.uark.edu/csceuht/92
• Bakliwal, A., Foster, J., van der Puil, J., O'Brien, R., Tounsi, L., Hughes, M. (2013). Sentiment analysis of political tweets: Towards an accurate classifier. Association for Computational Linguistics. 49-58.
• Becker, L., Erhart, G., Skiba, D. & Matula, V. (2013). AVAYA: Sentiment Analysis on Twitter with Self-Trainingand Polarity Lexicon Expansion. Second Joint Conference on Lexical and Computational Semantics (*SEM).
• Blitzer, J., Dredze, M. & Pereira, F. (2007). Biographies, Bollywood, Boom-Boxes and Blenders. Domain Adaptation for Sentiment Classification, Proceedings of the 45th Annual Meeting of the Association for Computational Linguistics, 440-447.
• Cesconi F. (2020). Natural language processing: Explaining BERT to business people. https://hackernoon.com/natural-language-processing-explaining-bert-to-business-people-obz3uno (accessed: 18.12.2022).
• Conneau, A., Khandelwal, K., Goyal, N., Chaudhary, V., Wenzek, G., Guzm ́an, F., Grave, E., Ott, M., Luke Zettlemoyer, L., & Stoyanov, V. (2020). Unsupervised Cross-lingual Representation Learning at Scale. In Proceedings of ACL, page to appear.
• Culnan, M., McHugh, P. & Zubillaga, J. (2010). How large U.S. Companies Can Use Twitter and Other Social Media to Gain Business Value MIS, Quarterly Executive, 9 (4), 243-259.
• Çelikyilmaz A., Hakkani-Tür, D. & Feng, F. (2010). Probabilistic Model-Based Sentiment Analysis of Twitter Messages, in 2010 IEEE Workshop on Spoken Language Technology, SLT 2010 – Proceedings.
• Çelikten, A. & Bulut, H. (2021). Turkish Medical Text Classification Using BERT. 29th Signal Processing and Communications Applications conference at İstanbul. https://doi.org/10.1109/SIU53274.2021.9477847
• Devlin, J., Chang, M., Lee, K., & Toutanova, K. (2019). BERT: Pre-training of deep bidirectional transformers for language understanding. Proceedings of the 2019 conference of the north American chapter of the association for computational linguistics: human language technologies. 1. Minneapolis, Minnesota, 4171-4186. https://www.aclweb.org/anthology/N19-1423
• Ghag, K. V., & Shah, K. (2018). Conceptual sentiment analysis model. International Journal of Electrical and Computer Engineering (IJECE), 8(4), 2358-2366. https://doi.org/10.11591/ijece.v8i4.pp2358-2366
• Ghiassi, M., Skinner, J. & Zimbra, D. (2013). Twitter Brand Sentiment Analysis: A Hybrid System Using N-Gram Analysis and Dynamic Artificial Neural Network. Expert Systems with Applications, 40(16), 6266-6282. https://doi.org/10.1016/j.eswa.2013.05.057
• Goodfellow, I., Bengio, Y., Courville, A.; & Bengio, Y. (2016). Deep learning, 1. MIT press Cambridge.
• He, W., Zha, S. & Li, L. (2013). Social media competitive analysis and text mining: A case study in the pizza industry. International Journal of Information Management, 33(3), 464-472.
• Horev, R. (2018). BERT Explained: State of the art language model for NLP. Retrieved from https://towardsdatascience.com/bert-explained-state-of-the-art-language-model-for-nlp-f8b21a9b6270 (accessed: 18.12.2022).
• Hu, M. & Liu, B. (2004). Mining and Summarizing Customer Reviews. Proceedings of the Tenth. ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 168-177.
• Jagtap, V. S. & Pawar, K. (2013). Sentence-Level Analysis of Sentiment Classification. National Confrence on Emerging Trends in Engineering, Technology & Architecture.
• Kapucugil, A. & Özdağoğlu, G. (2015). Text mining as a supporting process for VoC clarification. Alphanumeric Journal, 3(1), 25-40.
• Kietzmann, J.H., Hermkens, K., I.P. & McCarthy, B.S. (2011). Silvestre Social Media? Get Serious! Understanding The Functional Building Blocks of Social Media. Business Horizons, 54 (3), pp. 241-251.
• Krizhevsky, A., Sutskever, I. & Hinton, G. E. (2012). ImageNet Classication with Deep Convolutional Neural Networks. NIPS Advances in Neural Information Processing Systems Conference. 1-9.
• Liu, Y.; Ott, M.; Goyal, N. Du, J.; Joshi, M.; Chen, D.; Levy, O.; Lewis, M.; Zettlemoyer, L., & Stoyanov, V. (2019). RoBERTa: ARobustly optimized BERT pretraining approach. Computer Science Computation and Language, arXiv:1907.11692
• Masarifoğlu, M., Tigrak, U., Hakyemez, S.; Gul, G.; Bozan, E.; Buyuklu, A. H. & Özgür, A. (2021). Sentiment Analysis of Customer Comments in Banking using BERT-based Approaches. Signal Processing and Communication Applications Conference (SIU). https://doi.org/10.1109/SIU53274.2021.9477890
• Mashalkar, A. (2020). Sentiment Analysis using Logistic Regression and Naive Bayes. https://towardsdatascience.com/sentiment-analysis-using-logistic-regression-and-naive-bayes-16b806eb4c4b (accessed: 07.02.2022).
• Mayfield, A. (2008). What is Social Media? icrossing.co.uk/ebooks. http://www.icrossing.com/uk/sites/default/files_uk/insight_pdf_files/What%20is%20Social%20Media_iCrossing_ebook.pdf
• Mundalik, A. (2018). Aspect Based Sentiment Analysis Using Data Mining Techniques Within Irish Airline Industry MSc Research Project Data Analytics.
• Nguyen, D. Q.; Vu, T., & Nguyen, A. (2020). BERTweet: A pre-trained language model for English Tweets. Computer Science. https://doi.org/10.18653/v1/2020.emnlp-demos.2
• Pak, A. & Paroubek, P. (2010). Twitter as a Corpus for Sentiment Analysis and Opinion Mining. Proceedings of the 7th International Conference on Language Resources and Evaluation, 320-1326.
• Pang, B., Lee, L. & Vaithyanathan S. (2002). Thumbs up? Sentiment Classification Using Machine Learning Techniques Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 79-86.
• Patil, P. P., Phansalkar, S. & Kryssanov, V. V. (2018). Topic Modelling for Aspect-Level Sentiment Analysis. Proceedings of the 2nd International Conference on Data Engineering and Communication Technology, 221-229.
• Pota, M., Ventura, M., Catelli, R., & Esposito, E. (2021). An effective BERT-based pipeline for twitter sentiment analysis: A case study in Italian. Sensor (Basil), 21(1), 133. https://doi.org/10.3390/s21010133
• Sel, İ. & Hanbay, D. (2021). Gender Identification from Turkish Tweets Using Pre-Trained Language Models. Fırat Üniversitesi Müh. Bil. Dergisi, 33(2), 675-684. https://doi.org/10.35234/fumbd.929133
• Sharma, R.; Nigam, S. & Jain, R. (2014). Opinion Mining of Movie Reviews a Document Level. International Journal on Information Theory (IJIT), 3(3), 13-21. Doi: http://dx.doi.org/10.5121/ijit.2014.3302
• Sinderen, M.V. & Almeida, J.P.A. (2011). Empowering Enterprises Through Next-Generation Enterprise Computing. Enterprise Information Systems, 5 (1), 1-8. https://doi.org/10.1080/17517575.2010.528802
• Singh, A. (2021). Evolving with BERT: Introduction to RoBERTa. https://medium.com/analytics-vidhya/evolving-with-bert-introduction-to-roberta-5174ec0e7c82 (date of access: 07.01.2022).
• Uçar, T. (2020). BERT modeli ile Türkçe metinlerde sınıflandırma yapmak. https://medium.com/@toprakucar/bert-modeli-ile-t%C3%BCrk%C3%A7e-metinlerde-s%C4%B1n%C4%B1fland%C4%B1rma-yapmak-260f15a65611 (accessed: 24.11.2021).
• Upadhyay, N. & Singh, A. (2016), Sentiment Analysis on Twitter by using Machine Learning Technique. International Journal for Research in Applied Science & Engineering Technolog (IJRASET), 4(5), 488-494.
• Weber, L. (2009). Marketing to The Social Web: How Digital Customer Communities Build Your Business. (2nd ed.), Wiley, Hoboken, NJ.
• Yıldırım, O. (2020). Internet and Social Media Use in the Period of Social Isolation and Voluntary Quarantine which comes to the Agenda Due to The New Coronavirus Outbreak. İletişim Kuram ve Araştırma Dergisi, 52. https://doi.org/10.47998/ikad.788255
• Yılmaz, M. C., & Orman, Z. (2021). Sentiment analysis from twitter data during the Covid-19 pandemic era with LSTM deep learning approach. ACTA INFOLOGICA, 5(2), 359-372. http://dx.doi.org/10.26650/acin.947747
• Zhang, W., Skiena, S. (2010). Trading Strategies to Exploit Blog and News Sentiment, In: Proceedings of the Fourth International AAAI Conference on Weblogs and Social Media.