Review of Studies Using Artificial Intelligence in Calculating Consumer Price Index (CPI)

Year 2022, Volume: 14 Issue: 1, 299 - 315, 31.01.2022

Abdulcebar On , Necaattin Barışçı

https://doi.org/10.29137/umagd.1022087

Abstract

The Consumer Price Index (CPI) is a method that measures the change in the prices of goods and services for household consumption over time. In this study, many independent studies on CPI in the field of artificial intelligence were investigated. As a result of the literature review, it was learned that the use of artificial intelligence in the calculation of CPI is in two processes: "collection and processing of data" and "classification of products". During the data collection and processing process, alternative method studies were carried out by the official statistics offices to the data collection process by means of questionnaires. For this purpose, product information and prices on online shopping sites have been compiled using methods such as web crawling technique. In the process of classification of products, various machine learning methods were used to match the product information obtained from companies in digital environment with the classification methods used within the scope of CPI. For this, natural language processing methods have been applied to digitize text bases data. As a result, with this study, studies using artificial intelligence within the scope of CPI were gathered under a single roof and a prediction was created for new alternative calculation methods.

Keywords

web crawling, product matching, artificial intelligence, machine learning, natural language processing, cpi, information extraction

Tüketici Fiyat Endeksi (TÜFE) Hesaplamasında Yapay Zeka Kullanan Çalışmalarının İncelenmesi

Abstract

Tüketici Fiyat Endeksi (TÜFE), hanehalklarının tüketimine yönelik mal ve hizmet fiyatlarının zaman içindeki değişimini ölçen bir yöntemdir. Bu çalışmada yapay zeka alanında TÜFE ile ilgili yapılan birbirinden bağımsız birçok çalışma araştırılmıştır. Yapılan literatür taraması neticesinde TÜFE hesaplanmasındaki yapay zeka kullanımı “verilerin toplanma ve işleme” ile “ürünlerin sınıflandırılması” olmak üzere iki süreçte olduğu bilgisine ulaşılmıştır. Verilerin toplanma ve işleme sürecinde resmi istatistik ofisleri tarafından anket aracılığıyla veri toplama işlemine alternatif yöntem çalışmaları gerçekleştirilmiştir. Bunun için çevrimiçi alışveriş sitelerindeki ürün bilgileri ve fiyatları web tarama tekniği gibi yöntemler kullanılarak derlenmiştir. Ürünlerin sınıflandırılması sürecinde ise çeşitli yöntemlerle firmalardan dijital ortamda alınan ürün bilgilerinin TÜFE kapsamındaki kullanılan sınıflandırma yöntemleri ile eşleşmesi için çeşitli makine öğrenme yöntemleri kullanılmıştır. Bunun için metin tabanları verileri dijitalleştirmek için doğal dil işleme yöntemleri uygulanmıştır. Sonuç olarak bu çalışma ile TÜFE kapsamında yapay zeka kullanan çalışmalar tek bir çatı altında toplanarak yeni alternatif hesaplama yöntemleri için öngörü oluşturulmuştur.

Keywords

web tarama, ürün eşleştirme, yapay zeka, makine öğrenme, doğal dil işleme, tüfe, bilgi çıkarımı

References

• Fiyat Endekleri ve Enflasyon, Sorularla Resmi İstatistik Dizisi-3, TÜİK, 2008
• United Nations. (2018). Classification of Individual Consumption According to Purpose (COICOP)
• https://data.tuik.gov.tr/Bulten/Index?p=Tuketici-Fiyat-Endeksi-Ocak-2019-30849, “Tüketici Fiyat Endeksi, Ocak 2019”, (2019)
• Polzonetti, A., Re, B., & Vaccari, C. Big Data in Official Statistics 2013-2014. Available on: http://www. academia. edu/7571682/PhD_Thesis_on_Big_Data_in_Offic ial_Statistics_, 26-43.
• Olston, C., & Najork, M. (2010). Web crawling. Now Publishers Inc.
• Cavallo, A. (2013). Online and official price indexes: Measuring Argentina's inflation. Journal of Monetary Economics, 60(2), 152-165.
• Breton, R., Flower, T., Mayhew, M., Metcalfe, E., Milliken, N., Payne, C., ... & Woods, A. (2016). Research indices using web scraped data: May 2016 update. Newport: Office for National Statistics. Available from www. ons. gov. uk/file.
• Chuanyang, F., & Hao, J. L. W. (2016). Experinces with the Use of Online Prices in Consumer Price Index.
• Cavallo, A., & Rigobon, R. (2016). The Billion Prices Project. Journal of Economic Perspectives. Forthcoming.
• Hang, C., Yi, S., Xin, Y., & Benfu, L. (2013, July). A study on construction of inflation index based on web search data. In 2013 International Conference on e-Business (ICE-B) (pp. 1-8). IEEE. Yuan, H., Zhang, D., Xu, W., Wang, M., & Dong, W. (2013, November). Forecasting the CPI Using a Hybrid Sarima and Neural Network Model with Web News Articles. In 2013 Sixth International Conference on Business Intelligence and Financial Engineering (pp. 84-88). IEEE.
• Griffioen, R., de Haan, J., & Willenborg, L. (2014, May). Collecting clothing data from the Internet. In Proceedings of Meeting of the Group of Experts on Consumer Price Indexes, May (Vol. 2628).
• Dubey, S., & Gennari, P. (2014). Now-casting food consumer price indexes with big data: Public-private complementarities. FAO Working Paper.
• Manik, D. P. (2015, November). A strategy to create daily Consumer Price Index by using big data in Statistics Indonesia. In 2015 International Conference on Information Technology Systems and Innovation (ICITSI) (pp. 1-5). IEEE.
• Swier, N. (2015). Using Web Scraped Data to Construct Consumer Price Indices, referat wygłoszony na konferencji NTTS 2015, 10-12 marca 2015, Bruksela.
• Nygaard, R. (2015). The use of online prices in the Norwegian Consumer Price Index. Statistics Norway.
• Polidoro, F., Giannini, R., Conte, R. L., Mosca, S., & Rossetti, F. (2015). Web scraping techniques to collect data on consumer electronics and airfares for Italian HICP compilation. Statistical Journal of the IAOS, 31(2), 165-176.
• Barcaroli, G., Scannapieco, M., Scarno, M., & Summa, D. (2015). Using internet as a data source for official statistics: a comparative analysis of web scraping technologies. In Proceedings of Proceedings of the New Techniques and Technologies for Statistics Conference (NTTS).
• Barcaroli, G., Scannapieco, M., & Summa, D. (2016). On the use of internet as a data source for official statistics: a strategy for identifying enterprises on the web. Rivista italiana di economia, demografia e statistica, 70(4), 20-41. Cavallo, A., & Rigobon, R. (2016). The billion prices project: Using online prices for measurement and research. Journal of Economic Perspectives, 30(2), 151-78. Gabrielli, L., Riccardi, G., & Pappalardo, L. Using retail market Big Data to nowcast Customer Price Index.
• Auer, J., & Boettcher, I. (2016). From price collection to price data analytics. How new large data sources require price statisticians to re-think their index compilation procedures. Experiences from web-scraped and scanner data. Statistics Austria. Metcalfe, E., Flower, T., Lewis, T., Mayhew, M., & Rowland, M. (2017, May). Research indices using web scraped price data: clustering large datasets into price indices (CLIP). In th meeting of the Ottawa Group (pp. 10-12).
• Thakur, G. S. M., Bhattacharyya, R., & Mondal, S. S. (2016). Artificial neural network based model for forecasting of inflation in India. Fuzzy Information and Engineering, 8(1), 87-100.
• Griffioen, A. R., & Ten Bosch, O. (2016, May). On the use of internet data for the Dutch CPI. In Statistics Netherlands. Paper presented at the UNECE/ILO Meeting of the Group of Experts on Consumer Price Indices, Geneva, May (pp. 2-4).
• Struijs, P., Consten, A., Daas, P., Debusschere, M., Ilves, M., Nikic, B., ... & Swier, N. (2017, May). The ESSnet Big Data: Experimental Results. In STATISTICS AND DATA SCIENCE: NEW CHALLENGES, NEW GENERATIONS.
• Nugroho, S. M. S., Budiastuti, I. A., & Hariadi, M. (2017, August). Predicting daily consumer price index using support vector regression method based cloud computing. In 2017 International Seminar on Intelligent Technology and Its Applications (ISITIA) (pp. 313-318). IEEE.
• Powell, B. J., Nason, G., Elliott, D., Mayhew, M., Davies, J., & Winton, J. (2018). Tracking and modelling prices using web-scraped price microdata: towards automated daily consumer price index forecasting. Journal of the Royal Statistical Society: Series A (Statistics in Society), 737-756.
• Sutiawan, N. S., & Nugraha, I. B. (2017, October). Online price prediction system of consumption commodities. In 2017 International Conference on Information Technology Systems and Innovation (ICITSI) (pp. 145-150). IEEE. Aparicio, D., & Bertolotto, M. I. (2020). Forecasting inflation with online prices. International Journal of Forecasting, 36(2), 232-247.
• Huang, N., Wimalaratne, W., & Pollard, B. (2017). The Effects of the Frequency and Implementation Lag of Basket Updates on the Canadian CPI. Journal of Official Statistics, 33(4), 979-1004.
• Bentley, A., & Krsinich, F. (2017). Towards a big data CPI for New Zealand. In 15th Meeting of the Ottawa Group on Price Indices, Eltville am Rhein.
• Cavallo, A. (2017). Are online and offline prices similar? Evidence from large multi-channel retailers. American Economic Review, 107(1), 283-303.
• Hull, I., Löf, M., Tibblin, M., & Riksbank, S. (2017). Price information collected online and short-term inflation forecasts. In Proc. IFC-Bank Indonesia Satellite Seminar on" Big Data" at the ISI Regional Statistics Conference.
• Zhou, Y. (2017). Modelling Swedish Inflation Using Market Data.
• https://www.equifax.com/business/analytic-dataset/, “Analytic Dataset”, (2018)
• Whitaker, S. D. (2018). Big data versus a survey. The Quarterly Review of Economics and Finance, 67, 285-296.
• Harchaoui, T. M., & Janssen, R. V. (2018). How can big data enhance the timeliness of official statistics?: The case of the US consumer price index. International Journal of Forecasting, 34(2), 225-234.
• Cavallo, A. (2018). Scraped data and sticky prices. Review of Economics and Statistics, 100(1), 105-119.
• Abe, N., & Shinozaki, K. (2018). Compilation of Experimental Price Indices Using Big Data and Machine Learning: A Comparative Analysis and Validity Verification of Quality Adjustments (No. 18-E-13). Bank of Japan.
• Cavallo, A., Diewert, W. E., Feenstra, R. C., Inklaar, R., & Timmer, M. P. (2018, May). Using online prices for measuring real consumption across countries. In AEA Papers and Proceedings (Vol. 108, pp. 483-87).
• ten Bosch, O., Windmeijer, D., van Delden, A., & van den Heuvel, G. (2018, October). Web scraping meets survey design: combining forces. In Big Data Meets Survey Science Conference, Barcelona, Spain.
• Ghani, R., Probst, K., Liu, Y., Krema, M., & Fano, A. (2006). Text mining for product attribute extraction. ACM SIGKDD Explorations Newsletter, 8(1), 41-48.
• Kannan, A., Givoni, I. E., Agrawal, R., & Fuxman, A. (2011, August). Matching unstructured product offers to structured product specifications. In Proceedings of the 17th ACM SIGKDD international conference on Knowledge discovery and data mining (pp. 404-412).
• Kannan, A., Talukdar, P. P., Rasiwasia, N., & Ke, Q. (2011, December). Improving product classification using images. In 2011 IEEE 11th International Conference on Data Mining (pp. 310-319). IEEE.
• Köpcke, H., Thor, A., Thomas, S., & Rahm, E. (2012, March). Tailoring entity resolution for matching product offers. In Proceedings of the 15th International Conference on Extending Database Technology (pp. 545-550).
• Zahavy, T., Magnani, A., Krishnan, A., & Mannor, S. (2016). Is a picture worth a thousand words? a deep multi-modal fusion architecture for product classification in e-commerce. arXiv preprint arXiv:1611.09534.
• Cevahir, A., & Murakami, K. (2016, December). Large-scale Multi-class and Hierarchical Product Categorization for an E-commerce Giant. In Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers (pp. 525-535).
• Ristoski, P., Petrovski, P., Mika, P., & Paulheim, H. (2018). A machine learning approach for product matching and categorization. Semantic web, 9(5), 707-728.
• Xia, Y., Levine, A., Das, P., Di Fabbrizio, G., Shinzato, K., & Datta, A. (2017, April). Large-scale categorization of japanese product titles using neural attention models. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 2, Short Papers (pp. 663-668).
• Loon, K.V, & Roels D., (2018). Le webscraping, la collecte et le traitement de données en ligne pour l'indice des prix à la consommation. Shah, K., Kopru, S., & Ruvini, J. D. (2018, June). Neural network based extreme classification and similarity models for product matching. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 3 (Industry Papers) (pp. 8-15).
• Bonnett, C. (2016). Classifying e-commerce products based on images and text. Adventures in Machine Learning.
• More, A. (2017). Product matching in ecommerce using deep learning.