Model Based Demand Order Estimation by Using Optimal Architecture Artificial Neural Network with Metaheuristic Optimizations

Year 2022, Volume: 12 Issue: 3, 1277 - 1291, 01.09.2022

Özlem İmik Şimşek , Barış Baykant Alagöz

https://doi.org/10.21597/jist.1099154

Abstract

With the increase of e-commerce volumes in recent years, it is useful to estimate daily demand order numbers in order to improve the demand forecasts, production-distribution planning and sales services. In this manner, data-driven modeling and machine learning tools have been preferred to enhance demand order predictions, timely delivery, incomes and customer satisfaction in electronic trading because real-time data collection is possible in e-commerce platforms. Artificial Neural Networks (ANNs) are widely used for data-driven modeling and prediction problems. Since affecting the approximation performance of neural network function, the modeling performance of ANNs strongly depends on the architecture of neural networks, and architectural optimization of ANN models has become a main topic in the neuroevolution field. This study presents an architecture optimization method that implements Particle Swarm Optimization (PSO) and Differential Evolution (DE) algorithms to optimize ANN model architecture for the estimation of total demand order numbers from the sparse demand order data. In this approach, PSO and DE algorithm only optimizes neural model architecture according to an effective network search policy and the training of ANN models is carried out by using backpropagation algorithm. This neural architecture model optimization approach considers generalization of data, reducing neuron and training epoch numbers and it can yield an optimal architecture data-driven neural model for estimation of the daily total orders. In the experimental study, optimal architecture ANN models are obtained according to the daily order forecasting dataset.

Keywords

Neural network model, hyper-parameter optimization, orders demand forecasting, particle swarm optimization, differential evolution

References

• Akay, B, Karaboga, D, Akay, R 2021. A comprehensive survey on optimizing deep learning models by metaheuristics. Artificial Intelligence Review 5: 1–66.
• Aminian J, Shahhosseini S, 2008. Evaluation of ANN modeling for prediction of crude oil fouling behavior. Applied Thermal Engineering 28(7): 668–674.
• Anochi JA, Velho HFC, Furtado HCM, 2014. Self-configuring Two Types of Neural Networks by MPCA. Conference: 2nd International Symposium on Uncertainty Quantification and Stochastic ModelingAt: Rouen - France 5(2). Doi: https://doi.org/10.17265/2159-5275/2015.02.008.
• Anochi J, Sambatti S, Luz E, Velho HC, 2016. New learning strategy for supervised neural network: MPCA meta-heuristic approach. 1st BRICS Countries & 11th CBIC Brazilian Congress on Computational Intelligence. Location: Recife, Brasil. Porto de Galinhas Beach, pp:1–6.
• Badr EM, Salam MA, Ahmed H, 2019. Optimizing Support Vector Machine using Gray Wolf Optimizer Algorithm for Breast Cancer Detection.
• Birs I, Folea S, Prodan O, Dulf E, Muresan C, 2020. An experimental tuning approach of fractional order controllers in the frequency domain. Applied Sciences (Switzerland) 10(7). Doi: https://doi.org/10.3390/APP10072379.
• Carvalho AR, Ramos FM, Chaves AA, 2011. Metaheuristics for the feedforward artificial neural network (ANN) architecture optimization problem. Neural Computing and Applications 20(8):1273–1284. Doi: https://doi.org/10.1007/s00521-010-0504-3.
• Carvalho M, Ludermir TB, 2008. Particle Swarm Optimization of Neural Network Architectures and Weights, 336–339. Doi: https://doi.org/10.1109/HIS.2007.45
• Caserta M, Voß S, 2009. Metaheuristics: Intelligent Problem Solving. Doi: https://doi.org/10.1007/978-1-4419-1306-7_1.
• Cho J, Lee K, Shin E, Choy G, Do S, 2015. How much data is needed to train a medical image deep learning system to achieve necessary high accuracy?. arXiv preprint arXiv:1511.06348.
• Clerc M, Kennedy J, 2002. The particle swarm-explosion, stability, and convergence in a multidimensional complex space. IEEE Transactions on Evolutionary Computation 6(1): 58–73. Doi: https://doi.org/10.1109/4235.985692.
• Çevik K, Koçer E, 2014. Parçacık Sürü Optimizasyonu ile Yapay Sinir Ağları Eğitimine Dayalı Bir Esnek Hesaplama Uygulaması. Süleyman Demirel Üniversitesi Fen Bilimleri Enstitüsü Dergisi 17(2): 39–45. https://dergipark.org.tr/tr/pub/sdufenbed/issue/20801/222008.
• Del Valle Y, Venayagamoorthy GK, Mohagheghi S, Hernandez JC, Harley RG, 2008. Particle swarm optimization: Basic concepts, variants and applications in power systems. IEEE Transactions on Evolutionary Computation 12(2): 171–195. Doi: https://doi.org/10.1109/TEVC.2007.896686.
• Dhaenens C, Jourdan L, 2022. Metaheuristics for data mining: survey and opportunities for big data. Annals of Operations Research 2022: 1–24. Doi: https://doi.org/10.1007/S10479-021-04496-0.
• Eberhart R, Kennedy J, 1995. A new optimizer using particle swarm theory. Undefined 39–43. Doi: https://doi.org/10.1109/MHS.1995.494215.
• Egmont-Petersen M, De Ridder D, Handels H, 2002. Image processing with neural networks- A review. In Pattern Recognition (Vol. 35, Issue 10). Doi: https://doi.org/10.1016/S0031-3203(01)00178-9.
• Ettaouil M, Ghanou Y, 2009. Neural architectures optimization and Genetic algorithms. Wseas Transactions On Computer 8(3): 526–537.
• Ferreira RP, Martiniano A, Ferreira A, Ferreira A, Sassi RJ, 2016. Study on Daily Demand Forecasting Orders using Artificial Neural Network. IEEE Latin America Transactions 14(3): 1519–1525. Doi: https://doi.org/10.1109/TLA.2016.7459644.
• Floreano D, Dürr P, Mattiussi C, 2008. Neuroevolution: from architectures to learning. Evolutionary Intelligence 2008 1(1): 47–62. Doi: https://doi.org/10.1007/S12065-007-0002-4.
• Hasanien HM, 2011. FPGA implementation of adaptive ANN controller for speed regulation of permanent magnet stepper motor drives. Energy Conversion and Management 52(2): 1252–1257. Doi: https://doi.org/10.1016/j.enconman.2010.09.021.
• Ilonen J, Kamarainen JK, Lampinen J, 2003. Differential evolution training algorithm for feed-forward neural networks. Neural Processing Letters 17(1): 93–105. Doi: https://doi.org/10.1023/A:1022995128597.
• Imik O, Alagoz BB, 2017. Discretization of fractional order transfer functions by weighted multi-objective particle swarm optimization method. IDAP 2017 - International Artificial Intelligence and Data Processing Symposium. Doi: https://doi.org/10.1109/IDAP.2017.8090162.
• Imik SO, Alagoz BB, 2021. Daily Forecasting of Demand Orders with Optimal Architecture Artificial Neural Network Learning Models. 2021 International Conference on Information Technology, ICIT 2021 - Proceedings, 355–360. Doi: https://doi.org/10.1109/ICIT52682.2021.9491784.
• İmik ŞÖ, 2018. Parcacık Sürüsü Optimizasyon Yöntemi ile Kesir Dereceli Filtre Gerçekleşmesi ,İnönü Üniversitesi, Master Thesis (Printed). http://abakus.inonu.edu.tr/xmlui/handle/11616/14805.
• Jeppesen JH, Jacobsen RH, Inceoglu F, Toftegaard TS, 2019. A cloud detection algorithm for satellite imagery based on deep learning. Remote sensing of environment 229:247-259.
• Kapanova KG, Dimov I, Sellier JM, 2018. A genetic approach to automatic neural network architecture optimization. Neural Computing and Applications 29(5): 1481–1492. Doi: https://doi.org/10.1007/S00521-016-2510-6.
• Kaya Y, Hong S, Dumitras T, 2019. Shallow-deep networks: Understanding and mitigating network overthinking. In International conference on machine learning, PMLR, pp: 3301-3310.
• Krogh A, 2008. What are artificial neural networks? Nature Biotechnology 2008 26(2): 195–197. Doi: https://doi.org/10.1038/nbt1386.
• Landa Becerra R, Coello CAC, 2006. Cultured differential evolution for constrained optimization. Computer Methods in Applied Mechanics and Engineering 195(33–36): 4303–4322. Doi: https://doi.org/10.1016/J.CMA.2005.09.006.
• Liu X, Wang GG, Wang L, 2021. LSFQPSO: quantum particle swarm optimization with optimal guided Lévy flight and straight flight for solving optimization problems. Engineering with Computers. Doi: https://doi.org/10.1007/S00366-021-01497-2.
• McCulloch WS, Pitts W, 2008. A logical calculus of the ideas immanent in nervous activity 5: 115–133.
• Nakisa B, Rastgoo MN, Rakotonirainy A, Maire F, Chandran V, 2018. Long short term memory hyperparameter optimization for a neural network based emotion recognition framework. IEEE Access 6:49325–49338.
• Oh SK, Kim WD, Pedrycz W, Joo SC, 2012. Design of K-means clustering-based polynomial radial basis function neural networks (pRBF NNs) realized with the aid of particle swarm optimization and differential evolution. Neurocomputing 78(1): 121–132. Doi: https://doi.org/10.1016/J.NEUCOM.2011.06.031.
• Pacal I, Karaboga D, Basturk A, Akay B, Nalbantoglu U, 2020. A comprehensive review of deep learning in colon cancer. Computers in Biology and Medicine 126:104003.
• Pacal I, Karaboga D, 2021. A robust real-time deep learning based automatic polyp detection system. Computers in Biology and Medicine 134: 104519.
• Pacal I, Karaman A, Karaboga D, Akay B, Basturk A, Nalbantoglu U, Coskun S, 2022. An efficient real-time colonic polyp detection with YOLO algorithms trained by using negative samples and large datasets. Computers in biology and medicine 141: 105031.
• Poli R, Kennedy J, Blackwell T, 2007. Particle swarm optimization. Swarm Intell 1(November): 33–57. Doi: https://doi.org/10.1007/s11721-007-0002-0.
• Qin AK, Huang VL, Suganthan PN, 2009. Differential Evolution Algorithm With Strategy Adaptation for Global Numerical Optimization. IEEE Transactions on Evolutionary Computation 13:398–417. https://doi.org/10.1109/TEVC.2008.927706.
• Shi Y, Eberhart R, 1998. Modified particle swarm optimizer. Proceedings of the IEEE Conference on Evolutionary Computation, ICEC, 69–73. Doi: https://doi.org/10.1109/icec.1998.699146
• Slowik A, Bialko M, 2008. Training of artificial neural networks using differential evolution algorithm. 2008 Conference on Human System Interaction, HSI 2008, 60–65. Doi: https://doi.org/10.1109/HSI.2008.4581409.
• Storn R, Price K, 1997. Differential Evolution – A Simple and Efficient Heuristic for Global Optimization over Continuous Spaces. Journal of Global Optimization 11:341:284–287.
• UCI, 2007. Daily Demand Forecasting Orders Data Set. https://archive.ics.uci.edu/ml/datasets/Daily+Demand+Forecasting+Orders
• Vijaya G, Kumar V, Verma HK, 1998. ANN-based QRS-complex analysis of ECG. Journal of Medical Engineering and Technology 22(4). Doi: https://doi.org/10.3109/03091909809032534.
• Wang D, Tan D, Liu L, 2018. Particle swarm optimization algorithm: an overview. Soft Computing 22(2): 387–408. Doi: https://doi.org/10.1007/S00500-016-2474-6.
• Widrow B, Lehr MA, 1990. 30 Years of Adaptive Neural Networks: Perceptron, Madaline, and Backpropagation. Proceedings of the IEEE 78(9): 1415–1442. Doi: https://doi.org/10.1109/5.58323.
• Yu J, Wang S, Xi L, 2008. Evolving artificial neural networks using an improved PSO and DPSO. Neurocomputing 71(4–6): 1054–1060. Doi: https://doi.org/10.1016/J.NEUCOM.2007.10.013.
• Zhao W, Chen F, Huang H, Li D, Cheng W, 2021. A new steel defect detection algorithm based on deep learning. Computational Intelligence and Neuroscience 2021: 5592878.