Graf ve derin pekiştirme öğrenme tabanlı yeni bir trafik sinyalizasyon modeli

Year 2025, Volume: 40 Issue: 1, 85 - 102, 16.08.2024

Erhan Turan , Beşir Dandıl , Engin Avcı

https://doi.org/10.17341/gazimmfd.1257860

Abstract

Topolojik yapı ve kavşaktaki araçların bekleme süreleri, trafik sıkışıklığının genel nedenleri olarak gösterilir. Topolojik yapıdaki iyileştirmeler uzun ve maliyetli projeler sonucunda gerçekleşebildiğinden kavşak sinyalizasyon uygulamaları akıllı kentlerin vazgeçilmez uygulama alanı olmaktadır. Kavşak sinyalizasyon uygulamalarında kavşak bazında veya ağ genelinde, araçların birim zamanda maksimum akışını sağlamak için faz sırası ve süresi hesaplanır. Kavşak sinyalizasyon optimizasyonu birçok değişken veriden etkilenen, gerçek zamanlı bir gerçek dünya problemidir. Bu nedenle en verimli sinyalizasyon yöntemini geliştirmek halen çok sayıda çalışma yürütülmektedir. Bu çalışmada ağ genelinde kavşak noktalarındaki bekleme sürelerini azaltmak için yeşil fazın sırasını ve süresini optimize eden bir yaklaşım önerilmiştir. Bu yaklaşım, gerçek dünya haritasındaki şehir kavşakları birebir ölçeğine göre gerçek zamanlı araç verileriyle birlikte SUMO simülatörüne aktarılarak geliştirilmiştir. Graf tabanlı faz süresi ve Derin Pekiştirmeli Öğrenme (Deep Reinforcment Learning-DRL)’ ye dayalı faz sırası tahminini birleştirerek GDRL adlı yeni bir sinyalizasyon yaklaşımı önerilmiştir. Bu yaklaşımda faz sırası DRL yöntemiyle hesaplanmaktadır. Faz süresi ise Ford-Fulkerson algoritmasının maksimum akış bulma yönteminden yola çıkılarak hesaplanır.
GDRL yaklaşımı gerçek haritadaki ardışık kavşaklar üzerinde paralel çalıştırılarak ve gerçek veriler kullanılarak SUMO simülatöründe test edilmiştir. GDRL yaklaşımının, kavşaklardaki kuyruk uzunluğunu % 44 oranla azaltarak, trafik sıkışıklığının çözümünde verimli sonuçlar ürettiği gözlemlenmiştir.

Keywords

Trafik sinyalizasyon, Derin Pekiştirmeli öğrenme, SUMO simülatörü

References

• 1. Dey S., Dhal G. C., Materials progress in the control of CO and CO2 emission at ambient conditions: An overview, Mater Sci Energy Technol, 2 (3), 607–623, 2019.
• 2. Schrank D., Ttı’s 2012 Urban Mobility Report, 2012.
• 3. Schrank D., Eisele B., Lomax T., and Bak J., Urban mobility, 2015.
• 4. Inrıx, Congestion Costs Each American Nearly 100 hours, 2020.
• 5. Wei H., Zheng G., Gayah V., and Li Z., A Survey on Traffic Signal Control Methods, 1 (1), 2019.
• 6. Qadri M., Gökçe M.A., Öner E., State-of-art review of traffic signal control methods: challenges and opportunities, European Transport Research Review, 12 (1), 55, 2020.
• 7. AlZubi A.A., Alarifi A., Al-Maitah M., Alheyasat O., Multi-sensor information fusion for Internet of Things assisted automated guided vehicles in smart city, Sustain Cities Soc, 64, 2021.
• 8. Jadhao N. S., Jadhao A.S., Traffic Signal Control Using Reinforcement Learning, 2014 Fourth International Conference on Communication Systems and Network Technologies, 1130–1135, 2014.
• 9. Wei H., PressLight: Learning Max Pressure Control to Coordinate Traffic Signals in Arterial Network, KDD '19: Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, 1290-1298, July 2019.
• 10. Rasheed F., Yau K.-L. A., Noor R. Md., Wu C., Low Y.-C., Deep Reinforcement Learning for Traffic Signal Control: A Review, IEEE Access, 8, 208016–208044, 2020.
• 11. Abdullah N., Hua T. K., Using Ford-Fulkerson Algorithm and Max Flow-Min Cut Theorem to Minimize Traffic Congestion in Kota Kinabalu, Sabah. Journal of Information System and Technology Management, 2 (4), 18–34, 2017.
• 12. Turan E., Dandıl B., Avcı E., A New Graph Method Based on Deep Learning for Smart Intersections, Innovations in Smart Cities Applications, 5, 211–221, 2022.
• 13. Mercader P., Uwayid W., Haddad J.g, Max-pressure traffic controller based on travel times:An experimental analysis, Transportation Research Part C, 110, 275–290, 2019.
• 14. Vilarinho C., Tavares J.P., Rossetti R.J.F., Vilarinho C., Intelligent Traffic Lights : Green Time Period Negotiaton, Transportation Research Procedia, 22 (2016), 325–334, 2017.
• 15. H., Yuan H., Chen, Y., Yu W., Wang C., Wang J., Gao Y., Traffic Light Optimization Based on Modified Webster Function, Journal of Advanced Transportation, 2021.
• 16. Abdulhai B., Pringle R., Karakoulas G., Reinforcement Learning for True Adaptive Traffic Signal Control, Journal of Transportation Engineering, 129 (3), 278–285, 2003.
• 17. Wiering M., Multi-agent reinforcement learning for traffic light control. ICML, 1151–1158, 2000.
• 18. LA P., Bhatnagar S., Reinforcement Learning With Function Approximation for Traffic Signal Control, IEEE Transactions on Intelligent Transportation Systems, 12 (2), 412–421,2011.
• 19. Miletić M., Ivanjko E., Mandžuka S., Nečoska D. K., Combining Neural Gas and Reinforcement Learning for Adaptive Traffic Signal Control, 2021 International Symposium, 179–182, 2021.
• 20. Lawe S., Wang R., Optimization of Traffic Signals Using Deep Learning Neural Networks, Australasian Joint Conference on Artificial Intelligence, 9992, 403–415,2016.
• 21. Vidali A., Crociani L., Vizzari G., Bandini S., A deep reinforcement learning approach to asset-liability management, Proceedings-2019 Brazilian Conference on Intelligent Systems, 216–221, 2019.
• 22. Muresan M., Fu L., Pan G., Adaptive Traffic Signal Control with Deep Reinforcement Learning An Exploratory Investigation, 97th Annual Meeting of the Transportation Research Board, 2019.
• 23. Ducrocq R., Farhi N., Deep Reinforcement Q-Learning for Intelligent Traffic Signal Control with Partial Detection, 2021.
• 24. Koh S. S., Zhou B., Yang P., Yang Z., Fang H., Feng J., Reinforcement Learning for Vehicle Route Optimization in SUMO, 2018 IEEE 20th International Conference on High Performance Computing and Communications; IEEE 16th International Conference on Smart City; IEEE 4th International Conference on Data Science & Systems (HPCC/SmartCity/DSS), 1468–1473, 2018.
• 25. Wei H., Yao H., Zheng G., Li Z., IntelliLight: A reinforcement learning approach for intelligent traffic light control, KDD 2018 - Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2496–2505,2018.
• 26. Zheng G., Xiong Y., Zang X., Feng J., Wei H., Zhang H., Li Y., Xu K., Li Z, Learning Phase Competition for Traffic Signal Control. Proceedings of the 28th ACM International Conference on Information and Knowledge Management, 1963–1972, 2019.
• 27. Wei H., Xu N., Zhang H., Zheng G., Zang X., Chen C., Zhang W., Zhu Y., Xu K., Li Z, CoLight Learning network-level cooperation for traffic signal control. Proceedings of the 28th ACM International Conference on Information and Knowledge Management, 2019.
• 28. Varaiya P., The Max-Pressure Controller for Arbitrary Networks of Signalized Intersections, Advances in Dynamic Network Modeling in Complex Transportation Systems, 27-66, 2013.
• 29. Kim S., Suh W., Kim J., Traffic Simulation Software: Traffic Flow Characteristics in CORSIM, 5th International Conference on Information Science and Applications, 1–3, 2014.
• 30. Yu M., Fan W.D, Calibration of microscopic traffic simulation models using metaheuristic algorithms, Int. J. Transp. Sci. Technol., (6-1), 63–77, 2017.
• 31. Guo J., Kong Y., Li Z., Huang W., Cao J., Wei Y., A model and genetic algorithm for area-wide intersection signal optimization under user equilibrium traffic, Mathematics and Computers in Simulation, 155, 92–104, 2019.
• 32. Behrisch M., Bieker L., Erdmann J., Krajzewicz D., SUMO – Simulation of Urban Mobility, The Third International Conference on Advances in System Simulation, 55–60, 2011.
• 33. Khumara M.A.D., Fauziyyah L., Kristalina P., Estimation of Urban Traffic State Using Simulation of Urban Mobility(SUMO) to Optimize Intelligent Transport System in Smart City, 2018 Int. Electron. Symp. Eng. Technol. Appl. IES-ETA, 163–169, 2019.
• 34. Kheterpal N., Parvate K., Wu C., Kreidieh A., Vinitsky E., Bayen A. M., Flow: Deep Reinforcement Learning for Control in SUMO, SUMO 2018- Simulating Auton. Intermodal Transp. Syst., 2, 134–151, 2018.
• 35. Kučera T., Chocholáč J., Design of the City Logistics Simulation Model Using PTV VISSIM Software, Transp. Res. Procedia, 53, 258–265, 2021.
• 36. Zhang H., Feng S., Liu C., Ding Y, Zhu Y., Zhou Z., Zhang W., Yu Y., Jin H., Li Z., CityFlow: A Multi-Agent Reinforcement Learning Environment for Large Scale City Traffic Scenario, International World Wide Web Conference Committee,2019.
• 37. Bautista P.B., Aguiar L.U., Igartua M.A., How does the traffic behavior change by using SUMO traffic generation tools, Computer Communications., 181 (c), 1–13, 2022.
• 38. Krajzewicz D., Erdmann J., Behrisch M., Bieker L., Recent Development and Applications of SUMO-Simulation of Urban MObility, Int. J. Adv. Syst. Meas., (3), 128–138, 2012.
• 39. Riedel T., Brunner U., Traffic Control Using Graph Theory, IFAC Proc., 26 (2), 131–134, 1993.
• 40. Ngaosai A., Chawachat J., Traffic Signal Management using Maximum Flow Approach for Consecutive Intersections, 2018 15th Int. Conf. Electr. Eng. Comput. Telecommun. Inf. Technol., 457–460, 2018.
• 41. Zhao W., Ye Y., Ding J., Wang T., Wei T., Chen M., IPDALight: Intensity- and phase duration-aware traffic signal control based on Reinforcement Learning, Journal of Systems Architecture, 123, 2021.
• 42. Li Z., Xu C., Zhang G., A Deep Reinforcement Learning Approach for Traffic Signal Control Optimization, 2021.
• 43. Wang Y., Xu T., Niu X., Tan C., Chen E., Xiong H., STMARL: A Spatio-Temporal Multi-Agent Reinforcement Learning Approach for Cooperative Traffic Light Control, IEEE Transactions on Mobile Computing, 2019.
• 44. Kanis S., Samson L., Bloembergen D., Bakker T., Back to Basics: Deep Reinforcement Learning in Traffic Signal Control, 10th Intl. Workshop on Urban Computing at ACM SIGSPATIAL, 2021.
• 45. Qi R., Huang J., Li H., Tan Q., Huang L., Cui J., Random Ensemble Reinforcement Learning for Traffic Signal Control, 2022.
• 46. Wegener A., Piorkowski M., Raya M., Hellbrück H., Fischer S., Hubaux J.-P., TraCI: An Interface for Coupling Road Traffic and Network Simulators, Proceedings of the 11th Communications and Networking Simulation Symposium, 155-163, 2008.
• 47. Olaverri-Monreal C., Errea-Moreno J., Díaz-Álvarez A., Biurrun-Quel C., Serrano-Arriezu L., Kuba M., Connection of the SUMO Microscopic Traffic Simulator and the Unity 3D Game Engine to Evaluate V2X Communication-Based Systems. Sensors, 18 (12), 2018.
• 48. Putra S., The Correction Value of Passenger-Car Equivalents for Motorcycle and Its Impact to Road Performance in Developing Countries, Procedia - Social and Behavioral Sciences, 16, 400–408, 2011.
• 49. Metlek S., Kayaalp K., Detection of bee diseases with a hybrid deep learning method, Journal of the Faculty of Engineering and Architecture of Gazi University, 36 (3), 1715-1732, 2021.
• 50. Şafak E., Dogru İ., Barışçı N., Toklu S., Internet of things based mobile driver fatigue detection using deep learning, Journal of the Faculty of Engineering and Architecture of Gazi University, 37 (4), 1869-1882, 2022.