Adaptive Traffic Signal Control Based on Neural Network Prediction of Weighted Traffic Flow

Decree of the President of the Russian Federation on December 1, 2016 no. 642. http://kremlin.ru/acts/bank/41449. Cited September 19, 2022. D. Schrank, L. Albert, B. Eisele, and T. Lomax, 2021 Urban Mobility Report. https://trid.trb.org/view/1862637. Cited May 20, 2022. V. V. Myasnikov, A. A. Agafonov, and A. S. Yumaganov, ‘‘A deterministic predictive traffic signal control model in intelligent transportation and geoinformation systems,’’ Kom’yut. Opt. 45, 917–925 (2021). https://doi.org/10.18287/2412-6179-CO-1031 H. Wei, G. Zheng, V. Gayah, and Z. Li, ‘‘A survey on traffic signal control methods,’’ 2020. arXiv:1904.08117 [cs.LG] M. Papageorgiou, C. Diakaki, V. Dinopoulou, A. Kotsialos, and Yi. Wang, ‘‘Review of road traffic control strategies,’’ Proc. IEEE 91, 2043–2065 (2003). https://doi.org/10.1109/JPROC.2003.819610 K.-L. A. Yau, J. Qadir, H. L. Khoo, and M. H. Ling, ‘‘A survey on reinforcement learning models and algorithms for traffic signal control,’’ ACM Comput. Surv. 50, 34 (2017). https://doi.org/10.1145/3068287 M. Gregurić, M. Vujić, C. Alexopoulos, and M. Miletić, ‘‘Application of deep reinforcement learning in traffic signal control: An overview and impact of open traffic data,’’ Appl. Sci. 10, 4011 (2020). https://doi.org/10.3390/app10114011 A. Haydari and Y. Yılmaz, ‘‘Deep reinforcement learning for intelligent transportation systems: A survey,’’ IEEE Trans. Intell. Transp. Syst. 23 (1), 11–32 (2022). https://doi.org/10.1109/TITS.2020.3008612 B. Abdulhai, R. Pringle, and G. Karakoulas, ‘‘Reinforcement learning for true adaptive traffic signal control,’’ J. Transp. Eng. 129, 278–285 (2003). M. Eom and B.-In Kim, ‘‘The traffic signal control problem for intersections: A review,’’ Eur. Transp. Res. Rev. 12, 50 (2022). https://doi.org/10.1186/s12544-020-00440-8 R. M. Savithramma and R. Sumathi, ‘‘Road traffic signal control and management system: A survey,’’ in 3rd Int. Conf. on Intelligent Sustainable Systems (ICISS), Palladam, India, 2020 (IEEE, 2020), pp. 104–110. https://doi.org/10.1109/ICISS49785.2020.9315970 F. V. Webster, Traffic Signal Settings (H.M. Stationery Office, London, 1958). J. Little, M. Kelson, and N. Gartner, ‘‘MAXBAND: A program for setting signals on arteries and triangular networks,’’ Transp. Res. Record J. Transp. Res. Board 795, 40–46 (1981). M.-T. Li and A. Gan, ‘‘Signal timing optimization for oversaturated networks using TRANSYT-7F,’’ Transp. Res. Record. 1683, 118–126 (1999). S. El-Tantawy and B. Abdulhai, ‘‘An agent-based learning towards decentralized and coordinated traffic signal control,’’ in 13th Int. IEEE Conf. on Intelligent Transportation Systems, Funchal, Portugal, 2010 (IEEE, 2010), pp. 665–670. https://doi.org/10.1109/ITSC.2010.5625066 S.-B. Cools, C. Gershenson, and B. D’Hooghe, ‘‘Self-organizing traffic lights: A realistic simulation,’’ in Advances in Applied Self-Organizing Systems, Ed. by M. Prokopenko, Advanced Information and Knowledge Processing (Springer, London, 2013), pp. 45–55. https://doi.org/10.1007/978-1-4471-5113-5_3 P. Varaiya, ‘‘The max-pressure controller for arbitrary networks of signalized intersections,’’ in Advances in Dynamic Network Modeling in Complex Transportation Systems, Ed. by S. Ukkusuri and K. Ozbay, Complex Networks and Dynamic Systems, Vol. 2 (Springer, New York, 2013), pp. 27–66. https://doi.org/10.1007/978-1-4614-6243-9_2 W. Genders and S. Razavi, ‘‘An open-source framework for adaptive traffic signal control,’’ 2019. arXiv:1909.00395 [eess.SY] A. Agafonov and V. Myasnikov, ‘‘Traffic signal control: A double q-learning approach,’’ in 16th Conf. on Computer Science and Intelligence Systems (FedCSIS), Sofia, 2021 (IEEE, 2021), pp. 365–369. doi 10.15439/2021F109 H. Wei, G. Zheng, H. Yao, and Z. Li, ‘‘IntelliLight: A reinforcement learning approach for intelligent traffic light control,’’ in KDD’18: Proc. 24th ACM SIGKDD Int. Conf. on Knowledge Discovery & Data Mining, London, 2018, pp. 2496–2505. doi 10.1145/3219819.3220096 Z. Zhang, J. Yang, and H. Zha, ‘‘Integrating independent and centralized multi-agent reinforcement learning for traffic signal network optimization,’’ in AAMAS’20: Proc. 19th Int. Conf. on Autonomous Agents and MultiAgent Systems, Auckland, New Zealand, 2020 (Int. Foundation for Autonomous Agents and Multiagent Systems, Richland, S.C., 2020), pp. 2083–2085. arXiv:1909.10651 [cs.LG] G. Zheng, Yu. Xiong, X. Zang, J. Feng, H. Wei, H. Zhang, Yo. Li, K. Xu, and Zh. Li, ‘‘Learning phase competition for traffic signal control,’’ in CIKM’19: Proc. 28th ACM Int. Conf. on Information and Knowledge Management, Bejing, 2019 (Association for Computing Machinery, New York, 2019), pp. 1963–1972. https://doi.org/10.1145/3357384.3357900 H. Wei, C. Chen, G. Zheng, K. Wu, V. Gayah, K. Xu, and Zh. Li, ‘‘PressLight: Learning max pressure control to coordinate traffic signals in arterial network,’’ in KDD ’19: Proc. 25th ACM SIGKDD Int. Conf. on Knowledge Discovery and Data Mining, Anchorage, Alaska, 2019 (Association for Computing Machinery, New York, 2019), pp. 1290–1298. doi 10.1145/3292500.3330949 Y. K. Chin, L. K. Lee, N. Bolong, S. S. Yang, and K. T. K. Teo, ‘‘Exploring Q-learning optimization in traffic signal timing plan management,’’ in Proc. 3rd Int. Conf. on Computational Intelligence, Communication Systems and Networks, Bali, Indonesia, 2011 (IEEE, 2011), pp. 269–274. https://doi.org/10.1109/CICSyN.2011.64 J. Gu, Y. Fang, Zh. Sheng, and P. Wen, ‘‘Double deep Q-network with a dual-agent for traffic signal control,’’ Appl. Sci. 10, 1622 (2020). https://doi.org/10.3390/app10051622 J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov, ‘‘Proximal policy optimization algorithms,’’ 2017. arXiv:1707.06347 [cs] J. Ault, J. P. Hanna, and G. Sharon, ‘‘Learning an interpretable traffic signal control policy,’’ in AAAMAS ’20: Proc. 19th Int. Conf. on Autonomous Agents and MultiAgent Systems, Auckland, New Zealand, 2020 (Int. Foundation for Autonomous Agents and Multiagent Systems, Richland, S.C., 2020), pp. 88–96. arXiv:1912.11023 [cs.LG] Y. Li, J. He, and Ya. Gao, ‘‘Intelligent traffic signal control with deep reinforcement learning at single intersection,’’ in ICCAI 2021: 7th Int. Conf. on Computing and Artificial Intelligence, Tianjin, China, 2021 (Association for Computing Machinery, New York, 2021), pp. 399–406. https://doi.org/10.1145/3467707.3467767 M. Aslani, M. S. Mesgari, and M. Wiering, ‘‘Adaptive traffic signal control with actor-critic methods in a real-world traffic network with different traffic disruption events,’’ Transp. Res. Part C: Emerging Technol. 85, 732–752 (2017). https://doi.org/10.1016/j.trc.2017.09.020 Sh. Yang, Bo Yang, Zh. Kang, and L. Deng, ‘‘IHG-MA: Inductive heterogeneous graph multi-agent reinforcement learning for multi-intersection traffic signal control,’’ Neural Networks 139, 265–277 (2021). https://doi.org/10.1016/j.neunet.2021.03.015 Y. Wu, E. Mansimov, S. Liao, R. Grosse, and J. Ba, ‘‘Scalable trust-region method for deep reinforcement learning using Kronecker-factored approximation,’’ in NIPS’17: Proc. 31st Int. Conf. on Neural Information Processing Systems, Ed. by U. von Luxburg, I. Guyon, S. Bengio, H. Wallach, and R. Fergus (Curran Associates, Red Hook, N.Y., 2017), pp. 5285–5294. arXiv:1708.05144 [cs] T. Haarnoja, A. Zhou, K. Hartikainen, G. Tucker, S. Ha, J. Tan, V. Kumar, H. Zhu, A. Gupta, P. Abbeel, and S. Levine, ‘‘Soft actor-critic algorithms and applications,’’ 2019. arXiv:1812.05905 [cs.LG] Traffic Lights — SUMO Documentation. https://sumo.dlr.de/docs/Simulation/Traffic_Lights.html. Cited June 8, 2022. TAPAS Cologne — SUMO Documentation. https://sumo.dlr.de/docs/Data/Scenarios/TAPASCologne. html.Cited June 8, 2022. RESCO: Reinforcement learning benchmarks for traffic signal control, 2021. https://github.com/Pi-Star-Lab/RESCO. Cited June 8, 2022. PFRL: A PyTorch-based deep reinforcement learning library. https://github.com/pfnet/pfrl. Cited June 8, 2022.