Deep Reinforcement Learning for Load Balancing of Edge Servers in IoV
Tóm tắt
In recent years, the use of edge computing to solve the problem of limited resources in the IoV has attracted more and more attention. Vehicles can upload tasks to the edge servers within their radio range for computing cooperation and offloading. However, due to shared resources among edge servers, the uneven distribution of vehicles may lead to many problems such as uneven tasks distribution, unbalanced load, and low computing efficiency on different edge servers. On the other hand, most of the existing works are proposed to address the load unbalance issue of edge servers with the help of remote cloud or vehicle cloud, but still leading to resource wasting problems especially for edge servers with a light load. To address this issue, a task transfer scheme has been proposed among different edge servers in this paper. First, we designed a partitioned and hierarchical software-defined network architecture for IoV. After that, a load balancing model is proposed under this network architecture, in which the global controller can manage and schedule the local tasks from each edge server. Next, we proved that the task allocation problem for our load balancing model under the proposed network architecture is an NP-hard problem. To solve this problem, a Deep Q-Network (DQN) model is proposed to minimize the mean square deviation of loads among different edge servers. Numerical results show that our proposed load balancing model can significantly improve the resource utilization of edge servers as well as reduce the computational latency of vehicular tasks.
Tài liệu tham khảo
Ray PP (2016) A survey on internet of things architectures. J King Saud University - Comput Inf Sci 30:291–319
Chen C, Liu L, Wan S, Hui X, Pei Q (2021) Data dissemination for industry 4.0 applications in internet of vehicles based on short-term traffic prediction. ACM Trans Internet Technol (TOIT) 22:1–18
Gao H, Liu C, Yin Y, Xu Y, Li Y (2021) A hybrid approach to trust node assessment and management for vanets cooperative data communication: Historical interaction perspective. IEEE Trans Intell Transp Syst. https://doi.org/10.1109/TITS.2021.3129458
Li X, Dang Y, Aazam M, Peng X, Chen T, Chen C (2020) Energy-efficient computation offloading in vehicular edge cloud computing. IEEE Access 8:37632–37644
Sharma S, Kaushik BN (2019) A survey on internet of vehicles: applications, security issues and solutions. Veh Commun 2. https://doi.org/10.1016/j.vehcom.2019.100182
Xu W, Zhou H, Cheng N, Lyu F, Shi W, Chen J, Shen X (2018) Internet of vehicles in big data era. IEEE/CAA J Automatica Sinica 5:19–35
Cong W, Chen C, Qingqi P, Zhiyuan J, Shugong X (2021) An information centric in-network caching scheme for 5G-enabled internet of connected vehicles. IEEE Trans Mobile Comput. https://doi.org/10.1109/TMC.2021.3137219
Wan S, Ding S, Chen C (2022) Edge computing enabled video segmentation for real-time traffic monitoring in internet of vehicles. Pattern Recogn 121:108146
Chen C, Zeng Y, Li H, Liu Y, Wan S (2022) A multi-hop task offloading decision model in MEC-enabled internet of vehicles. IEEE Internet Things J, 1–1. https://doi.org/10.1109/JIOT.2022.3143529
Bozkaya E, Canberk B (2015) QoE-based flow management in software defined vehicular networks. In: 2015 IEEE Globecom workshops (GC Wkshps)
Liu S, Yu J, Deng X, Wan S (2021) FedCPF: An efficient-communication federated learning approach for vehicular edge computing in 6G communication networks. IEEE Trans Intell Trans Syst
Yin Y, Huang Q, Gao H, Xu Y (2020) Personalized APIs recommendation with cognitive knowledge mining for industrial systems. IEEE Trans Industr Inform PP(99):1–1
Patel KD, Bhalodia TM (2019) An Efficient Dynamic Load Balancing Algorithm for Virtual Machine in Cloud Computing. In: 2019 International conference on intelligent computing and control systems (ICCS)
Chien NK, Son NH, Loc HD (2016) Load balancing algorithm based on estimating finish time of services in cloud computing. In: 2016 18th International conference on advanced communication technology (ICACT)
Tamilarasi P, Akila D (2021) Prediction based load balancing and VM migration in big data cloud environment. In: 2021 2nd International conference on computation, automation and knowledge management (ICCAKM)
Maswood M, Rahman MR, Alharbi AG, Medhi D (2020) A novel strategy to achieve bandwidth cost reduction and load balancing in a cooperative three-layer fog-cloud computing environment. IEEE Access PP(99):1–1
Wang KY, Kao SJ, Kao MT (2018) An efficient load adjustment for balancing multiple controllers in reliable sdn systems. In: 2018 IEEE International conference on applied system innovation (ICASI)
Li J, Lei B, Li N, Lv H (2020) A load balancing approach for distributed SDN architecture based on sharing data store. In: 2020 21st Asia-pacific network operations and management symposium (APNOMS)
Li J, Yang L, Wang J, Yang S (2018) Research on SDN Load balancing based on ant colony optimization algorithm. In: IEEE Information technology and mechatronics engineering conference (ITOEC), pp 979–982. https://doi.org/10.1109/ITOEC.2018.8740366
Patil S (2018) Load balancing approach for finding best path in SDN. In: 2018 International conference on inventive research in computing applications (ICIRCA)
Nkosi MC, Lysko AA, Dlamini S (2018) Multi-path load balancing for SDN data plane. In: 2018 International conference on intelligent and innovative computing applications (ICONIC). IEEE, pp 1–6
Liu L, Jiang Y, Shen G, Li Q, Lin D, Li L, Wang Y (2019) An SDN-based hybrid strategy for load balancing in data center networks. In: 2019 IEEE Symposium on computers and communications (ISCC). IEEE, pp 1–6
Hu J, Chen C, Cai L, Khosravi MR, Pei Q, Wan S (2021) UAV-assisted vehicular edge computing for the 6G internet of vehicles: architecture, intelligence, and challenges. IEEE Commun Standards Magazine 5(2):12–18
Yang L, Yao H, Wang J, Jiang C, Liu Y (2019) Multi-UAV enabled load-balance mobile edge computing for IoT networks. IEEE Internet Things J 7(8):6898–6908. https://doi.org/10.1109/JIOT.2020.2971645
Li L, Chan S, Han G, Guizani M, Bandai M (2018) Performance modeling of representative load sharing schemes for clustered servers in multiaccess edge computing. IEEE Internet Things J PP (99):1–1
Chen C, Zhang Y, Wang Z, Wan S, Pei Q (2021) Distributed computation offloading method based on deep reinforcement learning in icv. Appl Soft Comput 103:107108
Li G, Yao Y, Wu J, Liu X, Lin Q (2020) A new load balancing strategy by task allocation in edge computing based on intermediary nodes. EURASIP Journal on Wireless Communications and Networking 2020
Babou C, Fall D, Kashihara S, Taenaka Y, Kadobayashi Y (2020) Hierarchical load balancing and clustering technique for home edge computing. IEEE Access PP:1–1
Tham CK, Chattopadhyay R (2017) A load balancing scheme for sensing and analytics on a mobile edge computing network. IEEE International Symposium on A World of Wireless, 1–9
Ma X, Xu H, Gao H, Bian M (2021) Real-time multiple-workflow scheduling in cloud environment. IEEE Trans Netw Serv Manag 18:4002–4018
Huang Y, Xu H, Gao H, Ma X, Hussain W (2021) SSUR: An approach to optimizing virtual machine allocation strategy based on user requirements for cloud data center. IEEE Trans Green Commun Netw 5:670–681
Wei W, Yang R, Gu H, Zhao W, Chen C, Wan S (2021) Multi-objective optimization for resource allocation in vehicular cloud computing networks. IEEE Trans Intell Transp Syst
Gao H, Zhang Y, Miao H, Barroso RJD, Yang X (2021) SDTIOA: Modeling the timed privacy requirements of iot service composition: a user interaction perspective for automatic transformation from BPEL to timed automata. Mobile Networks and Applications, 1–26