Particle swarm optimization based on temporal-difference learning for solving multi-objective optimization problems

Computing - Tập 105 - Trang 1795-1820 - 2023
Desong Zhang1, Guangyu Zhu1
1School of Mechanical Engineering and Automation, Fuzhou University, Fuzhou, People’s Republic of China

Tóm tắt

Multi-objective evolutionary algorithms have become the most important method to deal with multi-objective optimization problems (MOP). To improve the performance of particle swarm optimization (PSO) in addressing MOPs, a multi-objective PSO based on temporal-difference learning (TDLMOPSO) is proposed in this paper. The iteration process of TDLMOPSO is transformed into a Markov decision process, particles are treated as agents, each agent has a personal archive, the states are designed for the connection of actions, the actions of particles contain all necessary behavior of them: basic movement, jump out of local optimum, and local search, and the rewards depend on the relationship between particles’ positions and their personal archives. Besides, the external archive deletion strategy and the leader selection strategy are redesigned based on the unsupervised learning algorithm to enhance the diversity of solutions in the external archive. The effectiveness of TDLMOPSO is verified by applying it with other seven advanced multi-objective algorithms in MOP benchmark test suites. Furthermore, the time complexity and parameter sensitivity of TDLMOPSO are analyzed.

Tài liệu tham khảo

Zhu G-Y, He L-J, Ju X-W, Zhang W-B (2018) A fitness assignment strategy based on the grey and entropy parallel analysis and its application to MOEA. Eur J Oper Res 265(3):813–828 Zhu G-Y, Ding C, Zhang W-B (2020) Optimal foraging algorithm that incorporates fuzzy relative entropy for solving many-objective permutation flow shop scheduling problems. IEEE Trans Fuzzy Syst 28(11):2738–2746 Han F, Chen W-T, Ling Q-H, Han H (2021) Multi-objective particle swarm optimization with adaptive strategies for feature selection. Swarm Evol Comput 62:100847 Ding R, Dong H, He J, Li T (2019) A novel two-archive strategy for evolutionary many-objective optimization algorithm based on reference points. Appl Soft Comput 78:447–464 Zhu Q, Lin Q, Chen W, Wong K-C, Coello CAC, Li J, Chen J, Zhang J (2017) An external archive-guided multiobjective particle swarm optimization algorithm. IEEE Transa Cyber 47(9):2794–2808 Santos R, Borges G, Santos A, Silva M, Sales C, Costa JC (2020) A rotationally invariant semi-autonomous particle swarm optimizer with directional diversity. Swarm Evol Comput 56:100700 Xu Y, Pi D (2020) A reinforcement learning-based communication topology in particle swarm optimization. Neural Comput Appl 32(14):10007–10032 Wang F, Wang X, Sun S (2022) A reinforcement learning level-based particle swarm optimization algorithm for large-scale optimization. Inf Sci 602:298–312 Zhang X, Xia S, Li X, Zhang T (2022) Multi-objective particle swarm optimization with multi-mode collaboration based on reinforcement learning for path planning of unmanned air vehicles. Knowl-Based Syst 250:109075 Meerza SIA, Islam M, Uzzal MM (2019) Q-learning based particle swarm optimization algorithm for optimal path planning of swarm of mobile robots. In: 2019 1st international conference on advances in science, engineering and robotics technology (ICASERT), pp 1–5. IEEE Liu Y, Lu H, Cheng S, Shi Y (2019) An adaptive online parameter control algorithm for particle swarm optimization based on reinforcement learning. In: 2019 IEEE congress on evolutionary computation (CEC), pp 815–822. IEEE Huang Y, Li W, Tian F, Meng X (2020) A fitness landscape ruggedness multiobjective differential evolution algorithm with a reinforcement learning strategy. Appl Soft Comput 96:106693 Zhao H, Zhang C (2020) A decomposition-based many-objective artificial bee colony algorithm with reinforcement learning. Appl Soft Comput 86:105879 Li T, Yin Y, Yang B, Hou J, Zhou K (2022) A self-learning bee colony and genetic algorithm hybrid for cloud manufacturing services. Computing 104:1977–2003 Oldewage ET, Engelbrecht AP, Cleghorn CW (2018) The importance of component-wise stochasticity in particle swarm optimization. International conference on swarm intelligence. Springer, Berlin, pp 264–276 Asghari A, Sohrabi MK (2021) Combined use of coral reefs optimization and reinforcement learning for improving resource utilization and load balancing in cloud environments. Computing 103(7):1545–1567 Zhou Z-H (2016) Machine learning. Tsinghua University Press, Beijing He C, Cheng R, Yazdani D (2020) Adaptive offspring generation for evolutionary large-scale multiobjective optimization. IEEE Trans Syst Man Cyber Syst 52:786–798 Zhang X, Zheng X, Cheng R, Qiu J, Jin Y (2018) A competitive mechanism based multi-objective particle swarm optimizer with fast convergence. Inf Sci 427:63–76 Zille H (2019) Large-scale multi-objective optimisation: new approaches and a classification of the state-of-the-art Farias LR, Araújo AF (2021) Im-moea/d: An inverse modeling multi-objective evolutionary algorithm based on decomposition. In: 2021 ieee international conference on systems, man, and cybernetics (SMC), pp 462–467. IEEE Qin S, Sun C, Jin Y, Tan Y, Fieldsend J (2021) Large-scale evolutionary multiobjective optimization assisted by directed sampling. IEEE Trans Evol Comput 25(4):724–738 Tian Y, Si L, Zhang X, Tan KC, Jin Y (2022) Local model-based pareto front estimation for multiobjective optimization. IEEE Trans Syst Man Cyber Syst 53:623–634 Liu Q, Jin Y, Heiderich M, Rodemann T, Yu G (2020) An adaptive reference vector-guided evolutionary algorithm using growing neural gas for many-objective optimization of irregular problems. IEEE Trans Cyber 52(5):2698–2711 Zitzler E, Deb K, Thiele L (2000) Comparison of multiobjective evolutionary algorithms: Empirical results. Evol Comput 8(2):173–195 Deb K, Thiele L, Laumanns M, Zitzler E (2002) Scalable multi-objective optimization test problems. In: Proceedings of the 2002 congress on evolutionary computation. CEC’02 (Cat. No. 02TH8600), vol 1, pp 825–830. IEEE Cheng R, Li M, Tian Y, Zhang X, Yang S, Jin Y, Yao X (2017) A benchmark test suite for evolutionary many-objective optimization. Complex Intell Syst 3(1):67–81 Van Veldhuizen DA, Lamont GB (2000) On measuring multiobjective evolutionary algorithm performance. In: Proceedings of the 2000 congress on evolutionary computation. CEC00 (Cat. No. 00TH8512), vol 1, pp 204–211. IEEE Mohammadi A, Omidvar MN, Li X (2013) A new performance metric for user-preference based multi-objective evolutionary algorithms. In: 2013 IEEE congress on evolutionary computation, pp 2825–2832. IEEE Zitzler E, Thiele L (1999) Multiobjective evolutionary algorithms: a comparative case study and the strength pareto approach. IEEE Trans Evol Comput 3(4):257–271