Particle swarm optimization based on temporal-difference learning for solving multi-objective optimization problems
Tóm tắt
Multi-objective evolutionary algorithms have become the most important method to deal with multi-objective optimization problems (MOP). To improve the performance of particle swarm optimization (PSO) in addressing MOPs, a multi-objective PSO based on temporal-difference learning (TDLMOPSO) is proposed in this paper. The iteration process of TDLMOPSO is transformed into a Markov decision process, particles are treated as agents, each agent has a personal archive, the states are designed for the connection of actions, the actions of particles contain all necessary behavior of them: basic movement, jump out of local optimum, and local search, and the rewards depend on the relationship between particles’ positions and their personal archives. Besides, the external archive deletion strategy and the leader selection strategy are redesigned based on the unsupervised learning algorithm to enhance the diversity of solutions in the external archive. The effectiveness of TDLMOPSO is verified by applying it with other seven advanced multi-objective algorithms in MOP benchmark test suites. Furthermore, the time complexity and parameter sensitivity of TDLMOPSO are analyzed.
Tài liệu tham khảo
Zhu G-Y, He L-J, Ju X-W, Zhang W-B (2018) A fitness assignment strategy based on the grey and entropy parallel analysis and its application to MOEA. Eur J Oper Res 265(3):813–828
Zhu G-Y, Ding C, Zhang W-B (2020) Optimal foraging algorithm that incorporates fuzzy relative entropy for solving many-objective permutation flow shop scheduling problems. IEEE Trans Fuzzy Syst 28(11):2738–2746
Han F, Chen W-T, Ling Q-H, Han H (2021) Multi-objective particle swarm optimization with adaptive strategies for feature selection. Swarm Evol Comput 62:100847
Ding R, Dong H, He J, Li T (2019) A novel two-archive strategy for evolutionary many-objective optimization algorithm based on reference points. Appl Soft Comput 78:447–464
Zhu Q, Lin Q, Chen W, Wong K-C, Coello CAC, Li J, Chen J, Zhang J (2017) An external archive-guided multiobjective particle swarm optimization algorithm. IEEE Transa Cyber 47(9):2794–2808
Santos R, Borges G, Santos A, Silva M, Sales C, Costa JC (2020) A rotationally invariant semi-autonomous particle swarm optimizer with directional diversity. Swarm Evol Comput 56:100700
Xu Y, Pi D (2020) A reinforcement learning-based communication topology in particle swarm optimization. Neural Comput Appl 32(14):10007–10032
Wang F, Wang X, Sun S (2022) A reinforcement learning level-based particle swarm optimization algorithm for large-scale optimization. Inf Sci 602:298–312
Zhang X, Xia S, Li X, Zhang T (2022) Multi-objective particle swarm optimization with multi-mode collaboration based on reinforcement learning for path planning of unmanned air vehicles. Knowl-Based Syst 250:109075
Meerza SIA, Islam M, Uzzal MM (2019) Q-learning based particle swarm optimization algorithm for optimal path planning of swarm of mobile robots. In: 2019 1st international conference on advances in science, engineering and robotics technology (ICASERT), pp 1–5. IEEE
Liu Y, Lu H, Cheng S, Shi Y (2019) An adaptive online parameter control algorithm for particle swarm optimization based on reinforcement learning. In: 2019 IEEE congress on evolutionary computation (CEC), pp 815–822. IEEE
Huang Y, Li W, Tian F, Meng X (2020) A fitness landscape ruggedness multiobjective differential evolution algorithm with a reinforcement learning strategy. Appl Soft Comput 96:106693
Zhao H, Zhang C (2020) A decomposition-based many-objective artificial bee colony algorithm with reinforcement learning. Appl Soft Comput 86:105879
Li T, Yin Y, Yang B, Hou J, Zhou K (2022) A self-learning bee colony and genetic algorithm hybrid for cloud manufacturing services. Computing 104:1977–2003
Oldewage ET, Engelbrecht AP, Cleghorn CW (2018) The importance of component-wise stochasticity in particle swarm optimization. International conference on swarm intelligence. Springer, Berlin, pp 264–276
Asghari A, Sohrabi MK (2021) Combined use of coral reefs optimization and reinforcement learning for improving resource utilization and load balancing in cloud environments. Computing 103(7):1545–1567
Zhou Z-H (2016) Machine learning. Tsinghua University Press, Beijing
He C, Cheng R, Yazdani D (2020) Adaptive offspring generation for evolutionary large-scale multiobjective optimization. IEEE Trans Syst Man Cyber Syst 52:786–798
Zhang X, Zheng X, Cheng R, Qiu J, Jin Y (2018) A competitive mechanism based multi-objective particle swarm optimizer with fast convergence. Inf Sci 427:63–76
Zille H (2019) Large-scale multi-objective optimisation: new approaches and a classification of the state-of-the-art
Farias LR, Araújo AF (2021) Im-moea/d: An inverse modeling multi-objective evolutionary algorithm based on decomposition. In: 2021 ieee international conference on systems, man, and cybernetics (SMC), pp 462–467. IEEE
Qin S, Sun C, Jin Y, Tan Y, Fieldsend J (2021) Large-scale evolutionary multiobjective optimization assisted by directed sampling. IEEE Trans Evol Comput 25(4):724–738
Tian Y, Si L, Zhang X, Tan KC, Jin Y (2022) Local model-based pareto front estimation for multiobjective optimization. IEEE Trans Syst Man Cyber Syst 53:623–634
Liu Q, Jin Y, Heiderich M, Rodemann T, Yu G (2020) An adaptive reference vector-guided evolutionary algorithm using growing neural gas for many-objective optimization of irregular problems. IEEE Trans Cyber 52(5):2698–2711
Zitzler E, Deb K, Thiele L (2000) Comparison of multiobjective evolutionary algorithms: Empirical results. Evol Comput 8(2):173–195
Deb K, Thiele L, Laumanns M, Zitzler E (2002) Scalable multi-objective optimization test problems. In: Proceedings of the 2002 congress on evolutionary computation. CEC’02 (Cat. No. 02TH8600), vol 1, pp 825–830. IEEE
Cheng R, Li M, Tian Y, Zhang X, Yang S, Jin Y, Yao X (2017) A benchmark test suite for evolutionary many-objective optimization. Complex Intell Syst 3(1):67–81
Van Veldhuizen DA, Lamont GB (2000) On measuring multiobjective evolutionary algorithm performance. In: Proceedings of the 2000 congress on evolutionary computation. CEC00 (Cat. No. 00TH8512), vol 1, pp 204–211. IEEE
Mohammadi A, Omidvar MN, Li X (2013) A new performance metric for user-preference based multi-objective evolutionary algorithms. In: 2013 IEEE congress on evolutionary computation, pp 2825–2832. IEEE
Zitzler E, Thiele L (1999) Multiobjective evolutionary algorithms: a comparative case study and the strength pareto approach. IEEE Trans Evol Comput 3(4):257–271