Embedding fuzzy mechanisms and knowledge in box-type reinforcement learning controllers
IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics) - Tập 32 Số 5 - Trang 645-653 - 2002
Tóm tắt
In this paper, we report our study on embedding fuzzy mechanisms and knowledge into box-type reinforcement learning controllers. One previous approach for incorporating fuzzy mechanisms can only achieve one successful run out of nine tests compared to eight successful runs in a nonfuzzy learning control scheme. After analysis, the credit assignment problem and the weighting domination problem are identified. Furthermore, the use of fuzzy mechanisms in temporal difference seems to play a negative factor. Modifications to overcome those problems are proposed. Furthermore, several remedies are employed in that approach. The effects of those remedies applied to our learning scheme are presented and possible variations are also studied. Finally, the issue of incorporating knowledge into reinforcement learning systems is studied. From our simulations, it is concluded that the use of knowledge for the control network can provide good learning results, but the use of knowledge for the evaluation network alone seems unable to provide any significant advantages. Furthermore, we also employ Makarovic's (1988) rules as the knowledge for the initial setting of the control network. In our study, the rules are separated into four groups to avoid the ordering problem.
Từ khóa
#Fuzzy control #Supervised learning #Control systems #Unsupervised learning #System performance #Testing #Learning systems #Control system synthesis #Neural networks #Fuzzy systemsTài liệu tham khảo
10.1109/91.273126
10.1109/72.159061
10.1109/ICSMC.1999.816639
schoknecht, 1999, using reinforcement learning for engine control, Proceedings of 9th International Conference on Artificial Neural Networks ICANN 99, 1, 329, 10.1049/cp:19991130
10.1109/IJCNN.1999.833414
10.1109/ICIT.2000.854247
10.1109/3477.836376
10.1007/BF00115009
hsieh, 1997, On the study of embedding fuzzy concept and prior knowledge in reinforcement learning
kim, 1995, Analysis and study of neural-networks-based reinforcement learning
10.1109/TAC.1965.1098193
10.1109/JRPROC.1961.287775
10.1109/TSMC.1983.6313077
10.1147/rd.33.0210
10.1002/int.4550060105
10.1109/37.24809
lin, 1996, Neural Fuzzy System A Neuro-Fuzzy Synergism to Intelligent Systems
kokar, 1992, learning control methods, needs, and architectures, An Introduction to Intelligent and Autonomous Control
10.1080/02533839.2001.9670633
10.1109/TSMC.1985.6313399
yager, 1994, Essentials of Fuzzy Modeling and Control
10.1109/FUZZY.1992.258745
10.1109/21.97472
10.1109/TAC.1997.633847
makarovic, 1988, A qualitative way of solving the pole balancing problem