Path selection in disaster response management based on Q-learning
Tóm tắt
Suitable rescue path selection is very important to rescue lives and reduce the loss of disasters, and has been a key issue in the field of disaster response management. In this paper, we present a path selection algorithm based on Q-learning for disaster response applications. We assume that a rescue team is an agent, which is operating in a dynamic and dangerous environment and needs to find a safe and short path in the least time. We first propose a path selection model for disaster response management, and deduce that path selection based on our model is a Markov decision process. Then, we introduce Q-learning and design strategies for action selection and to avoid cyclic path. Finally, experimental results show that our algorithm can find a safe and short path in the dynamic and dangerous environment, which can provide a specific and significant reference for practical management in disaster response applications.
Tài liệu tham khảo
A. Mansourian, A. Rajabifard, M. J. V. Zoej, I. Williamson. Using SDI and web-based system to facilitate disaster management. Computers and Geosciences, vol. 32, no. 3, pp. 303–315, 2006.
K. Friesen, D. Bell. Regional health authorities, disaster management, and geomatics: Opportunities and barriers. International Journal of Emergency Management, vol.4, no. 2, pp. 141–165, 2007.
W. E. Roper. Waste management policy revisions: Lessons learned from the Katrina disaster. International Journal of Environmental Technology and Management, vol.8, no. 2/3, pp. 275–309, 2008.
M. H. Tsai, L. Y. Liu, F. P. Mora, C. Arboleda. A preliminary design of disaster-survivable building blackbox system for Urban disaster response. Electronic Journal of Information Technology in Construction, vol. 13, no. 2, pp. 179–192, 2008.
P. G. Bello, I. Aedo, F. Sainz, P. Diaz, J. Munnelly, S. Clarke. Improving communication for mobile devices in disaster response. In Proceedings of the 1st International Conference on Mobile Information Technology for Emergency Response, Lecture Notes in Computer Science, Springer, vol. 4458, pp. 126–134, 2007.
F. R. Yu, J. Zhang, H. Tang, H. C. B. Chan, V. C. M. Leung. Enhancing interoperability in heterogeneous mobile wireless networks for disaster response. In Proceedings of IEEE Military Communications Conference, Orlando, Florida, USA, pp. 1–7, 2007.
Télécoms Sans Frontières. Emergency Communications Aid Disaster Response, Geneva: International Telecommunications Union, pp. 34, 2005.
S. Thompson, N. Altay, W. G. Green III, J. Lepetina. Improving disaster response efforts with decision support systems. International Journal of Emergency Management, vol. 3, no. 4, pp. 250–263, 2006.
Y. L. Tu, W. J. Zhang, X. Liu, W. Li, C. L. Chai, R. Deters. A disaster response management system based on the control systems technology. International Journal of Critical Infrastructures, vol. 4, no. 3, pp. 274–295, 2008.
Z. W. Hu, X. J. Li, Y. H. Sun, L. Zhu. Flood disaster response and decision-making support system based on remote sensing and GIS. In Proceedings of International Geoscience and Remote Sensing Symposium, Barcelona, Spain, pp. 2435–2438, 2008.
F. Fiedrich, P. Burghardt. Agent-based systems for disaster management. Communications of the ACM, vol. 50, no. 3, pp. 41–42, 2007.
L. L. Yang, R. Prasanna, M. King. On-site information systems design for emergency first responders. Journal of Information Technology Theory and Application, vol. 10, no. 1, pp. 5–27, 2009.
M. Turoff, M. Chumer, B. Van deWalle, X. Yao. The design of a dynamic emergency response management information system. Journal of Information Technology Theory and Application, vol. 5, no. 4, pp. 1–36, 2004.
A. F. Chandio, L. Y. Shu, N. M. Memon, A. Khawaja. GIS based route guiding system for optimal path planning in disaster/crisis management. In Proceedings of the 10th IEEE International Multitopic Conference, Islamabad, Pakistan, pp. 207–210, 2006.
L. Ozdamar, W. Yi. Greedy neighborhood search for disaster relief and evacuation logistics. IEEE Intelligent Systems, vol. 13, no. 1, pp. 14–23, 2008.
Y. C. Chiou, Y. H. Lai. An integrated multi-objective model to determine the optimal rescue path and traffic controlled arcs for disaster relief operations under uncertainty environments. Journal of Advanced Transportation, vol. 42, no. 4, pp. 493–519, 2008.
Y. Yuan, D. W. Wang. Path selection model and algorithm for emergency logistics management. Computers and Industrial Engineering, vol. 56, no. 3, pp. 1081–1094, 2009.
E. G. Lopez. Efficient graph-based genetic programming representation with multiple outputs. International Journal of Automation and Computing, vol. 5, no. 1, pp. 81–89, 2008.
L. P. Kaelbling, M. L. Littman, A. W. Moore. Reinforcement learning: A survey. Journal of Artificial Intelligence Research, vol. 4, no. 1, pp. 237–285, 1996.
M. Kumar, A. K. Verma, A. Srividya. Analyzing effect of demand rate on safety of systems with periodic proof-tests. International Journal of Automation and Computing, vol.4, no. 4, pp. 335–341, 2007.
C. J. C. H. Watkins, P. Dayan. Technical note: Q-learning. Machine Learning, vol. 8, no. 3–4, pp. 279–292, 1992.
V. S. Borkar. Q-learning for risk-sensitive control, Mathematics of Operations Research, vol. 27, no. 2, pp. 294–311, 2002.
M. Andrecut, M. K. Ali. Q learning in the minority game. Physical Review E, vol. 64, no. 6, pp. 1–4, 2001.
J. G. Jiang, Z. P. Su, M. B. Qi, G. F. Zhang. Multi-task coalition parallel formation strategy based on reinforcement learning. Acta Automatica Sinica, vol. 34, no. 3, pp. 349–352, 2008.
S. M. Lucas. Computational intelligence and games: Challenges and opportunities. International Journal of Automation and Computing, vol. 5, no. 1, pp. 45–57, 2008.
S. Q. Yu, H. Q. Wang, F. M. Ye, S. Mabu, K. Shimada, K. Hirasawa. A Q value-based dynamic programming algorithm with Boltzmann distribution for optimizing the global traffic routing strategy. In Proceedings of SICE Annual Conference, Tokyo, Japan, pp. 619–622, 2008.
K. Takadama, T. Kawai, Y. Koyama. Can agents acquire human-like behaviors in a sequential bargaining game? — Comparison of Roth’s and Q-learning agents. In Proceedings of the 7th International Workshop on Multi-agent-based Simulation, Hakodate, Japan, pp. 156–171, 2006.