A review of approximate dynamic programming applications within military operations research

Hausman, 1969, Sequential decision problems: A model to exploit existing forecasters, Manage Sci, 16, B93, 10.1287/mnsc.16.2.B93 Puterman, 2005 Powell, 2011 Bellman, 1957, A markovian decision process, J Math Mech, 6, 670 Bellman, 1957 Anderson, 1988, A decision support system for the procurement of military equipment, Nav Res Logist, 35, 619, 10.1002/1520-6750(198808)35:4<619::AID-NAV3220350413>3.0.CO;2-L McGinnis M, Fernandez-Gaucherand E. A dynamic programming model for the initial entry training program of the United States Army. In: Proceedings of 33rd IEEE conference on decision and control, vol. 4. 1994. p. 3632–3. Pecht, 2013, On the choice of multi-task R & D defence projects: A case study of the Israeli missle defence system, Defence Peace Econ, 24, 429, 10.1080/10242694.2012.717205 Zais, 2016, A markov chain model of military personnel dynamics, Int J Prod Res, 54, 1863, 10.1080/00207543.2015.1108533 Keneally, 2016, A markov decision process model for the optimal dispatch of military medical evacuation assets, Health Care Manag Sci, 19, 111, 10.1007/s10729-014-9297-8 Kuo, 2005, Lifting the curse of dimensionality, Notices Amer Math Soc, 52, 1320 Bertsekas, 1996 Sutton, 2018 Bellman, 1959, Functional approximations and dynamic programming, Vol. 13, 247 Powell, 2016, Perspectives of approximate dynamic programming, Ann Oper Res, 241, 319, 10.1007/s10479-012-1077-6 Birge, 2011 Powell W. The optimizing-simulator: Merging simulation and optimization using approximate dynamic programming. In: Proceedings of the winter simulation conference, 2005. p. 96–109. Powell, 2011, The effect of robust decisions on the cost of uncertainty in military airlift operations, ACM Trans Model Comput Simul, 22, 10.1145/2043635.2043636 Bertsekas, 2019 Powell, 2021 Powell, 2002, Implementing real-time optimization models: A case application from the motor carrier industry, Oper Res, 50, 571, 10.1287/opre.50.4.571.2852 Powell, 2007 Simão, 2010, Approximate dynamic programming captures fleet operations for Schneider National, INFORMS J Appl Anal, 40, 342, 10.1287/inte.1100.0510 Simão, 2009, An approximate dynamic programming algorithm for large-scale fleet management: A case application, Transp Sci, 43, 178, 10.1287/trsc.1080.0238 Powell, 2014, Locomotive planning at Norfolk Southern: An optimizing simulator using approximate dynamic programming, INFORMS J Appl Anal, 44, 567, 10.1287/inte.2014.0741 Simão, 2008, Approximate dynamic programming for management of high-value spare parts, J Manuf Technol Manag, 20, 147, 10.1108/17410380910929592 Schramm, 2019 Jiang, 2018, Risk-averse approximate dynamic programming with quantile-based risk measures, Math Oper Res, 43, 554, 10.1287/moor.2017.0872 Powell, 2014, Clearing the jungle of stochastic optimization, 109 Powell, 2009, What you should know about approximate dynamic programming, Nav Res Logist, 56, 239, 10.1002/nav.20347 Mes, 2017, Approximate dynamic programming by practical examples, 63 McKenna, 2020, Approximate dynamic programming for the military inventory routing problem, Ann Oper Res, 288, 391, 10.1007/s10479-019-03469-8 Hastie, 2017 Powell, 2019, A unified framework for stochastic optimization, European J Oper Res, 275, 795, 10.1016/j.ejor.2018.07.014 Spall, 2003, Introduction to stochastic search and optimization: Estimation, simulation, and control Bhatnagar, 2013 Bertsekas, 2011, Approximate policy iteration: a survey and some new methods, J Control Theory Appl, 9, 310, 10.1007/s11768-011-1005-3 Geist, 2013, Algorithmic survey of parametric value function approximation, IEEE Trans Neural Netw Learn Syst, 24, 845, 10.1109/TNNLS.2013.2247418 Bradtke, 1996, Linear least-squares algorithms for temporal difference learning, Mach Learn, 22, 33, 10.1007/BF00114723 Nedić, 2003, Least squares policy evaluation algorithms with linear function approximation, Discrete Event Dyn Syst, 13, 79, 10.1023/A:1022192903948 Bethke B, How JP, Ozdaglar A. Approximate dynamic programming using support vector regression. In: 2008 47th IEEE conference on decision and control, 2008. p. 3811–6. George, 2006, Adaptive stepsizes for recursive estimation with applications in approximate dynamic programming, Mach Learn, 65, 167, 10.1007/s10994-006-8365-9 Barr, 1995, Designing and reporting on computational experiments with heuristic methods, J Heuristics, 1, 9, 10.1007/BF02430363 Flint, 2009, Simulation analysis for UAV search algorithm design using approximate dynamic programming, Military Oper Res, 14, 41, 10.5711/morj.14.2.41 Fisher, 2015, An approximate dynamic programming heuristic to support non-strategic project selection for the Royal Canadian Navy, J Defense Model Simul, 12, 83, 10.1177/1548512913509031 Southerland, 2018, Using approximate dynamic programming to model military force mix adaptation, Military Oper Res, 23, 25 Hoecherl JC, Robbins MJ, Hill RR, Ahner DK. Approximate dynamic programming algorithms for United States Air Force officer sustainment. In: 2016 winter simulation conference (WSC), 2016. p. 3075–86. Ross K, Chaney R, Patek S. Neuro-dynamic programming for adaptive control of bayesian networks for global awareness. In: 1998 IEEE information technology conference, information environment for the future (Cat. No. 98EX228), 1998. p. 10–3. Bertsekas, 2000, Missile defense and interceptor allocation by neuro-dynamic programming, IEEE Trans Syst Man Cybern, 30, 42, 10.1109/3468.823480 Popken, 2004, A simulation–optimization approach to air warfare planning, J Defense Model Simul, 1, 127, 10.1177/875647930400100301 Sztykgold A, Coppin G, Hudry O. Dynamic optimization of the strength ratio during a terrestrial conflict. In: 2007 IEEE international symposium on approximate dynamic programming and reinforcement learning, 2007. p. 241–6. Wu, 2009, The optimizing-simulator: An illustration using the military airlift problem, ACM Trans Model Comput Simul, 19, 10.1145/1540530.1540535 Ahner, 2013, Weapon tradeoff analysis using dynamic programming for a dynamic weapon target assignment problem within a simulation, 2831 Rettke, 2016, Approximate dynamic programming for the dispatch of military medical evacuation assets, European J Oper Res, 254, 824, 10.1016/j.ejor.2016.04.017 Davis, 2017, Approximate dynamic programming for missile defence interceptor fire control, European J Oper Res, 259, 873, 10.1016/j.ejor.2016.11.023 Laan, 2018, 171 Robbins, 2020, Approximate dynamic programming for the aeromedical evacuation dispatching problem: Value function approximation utilizing multiple level aggregation, Omega, 91, 10.1016/j.omega.2018.12.009 Summers, 2020, An approximate dynamic programming approach for comparing firing policies in a networked air defence environment, Comput Oper Res, 117, 10.1016/j.cor.2020.104890 Jenkins, 2021, Approximate dynamic programming for the military aeromedical evacuation dispatching, preemption-rerouting, and redeployment problem, European J Oper Res, 290, 132, 10.1016/j.ejor.2020.08.004 Jenkins, 2021, Approximate dynamic programming for military medical evacuation dispatching policies, INFORMS J Comput, 33, 2, 10.1287/ijoc.2019.0930 Coifman, 2006, Diffusion wavelets, Appl Comput Harmon Anal, 21, 53, 10.1016/j.acha.2006.04.004 Balakrishna, 2009 Southerland, 2017 Godfrey, 2002, An adaptive dynamic programming algorithm for dynamic fleet management, I: Single period travel times, Transp Sci, 36, 21, 10.1287/trsc.36.1.21.570 Bradshaw, 2016 West, 2017 Situ, 2018 Powell, 2004, Learning algorithms for separable approximations of discrete stochastic optimization problems, Math Oper Res, 29, 814, 10.1287/moor.1040.0107 Salgado, 2016 Government of Canada, 2021 Brown, 2004, Optimizing military capital planning, Interfaces, 34, 415, 10.1287/inte.1040.0107 Rempel, 2017 Harrison, 2020, Portfolio optimization for defence applications, IEEE Access, 8, 60152, 10.1109/ACCESS.2020.2983141 Gallo, 2018 Sacco, 2005, Precise formulation and evidence-based application of resource-constrained triage, Acad Emerg Med, 12, 759, 10.1197/j.aem.2005.04.003 Saran, 2019 Bakhshi, 2020 Scott, 2021 Teeple, 2020 Walker, 2001, Adaptive policies, policy analysis, and policy-making, European J Oper Res, 128, 282, 10.1016/S0377-2217(00)00071-0 Stasko, 2012, Developing green fleet management strategies: Repair/retrofit/replacement decisions under environmental regulation, Transp Res A, 46, 1216 Abdul-Malak, 2018, Optimally replacing multiple systems in a shared environment, Probab Engrg Inform Sci, 32, 179, 10.1017/S026996481700016X Sadeghpour, 2019, A novel approximate dynamic programming approach for constrained equipment replacement problems: A case study, Adv Prod Eng Manag, 14, 355 Fang, 2013, Sourcing strategies in supply risk management: An approximate dynamic programming approach, Comput Oper Res, 40, 1371, 10.1016/j.cor.2012.08.016 Geng, 2014 Ghanmi A. A stochastic model for military air-to-ground munitions demand forecasting. In: 2016 3rd international conference on logistics operations management (GOL), 2016. p. 1–8. Nozhati, 2020, Stochastic optimal control methodologies in risk-informed community resilience planning, Struct Saf, 84, 10.1016/j.strusafe.2019.101920 Karamanis, 2013 MacLeod, 2019, Decision support for optimal use of joint training funds in the Canadian Armed Forces, 255 Séguin, 2015, PARSim, a simulation model of the Royal Canadian Air Force (RCAF) pilot occupation: An assessment of the pilot occupation sustainability under high student production and reduced flying rates, 51 Hunter, 2021 Shin, 2020, Emergency medical service resource allocation in a mass casualty incident by integrating patient prioritization and hospital selection problems, IISE Trans, 52, 1141, 10.1080/24725854.2020.1727069 Sidoti, 2020, Context-aware dynamic asset allocation for maritime interdiction operations, IEEE Trans Syst Man Cybern, 50, 1055, 10.1109/TSMC.2017.2767568 Rempel, 2021