Real-time data-driven dynamic scheduling for flexible job shop with insufficient transportation resources using hybrid deep Q network

Li, 2020, An improved artificial bee colony algorithm for solving multi-objective low-carbon flexible job shop scheduling problem, Appl. Soft Comput., 95, 10.1016/j.asoc.2020.106544 Brandimarte, 1993, Routing and scheduling in a flexible job shop by tabusearch, Ann. Oper. Res., 41, 157, 10.1007/BF02023073 Dai, 2019, Multi-objective optimization for energy-efficient flexible job shop scheduling problem with transportation constraints, Rob. Comput. Integr. Manuf., 59, 143, 10.1016/j.rcim.2019.04.006 Zhang, 2020, Evolving Scheduling Heuristics via Genetic Programming With Feature Selection in Dynamic Flexible Job-Shop Scheduling, IEEE Trans. Cybernet. Cui, 2020, Manufacturing big data ecosystem: A systematic literature review, Rob. Comput. Integr. Manuf., 62, 10.1016/j.rcim.2019.101861 Oztemel, 2020, Literature review of Industry 4.0 and related technologies, J. Intell. Manuf., 31, 127, 10.1007/s10845-018-1433-8 Cao, 2020, An integrated processing energy modeling and optimization of automated robotic polishing system, Rob. Comput. Integr. Manuf., 65, 10.1016/j.rcim.2020.101973 Fan, 2021, A machining accuracy informed adaptive positioning method for finish machining of assembly interfaces of large-scale aircraft components, Rob. Comput. Integr. Manuf., 67, 10.1016/j.rcim.2020.102021 Zhang, 2021, A discrete whale swarm algorithm for hybrid flow-shop scheduling problem with limited buffers, Rob. Comput. Integr. Manuf., 68, 10.1016/j.rcim.2020.102081 Liang, 2021, Logistics-involved QoS-aware service composition in cloud manufacturing with deep reinforcement learning, Rob. Comput. Integr. Manuf., 67, 10.1016/j.rcim.2020.101991 Hu, 2020, Deep reinforcement learning based AGVs real-time scheduling with mixed rule for flexible shop floor in industry 4.0, Comput. Ind. Eng., 149, 10.1016/j.cie.2020.106749 Amin, 2018, A minimax linear programming model for dispatching rule selection, Comput. Ind. Eng., 121, 27, 10.1016/j.cie.2018.05.021 Jing, 2021, Local search-based metaheuristics for the robust distributed permutation flowshop problem, Appl. Soft Comput., 105, 10.1016/j.asoc.2021.107247 Luo, 2020, Solving the dynamic energy aware job shop scheduling problem with the heterogeneous parallel genetic algorithm, Fut. Gen. Comp. Syst. Int. J. eSci., 108, 119, 10.1016/j.future.2020.02.019 Xu, 2021, Genetic Programming with Delayed Routing for Multiobjective Dynamic Flexible Job Shop Scheduling, Evol. Comput., 29, 75, 10.1162/evco_a_00273 Nguyen, 2019, A Hybrid Genetic Programming Algorithm for Automated Design of Dispatching Rules, Evol. Comput., 27, 467, 10.1162/evco_a_00230 Zhang, 2021, Correlation Coefficient-Based Recombinative Guidance for Genetic Programming Hyperheuristics in Dynamic Flexible Job Shop Scheduling, IEEE Trans. Evol. Comput., 25, 552, 10.1109/TEVC.2021.3056143 Zhang, 2021, Evolving Scheduling Heuristics via Genetic Programming With Feature Selection in Dynamic Flexible Job-Shop Scheduling, IEEE Trans. Cybernet., 51, 1797, 10.1109/TCYB.2020.3024849 Fang, 2019, Digital-Twin-Based Job Shop Scheduling Toward Smart Manufacturing, IEEE Trans. Ind. Inf., 15, 6425, 10.1109/TII.2019.2938572 Abidi, 2020, Optimal scheduling of flexible manufacturing system using improved lion-based hybrid machine learning approach, IEEE Access, 8, 96088, 10.1109/ACCESS.2020.2997663 Cheng, 2020, Data mining for fast and accurate makespan estimation in machining workshops, J. Intell. Manuf. Feng, 2020, Integrated intelligent green scheduling of sustainable flexible workshop with edge computing considering uncertain machine state, J. Cleaner Prod., 246, 10.1016/j.jclepro.2019.119070 Wang, 2019, Multiagent and Bargaining-Game-Based Real-Time Scheduling for Internet of Things-Enabled Flexible Job Shop, IEEE Internet of Things J., 6, 2518, 10.1109/JIOT.2018.2871346 Li, 2021, An effective MCTS-based algorithm for minimizing makespan in dynamic flexible job shop scheduling problem, Comput. Ind. Eng., 155, 10.1016/j.cie.2021.107211 Tian, 2021, A dynamic job-shop scheduling model based on deep learning, Adv. Prod. Eng. Manage., 16, 23 Zhang, 2012, Minimizing mean weighted tardiness in unrelated parallel machine scheduling with reinforcement learning, Comp. Oper. Res., 39, 1315, 10.1016/j.cor.2011.07.019 Xanthopoulos, 2013, Intelligent controllers for bi-objective dynamic scheduling on a single machine with sequence-dependent setups, Appl. Soft Comput., 13, 4704, 10.1016/j.asoc.2013.07.015 Wang, 2016, An interoperable adaptive scheduling strategy for knowledgeable manufacturing based on SMGWQ-learning, J. Intell. Manuf., 27, 1085, 10.1007/s10845-014-0936-1 Shahrabi, 2017, A reinforcement learning approach to parameter estimation in dynamic job shop scheduling, Comput. Ind. Eng., 110, 75, 10.1016/j.cie.2017.05.026 Shiue, 2018, Real-time scheduling for a smart factory using a reinforcement learning approach, Comput. Ind. Eng., 125, 604, 10.1016/j.cie.2018.03.039 Wang, 2020, Adaptive job shop scheduling strategy based on weighted Q-learning algorithm, J. Intell. Manuf., 31, 417, 10.1007/s10845-018-1454-3 Lin, 2019, Smart manufacturing scheduling with edge computing using multiclass deep Q network, IEEE Trans. Ind. Inf., 15, 4276, 10.1109/TII.2019.2908210 Luo, 2020, Dynamic scheduling for flexible job shop with new job insertions by deep reinforcement learning, Appl. Soft Comput., 91, 10.1016/j.asoc.2020.106208 Park, 2020, A reinforcement learning approach to robust scheduling of semiconductor manufacturing facilities, IEEE Trans. Autom. Sci. Eng., 17, 1420 Shi, 2020, Intelligent scheduling of discrete automated production line via deep reinforcement learning, Int. J. Prod. Res., 58, 3362, 10.1080/00207543.2020.1717008 Wu, 2020, Real-time neural network scheduling of emergency medical mask production during COVID-19, Appl. Soft Comput., 97, 10.1016/j.asoc.2020.106790 Liu, 2020, Actor-Critic Deep Reinforcement Learning for Solving Job Shop Scheduling Problems, IEEE Access, 8, 71752, 10.1109/ACCESS.2020.2987820 Han, 2020, Research on Adaptive Job Shop Scheduling Problems Based on Dueling Double DQN, IEEE Access, 8, 186474, 10.1109/ACCESS.2020.3029868 V. Mnih, K. Kavukcuoglu, D. Silver, et al., Playing atari with deep reinforcement learning, arXiv (2013) preprint arXiv:1312.5602. Mnih, 2015, Human-level control through deep reinforcement learning, Nature, 518, 529, 10.1038/nature14236 van Hasselt, 2010, Double Q-learning, Adv. Neur. Inform. Process. Syst., 23, 2613 H. van Hasselt, A. Guez, D. Silver, Deep reinforcement learning with double Q-learning, in Proc. 30th AAAI Conf. Artif. Intell. (2016) 2094–2100. T. Schaul, J. Quan, I. Antonoglou, D. Silver, Prioritized experience replay, in Proc. 4th Int. Conf. Learn. Represent. (ICLR), San Juan, PR, May 2016. T.P. Lillicrap, J.J. Hunt, A. Pritzel, et al., Continuous control with deep reinforcement learning, arXiv (2015) preprint arXiv:1509.02971. Xi, 2012, Scheduling jobs on identical parallel machines with unequal future ready time and sequence dependent setup: An experimental study, Int. J. Prod. Econ., 137, 1, 10.1016/j.ijpe.2012.01.026 Huber, 1964, Robust estimation of a location parameter, Ann. Math. Statist., 35, 73, 10.1214/aoms/1177703732 He, 2015, Delving deep into rectifiers: surpassing human-level performance on imagenet classification, CVPR IEEE Comp. Soc. Bergstra, 2012, Random search for hyper-parameter optimization, J. Mach. Learn. Res., 13, 281 Golemo, 2018, Sim-to-real transfer with neural-augmented robot simulation, PMLR, 87, 817 Z.P. He, R. Julian, E. Heiden, et al., Zero-shot skill composition and simulation-to-real transfer by learning task representations, arXiv (2021) arXiv:1810.02422v3. Hu, 2020, Petri-net-based dynamic scheduling of flexible manufacturing system via deep reinforcement learning with graph convolutional network, J. Manuf. Syst., 55, 1, 10.1016/j.jmsy.2020.02.004