A tutorial on optimal control and reinforcement learning methods for quantum technologies

Acín, 2018, The quantum technologies roadmap: a European community view, New J. Phys., 20, 10.1088/1367-2630/aad1ea Glaser, 2015, Training Schrödinger's cat: quantum optimal control: strategic report on current status, visions and goals for research in Europe, Eur. Phys. J. D, 69, 279, 10.1140/epjd/e2015-60464-1 Boscain, 2021, Introduction to the Pontryagin maximum principle for quantum optimal control, PRX Quantum, 2, 10.1103/PRXQuantum.2.030203 Rembold, 2020, Introduction to quantum optimal control for quantum sensing with nitrogen-vacancy centers in diamond, AVS Quantum Sci., 2, 10.1116/5.0006785 Wilhelm Liberzon, 2012 D'Alessandro, 2021 Bressan, 2007, Introduction to the Mathematical Theory of Control: With 102 Figures and 107 Exercises, vol. 2 Berry, 2009, Transitionless quantum driving, J. Phys. A, Math. Theor., 42, 10.1088/1751-8113/42/36/365303 Guéry-Odelin, 2019, Shortcuts to adiabaticity: concepts, methods, and applications, Rev. Mod. Phys., 91, 10.1103/RevModPhys.91.045001 Dunjko, 2018, Machine learning & artificial intelligence in the quantum domain: a review of recent progress, Rep. Prog. Phys., 81, 10.1088/1361-6633/aab406 Carleo, 2019, Machine learning and the physical sciences, Rev. Mod. Phys., 91, 10.1103/RevModPhys.91.045002 Mehta, 2019, A high-bias, low-variance introduction to machine learning for physicists, Phys. Rep., 810, 1, 10.1016/j.physrep.2019.03.001 Marquardt, 2021, Machine Learning and Quantum Devices, 029 Alchieri, 2021, An introduction to quantum machine learning: from quantum logic to quantum deep learning, Quantum Mach. Intell., 3, 28, 10.1007/s42484-021-00056-8 Carleo, 2017, Solving the quantum many-body problem with artificial neural networks, Science, 355, 602, 10.1126/science.aag2302 Youssry, 2020, Characterization and control of open quantum systems beyond quantum noise spectroscopy, npj Quantum Inf., 6, 95, 10.1038/s41534-020-00332-8 Sgroi, 2021, Reinforcement learning approach to nonequilibrium quantum thermodynamics, Phys. Rev. Lett., 126, 10.1103/PhysRevLett.126.020601 Beer, 2020, Training deep quantum neural networks, Nat. Commun., 11, 808, 10.1038/s41467-020-14454-2 Romero, 2017, Quantum autoencoders for efficient compression of quantum data, Quantum Sci. Technol., 2, 10.1088/2058-9565/aa8072 Saggio, 2021, Experimental quantum speed-up in reinforcement learning agents, Nature, 591, 229, 10.1038/s41586-021-03242-7 Brown, 2021, Reinforcement learning-enhanced protocols for coherent population-transfer in three-level quantum systems, New J. Phys., 23, 10.1088/1367-2630/ac2393 Porotti, 2019, Coherent transport of quantum states by deep reinforcement learning, Commun. Phys., 2, 1, 10.1038/s42005-019-0169-x Paparelle, 2020, Digitally stimulated Raman passage by deep reinforcement learning, Phys. Lett. A, 384, 10.1016/j.physleta.2020.126266 Costa, 2021, Benchmarking machine learning algorithms for adaptive quantum phase estimation with noisy intermediate-scale quantum sensors, EPJ Quantum Technol., 8 Hentschel, 2010, Machine learning for precise quantum measurement, Phys. Rev. Lett., 104, 10.1103/PhysRevLett.104.063603 Hentschel, 2011, Efficient algorithm for optimizing adaptive quantum metrology processes, Phys. Rev. Lett., 107, 10.1103/PhysRevLett.107.233601 Kuklinski, 1989, Adiabatic population transfer in a three-level system driven by delayed laser pulses, Phys. Rev. A, 40, 6741, 10.1103/PhysRevA.40.6741 Bergmann, 1998, Coherent population transfer among quantum states of atoms and molecules, Rev. Mod. Phys., 70, 1003, 10.1103/RevModPhys.70.1003 Vitanov, 2017, Stimulated Raman adiabatic passage in physics, chemistry, and beyond, Rev. Mod. Phys., 89, 10.1103/RevModPhys.89.015006 Boscain, 2002, Optimal control in laser-induced population transfer for two- and three-level quantum systems, J. Math. Phys., 43, 2107, 10.1063/1.1465516 Yuan, 2012, Controllability on relaxation-free subspaces: on the relationship between adiabatic population transfer and optimal control, Phys. Rev. A, 85, 10.1103/PhysRevA.85.033417 Goerz, 2019, Krotov: a Python implementation of Krotov's method for quantum optimal control, SciPost Phys., 7, 10.21468/SciPostPhys.7.6.080 Rabi, 1954, Use of rotating coordinates in magnetic resonance problems, Rev. Mod. Phys., 26, 167, 10.1103/RevModPhys.26.167 Shore, 1990 Carmichael, 1999 Born, 1928, Beweis des adiabatensatzes, Z. Phys., 51, 165, 10.1007/BF01343193 Messiah, 1961 Arimondo, 1976, Nonabsorbing atomic coherences by coherent two-photon transitions in a three-level optical pumping, Lett. Nuovo Cimento (1971–1985), 17, 333, 10.1007/BF02746514 Fleischhauer, 1996, Propagation of laser pulses and coherent population transfer in dissipative three-level systems: an adiabatic dressed-state picture, Phys. Rev. A, 54, 794, 10.1103/PhysRevA.54.794 Giannelli, 2014, Three-level superadiabatic quantum driving, Phys. Rev. A, 89, 10.1103/PhysRevA.89.033419 Petiziol, 2020, Optimized three-level quantum transfers based on frequency-modulated optical excitations, Sci. Rep., 10, 2185, 10.1038/s41598-020-59046-8 Di Stefano, 2015, Population transfer in a Lambda system induced by detunings, Phys. Rev. B, 91, 10.1103/PhysRevB.91.224506 Di Stefano, 2016, Coherent manipulation of noise-protected superconducting artificial atoms in the Lambda scheme, Phys. Rev. A, 93, 10.1103/PhysRevA.93.051801 Falci, 2017, Advances in quantum control of three-level superconducting circuit architectures, Fortschr. Phys., 65 Falci, 2019, Ultrastrong coupling probed by coherent population transfer, Sci. Rep., 9, 9249, 10.1038/s41598-019-45187-y Ridolfo, 2021, Probing ultrastrong light–matter coupling in open quantum systems, Eur. Phys. J. Spec. Top., 230, 941, 10.1140/epjs/s11734-021-00070-8 Koch, 2016, Controlling open quantum systems: tools, achievements, and limitations, J. Phys. Condens. Matter, 28, 10.1088/0953-8984/28/21/213001 Watts, 2015, Optimizing for an arbitrary perfect entangler. I. Functionals, Phys. Rev. A, 91, 10.1103/PhysRevA.91.062306 Goerz, 2015, Optimizing for an arbitrary perfect entangler. II. Application, Phys. Rev. A, 91, 10.1103/PhysRevA.91.062307 Müller, 2011, Optimizing entangling quantum gates for physical systems, Phys. Rev. A, 84, 10.1103/PhysRevA.84.042315 Basilewitsch, 2020, Optimally controlled quantum discrimination and estimation, Phys. Rev. Res., 2, 10.1103/PhysRevResearch.2.033396 Khaneja, 2005, Optimal control of coupled spin dynamics: design of NMR pulse sequences by gradient ascent algorithms, J. Magn. Res., 172, 296, 10.1016/j.jmr.2004.11.004 Palao, 2003, Optimal control theory for unitary transformations, Phys. Rev. A, 68, 10.1103/PhysRevA.68.062308 Montangero, 2007, Robust optimal quantum gates for Josephson charge qubits, Phys. Rev. Lett., 99, 10.1103/PhysRevLett.99.170501 Said, 2009, Robust control of entanglement in a nitrogen-vacancy center coupled to a C 13 nuclear spin in diamond, Phys. Rev. A, 80, 10.1103/PhysRevA.80.032303 Goerz, 2014, Optimal control theory for a unitary operation under dissipative evolution, New J. Phys., 16, 10.1088/1367-2630/16/5/055012 Palao, 2008, Protecting coherence in optimal control theory: state-dependent constraint approach, Phys. Rev. A, 77, 10.1103/PhysRevA.77.063412 Gorini, 1976, Completely positive dynamical semigroups of N-level systems, J. Math. Phys., 17, 821, 10.1063/1.522979 Lindblad, 1976, On the generators of quantum dynamical semigroups, Commun. Math. Phys., 48, 119, 10.1007/BF01608499 Rabitz, 2004, Quantum optimally controlled transition landscapes, Science, 303, 1998, 10.1126/science.1093649 Goerz, 2015, Hybrid optimization schemes for quantum control, EPJ Quantum Technol., 2, 10.1140/epjqt/s40507-015-0034-0 Goerz, 2017, Charting the circuit QED design landscape using optimal control theory, npj Quantum Inf., 3, 1, 10.1038/s41534-017-0036-0 Basilewitsch Lucarelli, 2018, Quantum optimal control via gradient ascent in function space and the time-bandwidth quantum speed limit, Phys. Rev. A, 97, 10.1103/PhysRevA.97.062346 Krotov, 1993, Global methods in optimal control theory, 74 Konnov, 1999, On global methods for the successive improvement of control processes, Autom. Remote Control, 60, 1427 Sklarz, 2002, Loading a Bose-Einstein condensate onto an optical lattice: an application of optimal control theory to the nonlinear Schrödinger equation, Phys. Rev. A, 66, 10.1103/PhysRevA.66.053619 Reich, 2012, Monotonically convergent optimization in quantum control using Krotov's method, J. Chem. Phys., 136, 10.1063/1.3691827 Machnes, 2018, Tunable, flexible, and efficient optimization of control pulses for practical qubits, Phys. Rev. Lett., 120, 10.1103/PhysRevLett.120.150401 Doria, 2011, Optimal control technique for many-body quantum dynamics, Phys. Rev. Lett., 106, 10.1103/PhysRevLett.106.190501 Caneva, 2011, Chopped random-basis quantum optimization, Phys. Rev. A, 84, 10.1103/PhysRevA.84.022326 Müller Rach, 2015, Dressing the chopped-random-basis optimization: a bandwidth-limited access to the trap-free landscape, Phys. Rev. A, 92, 10.1103/PhysRevA.92.062343 Johansson, 2013, QuTiP 2: a Python framework for the dynamics of open quantum systems, Comput. Phys. Commun., 184, 1234, 10.1016/j.cpc.2012.11.019 Powell, 1964, An efficient method for finding the minimum of a function of several variables without calculating derivatives, Comput. J., 7, 155, 10.1093/comjnl/7.2.155 Nelder, 1965, A simplex method for function minimization, Comput. J., 7, 308, 10.1093/comjnl/7.4.308 Byrd, 1995, A limited memory algorithm for bound constrained optimization, SIAM J. Sci. Comput., 16, 1190, 10.1137/0916069 Virtanen, 2020, SciPy 1.0: fundamental algorithms for scientific computing in Python, Nat. Methods, 17, 261, 10.1038/s41592-019-0686-2 Kumar, 2016, Stimulated Raman adiabatic passage in a three-level superconducting circuit, Nat. Commun., 7, 10.1038/ncomms10628 Vepsäläinen, 2019, Superadiabatic population transfer in a three-level superconducting circuit, Sci. Adv., 5, 10.1126/sciadv.aau5999 Vepsäläinen, 2020, Simulating spin chains using a superconducting circuit: gauge invariance, superadiabatic transport, and broken time-reversal symmetry, Adv. Quantum Tech., 3, 10.1002/qute.201900121 Hornik, 1989, Multilayer feedforward networks are universal approximators, Neural Netw., 2, 359, 10.1016/0893-6080(89)90020-8 Lillicrap Mnih Mnih, 2015, Human-level control through deep reinforcement learning, Nature, 518, 529, 10.1038/nature14236 Silver, 2017, Mastering the game of Go without human knowledge, Nature, 550, 354, 10.1038/nature24270 Sivak Haug, 2020, Classifying global state preparation via deep reinforcement learning, Mach. Learn. Sci. Tech., 2, 10.1088/2632-2153/abc81f Porotti Kuo Niu, 2019, Universal quantum control through deep reinforcement learning, npj Quantum Inf., 5, 1, 10.1038/s41534-019-0141-3 An, 2021, Quantum optimal control of multilevel dissipative quantum systems with reinforcement learning, Phys. Rev. A, 103, 10.1103/PhysRevA.103.012404 Borah, 2021, Measurement-based feedback quantum control with deep reinforcement learning for a double-well nonlinear potential, Phys. Rev. Lett., 127, 10.1103/PhysRevLett.127.190403 Fallani Mavadia, 2017, Prediction and real-time compensation of qubit decoherence via machine learning, Nat. Commun., 8, 10.1038/ncomms14106 Fösel, 2018, Reinforcement learning with neural networks for quantum feedback, Phys. Rev. X, 8 Moro, 2021, Quantum compiling by deep reinforcement learning, Commun. Phys., 4, 1, 10.1038/s42005-021-00684-3 Sutton, 2018 Guadarrama, 2018 Kingma Zhang, 2019, When does reinforcement learning stand out in quantum control? A comparative study on state preparation, npj Quantum Inf., 5, 85, 10.1038/s41534-019-0201-8