It's not (only) the mean that matters: variability, noise and exploration in skill learning

Shadmehr, 1994, Adaptive representation of dynamics during learning of a motor task, J Neurosci, 14, 3208, 10.1523/JNEUROSCI.14-05-03208.1994 Krakauer, 1999, Indepedent learning of internal models for kinematic and dynamic control of reaching, Nat Neurosci, 2, 1026, 10.1038/14826 Guthrie, 1935 Welford, 1968 Willingham, 1998, A neuropsychological theory of motor skill learning, Psychol Rev, 105, 558, 10.1037/0033-295X.105.3.558 Adams, 1987, Historical review and appraisal of research on the learning, retention, and transfer of human motor skills, Psychol Bull, 101, 41, 10.1037/0033-2909.101.1.41 Shmuelof, 2012, How is a motor skill learned? Change and invariance at the levels of task success and trajectory control, J Neurophysiol, 108, 578, 10.1152/jn.00856.2011 Reis, 2009, Noninvasive cortical stimulation enhances motor skill aquisition over multiple days through an effect of consolidation, Proc Natl Acad Sci, 106, 1590, 10.1073/pnas.0805413106 Faisal, 2008, Noise in the nervous system, Nat Rev Neurosci, 9, 292, 10.1038/nrn2258 van Beers, 2002, Role of uncertainty in sensorimotor control, Philos Trans R Soc Lond B, 357, 1137, 10.1098/rstb.2002.1101 Sternad, 2014, Acquisition of novel and complex motor skills: stable solutions where intrinsic noise matters less, Adv Exp Med Biol, 826, 101, 10.1007/978-1-4939-1338-1_8 Sternad, 2017, 301 Kosko, 2006 Krakauer, 2011, Human sensorimotor learning: adaptation, skill and beyond, Curr Opin Neurobiol, 21, 636, 10.1016/j.conb.2011.06.012 Saltzman, 1992, Skill acquisition and development: the roles of state-, parameter-, and graph-dynamics, J Motor Behav, 24, 84, 10.1080/00222895.1992.9941600 Newell, 2001, Time scales in motor learning and development, Psychol Rev, 108, 57, 10.1037/0033-295X.108.1.57 Braun, 2009, Motor task variation induces structural learning, Curr Biol, 19, 352, 10.1016/j.cub.2009.01.036 Mattar, 2007, Modifyability of generalization in dynamics learning, J Neurophysiol, 98, 3321, 10.1152/jn.00576.2007 Tanaka, 2009, Adaptation to visuomotor rotation through interaction between posterior parietal and motor cortical areas, J Neurophysiol, 102, 2921, 10.1152/jn.90834.2008 Shmuelof, 2012, Overcoming motor forgetting through reinforcement of learned actions, J Neurosci, 32, 14617, 10.1523/JNEUROSCI.2184-12.2012 Woodworth, 1899, The accuracy of voluntary movement, Psychol Rev Monogr Suppl, 3, 1 Shannon, 1948, A mathematical theory of communication, Bell Syst Tech J, 27, 10.1002/j.1538-7305.1948.tb01338.x Fitts, 1954, The information capacity of the human motor system in controlling the amplitude of movement, J Exp Psychol, 47, 381, 10.1037/h0055392 Plamondon, 1997, Speed/accuracy trade-offs in target-directed movements, Behav Brain Sci, 20, 1, 10.1017/S0140525X97001441 Harris, 1998, Signal-dependent noise determines motor planning, Nature, 394, 780, 10.1038/29528 Jones, 2001, Sources of signal-dependent noise during isometric force production, J Neurophysiol, 88, 1533, 10.1152/jn.2002.88.3.1533 Strogatz, 1994 Kelso, 1995 Haken, 1985, A theoretical model of phase transition in human hand movements, Biol Cybern, 51, 347, 10.1007/BF00336922 Kay, 1987, Space–time behavior of single and bimanual rhythmical movements: data and limit cycle model, J Exp Psychol Hum Percept Perform, 13, 178, 10.1037/0096-1523.13.2.178 Newell, 1993 Turvey, 1990, Coordination, Am Psychol, 45, 938, 10.1037/0003-066X.45.8.938 Schöner, 1986, A stochastic theory of phase transition in human hand movements, Biol Cybern, 53, 247, 10.1007/BF00336995 Davids, 2006 Schöner, 1988, A dynamic pattern theory of behavioral change, J Theor Biol, 135, 501, 10.1016/S0022-5193(88)80273-X Turvey, 1982, 239 Tuller, 1982, 253 Bernstein, 1967 Schröder, 1991 Scholz, 2014, Use of the uncontrolled manifold (UCM) approach to understand motor variability, motor equivalence, and self-motion, Adv Exp Med Biol, 826, 91, 10.1007/978-1-4939-1338-1_7 Scholz, 1999, The uncontrolled manifold concept: identifying control variables for a functional task, Exp Brain Res, 126, 289, 10.1007/s002210050738 Latash, 2002, Motor control strategies revealed in the structure of motor variability, Exerc Sport Sci Rev, 30, 26, 10.1097/00003677-200201000-00006 Mattos, 2011, Unpredictable elbow joint perturbation during reaching results in multijoint motor equivalence, J Neurophysiol, 106, 1424, 10.1152/jn.00163.2011 Sternad, 2010, Coordinate dependency of variability analysis, PLoS Comput Biol, 6, e1000751, 10.1371/journal.pcbi.1000751 John, 2016, Error correction and the structure of inter-trial fluctuations in a redundant movement task, PLoS Comput Biol, 10.1371/journal.pcbi.1005118 Cusumano, 2006, Body-goal variability mapping in an aiming task, Biol Cybern, 94, 367, 10.1007/s00422-006-0052-1 Müller, 2004, Decomposition of variability in the execution of goal-oriented tasks—three components of skill improvement, J Exp Psychol Hum Percept Perform, 30, 212, 10.1037/0096-1523.30.1.212 Müller, 2009, Motor learning: changes in the structure of variability in a redundant task, Adv Exp Med Biol, 629, 439, 10.1007/978-0-387-77064-2_23 Cohen, 2009, Variability in motor learning: relocating, channeling and reducing noise, Exp Brain Res, 193, 69, 10.1007/s00221-008-1596-1 Sternad, 2011, Neuromotor noise, sensitivity to error and signal-dependent noise in trial-to-trial learning, PLoS Comput Biol, 7, e1002159, 10.1371/journal.pcbi.1002159 Dingwell, 2012, Trial-to-trial dynamics and learning in a generalized redundant reaching task, J Neurophysiol, 109, 225, 10.1152/jn.00951.2011 Abe, 2013, Directionality in distribution and temporal structure of variability in skill acquisition, Frontiers Hum Neurosci, 7, 10.3389/fnhum.2013.00225 Cohen, 2012, State space analysis of intrinsic timing: exploiting task redundancy to reduce sensitivity to timing, J Neurophysiol, 107, 618, 10.1152/jn.00568.2011 Zhang, 2018, Exploiting the geometry of the solution space to reduce sensitivity to neuromotor noise, PLoS Comput Biol, 14, e1006013, 10.1371/journal.pcbi.1006013 Nasu, 2014, Two types of motor strategy for accurate dart throwing, PLoS ONE, 9, e88536, 10.1371/journal.pone.0088536 Müller, 1999, Functional variability and an equifinal path of movement during targeted throwing, J Hum Mov Stud, 36, 103 Chu, 2016, Dystonic children can learn a novel motor skill: strategies that are tolerant to high variability, IEEE Trans Neural Syst Rehabil Eng, 24, 847, 10.1109/TNSRE.2016.2521404 Venkasedan, 2017, Optimal strategies for throwing accurately, R Soc Open Sci, 4, 170136, 10.1098/rsos.170136 Sabes, 1997, Obstacle avoidance and a perturbation sensitivity model for motor planning, J Neurosci, 17, 7119, 10.1523/JNEUROSCI.17-18-07119.1997 Sabes, 1998, The role of inertial sensitivity in motor planning, J Neurosci, 18, 5948, 10.1523/JNEUROSCI.18-15-05948.1998 Dhawale, 2017, The role of variability in motor learning, Annu Rev Neurosci, 40, 479, 10.1146/annurev-neuro-072116-031548 Newell, 1989, 650 McDonald, 1995, Perceptual-motor exploration as a function of biomechanical and task constraints, Acta Psychol, 88, 127, 10.1016/0001-6918(93)E0056-8 Adolph, 2000, Exploration in the service of prospective control, Infant Behav Dev, 23, 441, 10.1016/S0163-6383(01)00052-2 Wu, 2014, Temporal structure of motor variability is dynamically regulated and predicts motor learning abiltiy, Nat Neurosci, 17, 312, 10.1038/nn.3616 He, 2016, The statistical determinants of the speed of motor learning, PLoS Comput Biol, 12, e1005023, 10.1371/journal.pcbi.1005023 Singh, 2016, Exploration of joint redundancy but not task space variability facilitates supervised motor learning, Proc Natl Acad Sci, 113, 14414, 10.1073/pnas.1613383113 Cardis, 2018, High variability impairs motor learning regardless of whether it affects task performance, J Neurophys, 119, 39, 10.1152/jn.00158.2017 Mehler, 2017, Minimizing endpoijt variability through reinforcement learning during reaching movements involving shoulder, elbow and wrist, PLoS ONE, 10.1371/journal.pone.0180803 Barbado, 2017, Can the structure o fmotor variability predict learning rate?, J Exp Psychol Hum Percept Perform, 43, 596, 10.1037/xhp0000303 LaValle, 2001, Randomized kinodynamic planning, Int J Robot Res, 20, 10.1177/02783640122067453 Cabrera, 2004, Human stick balancing: tuning Levy flights to improve balance control, Chaos, 14, 691, 10.1063/1.1785453 Cluff, 2009, Motor learning characterized by changing Levy distributions, PLoS ONE, 4, e5998, 10.1371/journal.pone.0005998 Kuznetsov, 2014 Turner, 2007, Performance variability enables plasticity of “crystallized” adult birdsong, Nature, 450, 1240, 10.1038/nature06390 Pekny, 2015, Reward-dependent modulation of movement variability, J Neurosci, 35, 4015, 10.1523/JNEUROSCI.3244-14.2015 Hasson, 2016, Neuromotor noise is malleable by amplification of perceived error, PLoS Comput Biol, 12, e1005044, 10.1371/journal.pcbi.1005044 Huber, 2016, Reducing neuromotor noise in long-term motor skill learning, J Neurophysiol, 116, 2922, 10.1152/jn.00263.2016 Wei, 2014, Serotonin affects movement gain control in the spinal cord, J Neurosci, 34, 12690, 10.1523/JNEUROSCI.1855-14.2014 Picard, 2013, Extended practice of a motor skill is associated with reduced metabolic activity in M1, Nat Neurosci, 16, 1340, 10.1038/nn.3477 Manley, 2014, When money is not enough: awareness, success, and variability in motor learning, PLoS ONE, 9, e86580, 10.1371/journal.pone.0086580 Mehler, 2017, Minimizing endpoint variability through reinforcement learning during reaching movements involving shoulder, elbow and wrist, PLoS ONE, 10.1371/journal.pone.0180803 Thorp, 2017, Using noise to shape motor learning, J Neurophysiol, 117, 728, 10.1152/jn.00493.2016 Chu, 2013, Healthy and dystonic children compensate for changes in motor variability, J Neurophysiol, 109, 2169, 10.1152/jn.00908.2012 Ajemian, 2013, A theory for how sensorimotor skills are learned and retained in noisy and nonstationary neural circuits, Proc Natl Acad Sci, 110, E5078, 10.1073/pnas.1320116110 Yu, 2011, Spine plasticity, Curr Opin Neurobiol, 21, 169, 10.1016/j.conb.2010.07.010 Schmidt, 2005 Therrien, 2016, Effective reinforcement learning following cerebellar damage requires a balance between exploration and motor noise, Brain, 139, 101, 10.1093/brain/awv329 Park, 2013, Learning to never forget: time scales and specificity of long-term memory of a motor skill, Frontiers Comput Neurosci, 7, 10.3389/fncom.2013.00111 Park, 2015, Robust retention of individual sensorimotor skill after self-guided practice, J Neurophysiol, 113, 2635, 10.1152/jn.00884.2014 Riley, 2002, Variability of determinism in motor behavior, J Motor Behav, 34, 99, 10.1080/00222890209601934 Ranganathan, 2013, Changing up the routine: intervention-induced variability in motor learning, Exerc Sport Sci Rev, 41, 64, 10.1097/JES.0b013e318259beb5 Diedrichsen, 2010, The coordination of movement: optimal feedback control and beyond, Trends Cogn Sci, 14, 31, 10.1016/j.tics.2009.11.004 Dayan, 2011, Neuroplasticity subserving motor skill learning, Neuron, 72, 443, 10.1016/j.neuron.2011.10.008 Priplata, 2003, Vibrating insoles and balance control in elderly people, Lancet, 362, 1123, 10.1016/S0140-6736(03)14470-4 Bloch-Salisbury, 2009, Stabilizing immature breathing patterns of preterm infants using stochastic mechanosensory stimulation, J Appl Physiol, 107, 1017, 10.1152/japplphysiol.00058.2009 McDonnell, 2011, The benefits of noise in neural systems: bridging theory and experiment, Nat Rev Neurosci, 12, 415, 10.1038/nrn3061 McDonnell, 2009, What is stochastic resonance? Definitions, musconceptions, debates, and its relevance to biology, PLoS Comput Biol, 5, e1000348, 10.1371/journal.pcbi.1000348 Eke, 2002, Fractal characterization of complexity in temporal physiological signals, Physiol Meas, 23, R1, 10.1088/0967-3334/23/1/201 Goldberger, 2002, Fractal dynamics in physiology: alterations with disease and aging, Proc Natl Acad Sci USA, 99, 2466, 10.1073/pnas.012579499 Pincus, 1991, Approximate entropy as a measure of system complexity, Proc Natl Acad Sci, 88, 2297, 10.1073/pnas.88.6.2297 Stergiou, 2011, Human movement variability, nonlinear dynamics, and pathology: is there a connection?, Hum Mov Sci, 30, 869, 10.1016/j.humov.2011.06.002 Yin, 2015, Noisy training for deep neural networks in speech recognition, EURASIP J Audio Speech Music Process, 2, 1 Miller, 1990, An empirical study of the reliability of UNIX utilities, Commun ACM, 33, 32, 10.1145/96267.96279 Plappert M, et al. Parameter space noise for exploration in deep reinforcement learning. arXiv preprint arXiv:1706.01905; 2017.