Principles of sensorimotor learning
Tóm tắt
Từ khóa
Tài liệu tham khảo
Itti, L. & Koch, C. Computational modelling of visual attention. Nature Rev. Neurosci. 2, 194–203 (2001).
Najemnik, J. & Geisler, W. S. Optimal eye movement strategies in visual search. Nature 434, 387–391 (2005). The authors derive an ideal Bayesian observer to select the eye movement that gains the most information about target location in a cluttered environment. By examining humans in this task they show that they are nearly optimal compared to this model, suggesting that eye movements are chosen to maximize information about target location.
Land, M. & Tatler, B. Looking and Acting: Vision and Eye Movements in Natural Behaviour (Oxford Univ. Press, New York, 2009).
Land, M. & McLeod, P. From eye movements to actions: how batsmen hit the ball. Nature Neurosci. 3, 1340–1345 (2000).
Sailer, U., Flanagan, J. R. & Johansson, R. S. Eye-hand coordination during learning of a novel visuomotor task. J. Neurosci. 25, 8833–8842 (2005).
Seki, K., Perlmutter, S. & Fetz, E. Sensory input to primate spinal cord is presynaptically inhibited during voluntary movement. Nature Neurosci. 6, 1309–1316 (2003).
Bays, P. M. & Wolpert, D. M. Computational principles of sensorimotor control that minimize uncertainty and variability. J. Physiol. 578, 387–396 (2007).
Most, S. B., Scholl, B. J., Clifford, E. R. & Simons, D. J. What you see is what you set: sustained inattentional blindness and the capture of awareness. Psychol. Rev. 112, 217–242 (2005).
Triesch, J., Ballard, D. H., Hayhoe, M. M. & Sullivan, B. T. What you see is what you need. J. Vis. 3, 86–94 (2003).
Green, C. S. & Bavelier, D. Action video game modifies visual selective attention. Nature 423, 534–537 (2003).
Nasir, S. M. & Ostry, D. J. Auditory plasticity and speech motor learning. Proc. Natl Acad. Sci. USA 106, 20470–20475 (2009).
Faisal, A. A., Selen, L. P. J. & Wolpert, D. M. Noise in the nervous system. Nature Rev. Neurosci. 9, 292–303 (2008).
Ernst, M. & Bulthoff, H. Merging the senses into a robust percept. Trends Cogn. Sci. 8, 162–169 (2004).
Takahashi, C., Diedrichsen, J. & Watt, S. J. Integration of vision and haptics during tool use. J. Vis. 9, 3 (2009).
Kording, K. & Wolpert, D. M. Bayesian decision theory in sensorimotor control. Trends Cogn. Sci. 10, 319–326 (2006).
Vaziri, S., Diedrichsen, J. & Shadmehr, R. Why does the brain predict sensory consequences of oculomotor commands? Optimal integration of the predicted and the actual sensory feedback. J. Neurosci. 26, 4188–4197 (2006).
Gold, J. I. & Shadlen, M. N. The neural basis of decision making. Annu. Rev. Neurosci. 30, 535–574 (2007).
Cisek, P. & Kalaska, J. F. Neural mechanisms for interacting with a world full of action choices. Annu. Rev. Neurosci. 33, 269–298 (2010).
Resulaj, A., Kiani, R., Wolpert, D. M. & Shadlen, M. N. Changes of mind in decision-making. Nature 461, 263–266 (2009).
Kahneman, D. & Tversky, A. Choices, Values, and Frames (Cambridge Univ. Press, Cambridge, UK, 2000).
Trommershäuser, J., Maloney, L. T. & Landy, M. S. Decision making, movement planning and statistical decision theory. Trends Cogn. Sci. 12, 291–297 (2008).
Nagengast, A. J., Braun, D. A. & Wolpert, D. M. Risk-sensitive optimal feedback control accounts for sensorimotor behavior under uncertainty. PLoS Comput. Biol. 6, e1000857 (2010).
Braun, D. A., Nagengast, A. J. & Wolpert, D. M. Risk-sensitivity in sensorimotor control. Front. Hum. Neurosci. 5, 1 (2011).
Nagengast, A. J., Braun, D. A. & Wolpert, D. M. Risk-sensitivity and the mean-variance trade-off: decision making in sensorimotor control. Proc. Biol. Sci. 278, 2325–2332 (2011).
Johansson, R. S. & Flanagan, J. R. Coding and use of tactile signals from the fingertips in object manipulation tasks. Nature Rev. Neurosci. 10, 345–359 (2009).
Flanagan, J. R., Bowman, M. C. & Johansson, R. S. Control strategies in object manipulation tasks. Curr. Opin. Neurobiol. 16, 650–659 (2006).
Wagner, M. J. & Smith, M. A. Shared internal models for feedforward and feedback control. J. Neurosci. 28, 10663–10673 (2008).
Pruszynski, J. A., Kurtzer, I. & Scott, S. H. Rapid motor responses are appropriately tuned to the metrics of a visuospatial task. J. Neurophysiol. 100, 224–238 (2008).
Pruszynski, J. A., Kurtzer, I., Lillicrap, T. P. & Scott, S. H. Temporal evolution of “automatic gain-scaling”. J. Neurophysiol. 102, 992–1003 (2009).
Diedrichsen, J. Optimal task-dependent changes of bimanual feedback control and adaptation. Curr. Biol. 17, 1675–1679 (2007).
Todorov, E. & Jordan, M. I. Optimal feedback control as a theory of motor coordination. Nature Neurosci. 5, 1226–1235 (2002).
Valero-Cuevas, F. J., Venkadesan, M. & Todorov, E. Structured variability of muscle activations supports the minimal intervention principle of motor control. J. Neurophysiol. 102, 59–68 (2009).
Burdet, E., Osu, R., Franklin, D., Milner, T. & Kawato, M. The central nervous system stabilizes unstable dynamics by learning optimal impedance. Nature 414, 446–449 (2001).
Franklin, D. W. et al. Endpoint stiffness of the arm is directionally tuned to instability in the environment. J. Neurosci. 27, 7705–7716 (2007).
Franklin, D. W. et al. CNS learns stable, accurate, and efficient movements using a simple algorithm. J. Neurosci. 28, 11165–11173 (2008). A new model of motor learning in which stability, accuracy and efficiency are simultaneously optimized by specifying how feedforward commands to individual muscles are adjusted based on error. This model is the first to account for the temporal evolution of both net force and impedance control during learning.
Mitrovic, D., Klanke, S., Osu, R., Kawato, M. & Vijayakumar, S. A computational model of limb impedance control based on principles of internal model uncertainty. PLoS ONE 5, e13601 (2010).
Marr, D. Vision: A Computational Investigation into the Human Representation and Processing of Visual Information (Freeman, New York, 1982).
Martin, T., Keating, J., Goodkin, H., Bastian, A. & Thach, W. Throwing while looking through prisms. I. Focal olivocerebellar lesions impair adaptation. Brain 119, 1183–1198 (1996).
Pelisson, D., Alahyane, N., Panouillères, M. & Tilikete, C. Sensorimotor adaptation of saccadic eye movements. Neurosci. Biobehav. Rev. 34, 1103–1120 (2010).
Shadmehr, R. & Mussa-Ivaldi, F. A. Adaptive representation of dynamics during learning of a motor task. J. Neurosci. 14, 3208–3224 (1994).
Thoroughman, K. A. & Shadmehr, R. Learning of action through adaptive combination of motor primitives. Nature 407, 742–747 (2000).
Krakauer, J., Pine, Z., Ghilardi, M. F. & Ghez, C. Learning of visuomotor transformations for vectorial planning of reaching trajectories. J. Neurosci. 20, 8916–8924 (2000).
Flanagan, J. R. & Wing, A. M. The role of internal models in motion planning and control: evidence from grip force adjustments during movements of hand-held loads. J. Neurosci. 17, 1519–1528 (1997).
van Beers, R. J. Motor learning is optimally tuned to the properties of motor noise. Neuron 63, 406–417 (2009).
Donchin, O., Francis, J. T. & Shadmehr, R. Quantifying generalization from trial-by-trial behavior of adaptive systems that learn with basis functions: theory and experiments in human motor control. J. Neurosci. 23, 9032–9045 (2003).
Diedrichsen, J., Hashambhoy, Y., Rane, T. & Shadmehr, R. Neural correlates of reach errors. J. Neurosci. 25, 9919–9931 (2005).
Srimal, R., Diedrichsen, J., Ryklin, E. B. & Curtis, C. E. Obligatory adaptation of saccade gains. J. Neurophysiol. 99, 1554–1558 (2008).
Diedrichsen, J., Verstynen, T., Lehman, S. L. & Ivry, R. B. Cerebellar involvement in anticipating the consequences of self-produced actions during bimanual movements. J. Neurophysiol. 93, 801–812 (2005).
Smith, M. A. & Shadmehr, R. Intact ability to learn internal models of arm dynamics in Huntington's disease but not cerebellar degeneration. J. Neurophysiol. 93, 2809–2821 (2005).
Morton, S. M. & Bastian, A. J. Cerebellar contributions to locomotor adaptations during splitbelt treadmill walking. J. Neurosci. 26, 9107–9116 (2006).
Tseng, Y.-W., Diedrichsen, J., Krakauer, J. W., Shadmehr, R. & Bastian, M. S. Sensory prediction errors drive cerebellum-dependent adaptation of reaching. J. Neurophysiol. 98, 54–62 (2007).
Golla, H. et al. Reduced saccadic resilience and impaired saccadic adaptation due to cerebellar disease. Eur. J. Neurosci. 27, 132–144 (2008).
Baraduc, P., Lang, N., Rothwell, J. C. & Wolpert, D. M. Consolidation of dynamic motor learning is not disrupted by rTMS of primary motor cortex. Curr. Biol. 14, 252–256 (2004).
Della-Maggiore, V., Malfait, N., Ostry, D. J. & Paus, T. Stimulation of the posterior parietal cortex interferes with arm trajectory adjustments during the learning of new dynamics. J. Neurosci. 24, 9971–9976 (2004).
Hadipour-Niktarash, A., Lee, C. K., Desmond, J. E. & Shadmehr, R. Impairment of retention but not acquisition of a visuomotor skill through time-dependent disruption of primary motor cortex. J. Neurosci. 27, 13413–13419 (2007).
Galea, J. M., Vazquez, A., Pasricha, N., Orban de Xivry, J.-J. & Celnik, P. Dissociating the roles of the cerebellum and motor cortex during adaptive learning: the motor cortex retains what the cerebellum learns. Cereb. Cortex 21, 1761–1770 (2011). This study showed that nodal transcranial direct current stimulation (tDCS) stimulation of the cerebellum increases the rate of error-based learning during adaptation of reaching movements under a visuomotor rotation. Anodal tDCS of primary motor cortex does not change the rate of learning, but makes the adaptation more resistant to washout.
Mosier, K. M., Scheidt, R. A., Acosta, S. & Mussa-Ivaldi, F. A. Remapping hand movements in a novel geometrical environment. J. Neurophysiol. 94, 4362–4372 (2005).
Johansson, R. S. et al. How a lateralized brain supports symmetrical bimanual tasks. PLoS Biol. 4, e158 (2006).
Liu, X., Mosier, K. M., Mussa-Ivaldi, F. A., Casadio, M. & Scheidt, R. A. Reorganization of finger coordination patterns during adaptation to rotation and scaling of a newly learned sensorimotor transformation. J. Neurophysiol. 105, 454–473 (2011).
Reis, J. et al. Noninvasive cortical stimulation enhances motor skill acquisition over multiple days through an effect on consolidation. Proc. Natl Acad. Sci. USA 106, 1590–1595 (2009).
Sutton, R. S. & Barto, A. G. Reinforcement Learning (The MIT Press, Cambridge, Massachusetts, 1998).
Deisenroth, M., Rasmussen, C. & Peters, J. Gaussian process dynamic programming. Neurocomputing 72, 1508–1524 (2009).
Izawa, J. & Shadmehr, R. Learning from sensory and reward prediction errors during motor adaptation. PLoS Comput. Biol. 7, e1002012 (2011).
Madelain, L., Paeye, C. & Wallman, J. Modification of saccadic gain by reinforcement. J. Neurophysiol. 106, 219–232 (2011).
Abe, M. et al. Reward improves long-term retention of a motor memory through induction of offline memory gains. Curr. Biol. 21, 557–562 (2011).
Huang, V. S., Haith, A., Mazzoni, P. & Krakauer, J. W. Rethinking motor learning and savings in adaptation paradigms: model-free memory for successful actions combines with internal models. Neuron 70, 787–801 (2011). This paper suggests that biases in movement direction and savings in relearning during visuomotor adaptation can be explained by two different use-dependent learning mechanisms. In both cases it is the repetition of the actual movement, rather than an error-signal, that underlies the learning changes.
Luft, A. R. & Schwarz, S. Dopaminergic signals in primary motor cortex. Int. J. Dev. Neurosci. 27, 415–421 (2009).
Hosp, J. A., Pekanovic, A., Rioult-Pedotti, M. S. & Luft, A. R. Dopaminergic projections from midbrain to primary motor cortex mediate motor skill learning. J. Neurosci. 31, 2481–2487 (2011). This study shows that dopaminergic neurons from the ventral tegmental area (VTA) in the rat project to primary motor cortex. Lesions of the VTA lead to profound deficits in learning of a reaching task, which could be partially reversed through levodopa administration.
Classen, J., Liepert, J., Wise, S. P., Hallett, M. & Cohen, L. G. Rapid plasticity of human cortical movement representation induced by practice. J. Neurophysiol. 79, 1117–1123 (1998).
Bütefisch, C. M. et al. Mechanisms of use-dependent plasticity in the human motor cortex. Proc. Natl Acad. Sci. USA 97, 3661–3665 (2000).
Verstynen, T. & Sabes, P. N. How each movement changes the next: an experimental and theoretical study of fast adaptive priors in reaching. J. Neurosci. 31, 10050–10059 (2011). Repeating an arm movement towards the same target decreases the variability of this movement while inducing directional biases for movements to neighbouring targets. This paper offers both a mechanistic (network model) and normative explanation (Bayesian model) for this use-dependent learning effect.
Diedrichsen, J., White, O., Newman, D. & Lally, N. Use-dependent and error-based learning of motor behaviors. J. Neurosci. 30, 5159–5166 (2010).
Jordan, M. & Rumelhart, D. Forward models: supervised learning with a distal teacher. Cogn. Sci. 16, 307–354 (1992).
d'Avella, A., Portone, A., Fernandez, L. & Lacquaniti, F. Control of fast-reaching movements by muscle synergy combinations. J. Neurosci. 26, 7791–7810 (2006).
Sing, G. C., Joiner, W. M., Nanayakkara, T., Brayanov, J. B. & Smith, M. A. Primitives for motor adaptation reflect correlated neural tuning to position and velocity. Neuron 64, 575–589 (2009). This study shows that when subjects are exposed to a novel force field that depends only on position or on the speed of the hand, the initial adaptation is biased towards an interpretation that the force field depends on both position and speed. The interpretation of these results is that there is a strong prior that forces experienced by the hand will depend on both position and velocity in a correlated manner, suggesting that distribution of motor primitives are biased for this correlation.
Smith, M. A., Ghazizadeh, A. & Shadmehr, R. Interacting adaptive processes with different timescales underlie short-term motor learning. PLoS Biol. 4, e179 (2006). This study shows that two distinct processes underlie motor adaptation, one that learns quickly but retains information poorly and one that learns slowly but retains information well. This two-process learning system can account for a range of empirical data that a single process system cannot.
Lee, J.-Y. & Schweighofer, N. Dual adaptation supports a parallel architecture of motor memory. J. Neurosci. 29, 10396–10404 (2009).
Tanaka, H., Sejnowski, T. J. & Krakauer, J. W. Adaptation to visuomotor rotation through interaction between posterior parietal and motor cortical areas. J. Neurophysiol. 102, 2921–2932 (2009).
Lackner, J. R. & DiZio, P. Motor control and learning in altered dynamic environments. Curr. Opin. Neurobiol. 15, 653–659 (2005).
Cothros, N., Wong, J. & Gribble, P. Are there distinct neural representations of object and limb dynamics? Exp. Brain Res. 173, 689–697 (2006).
Kluzik, J., Diedrichsen, J., Shadmehr, R. & Bastian, A. Reach adaptation: what determines whether we learn an internal model of the tool or adapt the model of our arm? J. Neurophysiol. 100, 1455–1464 (2008).
Cothros, N., Wong, J. & Gribble, P. L. Visual cues signaling object grasp reduce interference in motor learning. J. Neurophysiol. 102, 2112–2120 (2009).
Berniker, M. & Kording, K. Estimating the sources of motor errors for adaptation and generalization. Nature Neurosci. 11, 1454–1461 (2008).
Haith, A., Jackson, C., Miall, C. & Vijayakumar, S. Unifying the sensory and motor components of sensorimotor adaptation. Adv. Neural Inf. Process. Syst. 21, 593–600 (2009).
Cressman, E. K. & Henriques, D. Y. P. Sensory recalibration of hand position following visuomotor adaptation. J. Neurophysiol. 102, 3505–3518 (2009).
Ostry, D. J., Darainy, M., Mattar, A. A. G., Wong, J. & Gribble, P. L. Somatosensory plasticity and motor learning. J. Neurosci. 30, 5384–5393 (2010).
Bursztyn, L. L. C. D., Ganesh, G., Imamizu, H., Kawato, M. & Flanagan, J. R. Neural correlates of internal-model loading. Curr. Biol. 16, 2440–2445 (2006).
Caithness, G. et al. Failure to consolidate the consolidation theory of learning for sensorimotor adaptation tasks. J. Neurosci. 24, 8662–8671 (2004).
Gupta, R. & Ashe, J. Lack of adaptation to random conflicting force fields of variable magnitude. J. Neurophysiol. 97, 738–745 (2007).
Krouchev, N. I. & Kalaska, J. F. Context-dependent anticipation of different task dynamics: rapid recall of appropriate motor skills using visual cues. J. Neurophysiol. 89, 1165–1175 (2003).
Nozaki, D., Kurtzer, I. & Scott, S. H. Limited transfer of learning between unimanual and bimanual skills within the same limb. Nature Neurosci. 9, 1364–1366 (2006).
Howard, I. S., Ingram, J. N. & Wolpert, D. M. Context-dependent partitioning of motor learning in bimanual movements. J. Neurophysiol. 104, 2082–2091 (2010).
Howard, I. S., Ingram, J. N. & Wolpert, D. M. Composition and decomposition in bimanual dynamic learning. J. Neurosci. 28, 10531–10540 (2008).
Nozaki, D. & Scott, S. H. Multi-compartment model can explain partial transfer of learning within the same limb between unimanual and bimanual reaching. Exp. Brain Res. 194, 451–463 (2009).
Kojima, Y., Iwamoto, Y. & Yoshida, K. Memory of learning facilitates saccadic adaptation in the monkey. J. Neurosci. 24, 7531–7539 (2004).
Kording, K. P., Tenenbaum, J. B. & Shadmehr, R. The dynamics of memory as a consequence of optimal adaptation to a changing body. Nature Neurosci. 10, 779–786 (2007).
Huang, V. S. & Shadmehr, R. Persistence of motor memories reflects statistics of the learning event. J. Neurophysiol. 102, 931–940 (2009).
Anguera, J., Reuter-Lorenz, P., Willingham, D. & Seidler, R. Contributions of spatial working memory to visuomotor learning. J. Cogn. Neurosci. 22, 1917–1930 (2009).
Fernandez-Ruiz, J., Wong, W., Armstrong, I. T. & Flanagan, J. R. Relation between reaction time and reach errors during visuomotor adaptation. Behav. Brain Res. 219, 8–14 (2011).
Keisler, A. & Shadmehr, R. A shared resource between declarative memory and motor memory. J. Neurosci. 30, 14817–14823 (2010).
Brown, R. M. & Robertson, E. M. Inducing motor skill improvements with a declarative task. Nature Neurosci. 10, 148–149 (2007).
Bongard, J., Zykov, V. & Lipson, H. Resilient machines through continuous self-modeling. Science 314, 1118–1121 (2006).
Braun, D. A., Aertsen, A., Wolpert, D. M. & Mehring, C. Learning optimal adaptation strategies in unpredictable motor tasks. J. Neurosci. 29, 6472–6478 (2009).
Braun, D. A., Aertsen, A., Wolpert, D. M. & Mehring, C. Motor task variation induces structural learning. Curr. Biol. 19, 352–357 (2009). This paper shows that after experiencing multiple sensorimotor transformations that conform to a structure (for example, the set of visuomotor rotations), interference between opposite visuomotor rotations is substantially reduced. These results show that subjects can extract the structure across a set of tasks and use this to facilitate learning of new tasks provided that they conform to the learned structure.
Braun, D. A., Mehring, C. & Wolpert, D. M. Structure learning in action. Behav. Brain Res. 206, 157–165 (2010).
Braun, D. A., Waldert, S., Aertsen, A., Wolpert, D. M. & Mehring, C. Structure learning in a sensorimotor association task. PLoS ONE 5, e8973 (2010).
Huang, V. S., Shadmehr, R. & Diedrichsen, J. Active learning: learning a motor skill without a coach. J. Neurophysiol. 100, 879–887 (2008).
Brashers-Krug, T., Shadmehr, R. & Bizzi, E. Consolidation in human motor memory. Nature 382, 252–255 (1996).
Brooks, V., Hilperath, F., Brooks, M., Ross, H. & Freund, H. Learning “what” and “how” in a human motor task. Learn. Mem. 2, 225–242 (1995).
Ahmed, A. A., Wolpert, D. M. & Flanagan, J. R. Flexible representations of dynamics are used in object manipulation. Curr. Biol. 18, 763–768 (2008).
Ingram, J. N., Howard, I. S., Flanagan, J. R. & Wolpert, D. M. Multiple grasp-specific representations of tool dynamics mediate skillful manipulation. Curr. Biol. 20, 618–623 (2010).
Churchland, M. M., Afshar, A. & Shenoy, K. V. A central source of movement variability. Neuron 52, 1085–1096 (2006).
Burstedt, M., Edin, B. & Johansson, R. S. Coordination of fingertip forces during human manipulation can emerge from independent neural networks controlling each engaged digit. Exp. Brain Res. 117, 67–79 (1997).
Reed, K. et al. Haptically linked dyads: are two motor-control systems better than one? Psychol. Sci. 17, 365–366 (2006).
Braun, D. A., Ortega, P. A. & Wolpert, D. M. Nash equilibria in multi-agent motor interactions. PLoS Comput. Biol. 5, e1000468 (2009).
Braun, D. A., Ortega, P. A. & Wolpert, D. M. Motor coordination: when two have to act as one. Exp. Brain Res. 221, 631–641 (2011).
Flanagan, J. R. & Johansson, R. S. Action plans used in action observation. Nature 424, 769–771 (2003).
Rotman, G., Troje, N. F., Johansson, R. S. & Flanagan, J. R. Eye movements when observing predictable and unpredictable actions. J. Neurophysiol. 96, 1358–1369 (2006).
Heyes, C. M. & Foster, C. L. Motor learning by observation: evidence from a serial reaction time task. Q. J. Exp. Physiol. 55, 593–607 (2002).
Wolpert, D. M., Doya, K. & Kawato, M. A unifying computational framework for motor control and social interaction. Phil. Trans. R. Soc. Lond. B 358, 593–602 (2003).
Oztop, E., Wolpert, D. M. & Kawato, M. Mental state inference using visual control parameters. Cogn. Brain Res. 22, 129–151 (2005).
Aglioti, S. M., Cesari, P., Romani, M. & Urgesi, C. Action anticipation and motor resonance in elite basketball players. Nature Neurosci. 11, 1109–1116 (2008).
Brass, M., Schmitt, R. M., Spengler, S. & Gergely, G. investigating action understanding: inferential processes versus action simulation. Curr. Biol. 17, 2117–2121 (2007).
Hesse, M. D., Sparing, R. & Fink, G. R. End or means-the “what” and “how” of observed intentional actions. J. Cogn. Neurosci. 21, 776–790 (2009).
Malfait, N. et al. fMRI activation during observation of others' reach errors. J. Cogn. Neurosci. 22, 1493–1503 (2010).
Burke, C. J., Tobler, P. N., Baddeley, M. & Schultz, W. Neural mechanisms of observational learning. Proc. Natl Acad. Sci. USA 107, 14431–14436 (2010).
White, O. & Diedrichsen, J. Responsibility assignment in redundant systems. Curr. Biol. 20, 1290–1295 (2010). This study shows that when participants control a single cursor that is located at the spatial average of the two hands, the left hand corrects more for cursor errors that are induced by a visuomotor rotation, although the right hand corrects more efficiently during unimanual movements. This indicates that the motor system assigns the error during redundant movement to the more likely source of the error.