The role of the basal ganglia in habit formation

Swanson, L. W. Cerebral hemisphere regulation of motivated behavior. Brain Res. 886, 113–164 (2000). A learned and provocative review of cerebral anatomy focusing on basal ganglia organization.

Wilson, C. J. in The Synaptic Organization of the Brain (ed. Shepherd, G. M.) 329–375 (Oxford Univ. Press, New York, 2004).

Deniau, J. M. & Chevalier, G. Disinhibition as a basic process in the expression of striatal functions. II. The striato-nigral influence on thalamocortical cells of the ventromedial thalamic nucleus. Brain Res. 334, 227–233 (1985).

Albin, R. L., Young, A. B. & Penney, J. B. The functional anatomy of basal ganglia disorders. Trends Neurosci. 12, 366–375 (1989).

Canales, J. J. Stimulant-induced adaptations in neostriatal matrix and striosome systems: transiting from instrumental responding to habitual behavior in drug addiction. Neurobiol. Learn. Mem. 83, 93–103 (2005).

Wickens, J. R. & Koetter, R. in Models of Information Processing in the Basal Ganglia (eds Houk, J. C., Davis, J. L. & Beiser, D. G.) 187–214 (MIT Press, Cambridge, Massachusetts, 1995).

Divac, I., Rosvold, H. E. & Szwarcbart, M. K. Behavioral effects of selective ablation of the caudate nucleus. J. Comp. Physiol. Psychol. 63, 184–190 (1967).

Konorski, J. Integrative Activity of the Brain (University of Chicago Press, Chicago, 1967).

Skinner, B. The Behavior of Organisms (Appleton-Century-Crofts, New York, 1938).

Tolman, E. C. Purposive Behavior in Animals and Man (Macmillan, New York, 1932).

Thorndike, E. L. Animal Intelligence: Experimental Studies (Macmillan, New York, 1911).

Hull, C. Principles of Behavior (Appleton-Century-Crofts, New York, 1943).

Dickinson, A. in Animal Learning and Cognition (ed. Mackintosh, N. J.) 45–79 (Academic, Orlando, 1994).

Colwill, R. M. & Rescorla, R. A. in The Psychology of Learning and Motivation (ed. Bower, G.) 55–104 (Academic, New York, 1986). References 13 and 14 are excellent introductions to the modern study of instrumental learning.

Hammond, L. J. The effect of contingency upon the appetitive conditioning of free-operant behavior. J. Exp. Anal. Behav. 34, 297–304 (1980).

Dickinson, A. & Balleine, B. in Spatial Representation: Problems in Philosophy and Psychology (eds Eilan, N. et al.) 277–293 (Blackwell, Malden, Massachusetts, 1993).

Yin, H. H., Knowlton, B. J. & Balleine, B. W. Inactivation of dorsolateral striatum enhances sensitivity to changes in the action–outcome contingency in instrumental conditioning. Behav. Brain Res. 166, 189–196 (2006).

Davis, J. & Bitterman, M. E. Differential reinforcement of other behavior (DRO): a yoked-control comparison. J. Exp. Anal. Behav. 15, 237–241 (1971).

Holman, E. W. Some conditions for the dissociation of consummatory and instrumental behavior in rats. Learn. Motiv. 6, 358–366 (1975).

Adams, C. D. Variations in the sensitivity of instrumental responding to reinforcer devaluation. Q. J. Exp. Psychol. 33B, 109–122 (1982).

Yin, H. H., Knowlton, B. J. & Balleine, B. W. Lesions of dorsolateral striatum preserve outcome expectancy but disrupt habit formation in instrumental learning. Eur. J. Neurosci. 19, 181–189 (2004).

Colwill, R., Rescorla, R. A. The role of response–reinforcer associations increases throughout extended instrumental training. Anim. Learn. Behav. 16, 105–111 (1988).

Dickinson, A. in Learning, Motivation, and Cognition (eds Bouton, M. E. & Fanselow, M. S.) 345–367 (American Psychological Association, Washington DC, 1997).

Dickinson, A. in Contemporary Learning Theories (eds Klein, S. B. & Mowrer, R. R.) 279–308 (Lawrence Erlbaum Associates, Hillsdale, New Jersey, 1989).

Dickinson, A., Nicholas, D. J. & Adams, C. D. The effect of the instrumental training contingency on susceptibility to reinforcer devaluation. Q. J. Exp. Psychol. B 35, 35–51 (1983).

Miller, R. Meaning and Purpose in the Intact Brain (Oxford Univ. Press, New York, 1981).

Mishkin, M., Malamut, B. & Bachevalier, J. in Neurobiology of Learning and Memory (eds Lynch, G. et al.) 65–77 (Guilford, New York, 1984).

Robbins, T. W., Giardini, V., Jones, G. H., Reading, P. & Sahakian, B. J. Effects of dopamine depletion from the caudate-putamen and nucleus accumbens septi on the acquisition and performance of a conditional discrimination task. Behav. Brain Res. 38, 243–261 (1990).

Packard, M. G. & Knowlton, B. J. Learning and memory functions of the basal ganglia. Annu. Rev. Neurosci. 25, 563–593 (2002).

White, N. M. A functional hypothesis concerning the striatal matrix and patches: mediation of S–R memory and reward. Life Sci. 45, 1943–1957 (1989).

Packard, M. G. Glutamate infused posttraining into the hippocampus or caudate-putamen differentially strengthens place and response learning. Proc. Natl Acad. Sci. USA 96, 12881–12886 (1999).

Packard, M. G. & McGaugh, J. L. Inactivation of hippocampus or caudate nucleus with lidocaine differentially affects expression of place and response learning. Neurobiol. Learn. Mem. 65, 65–72 (1996).

Poldrack, R. A. & Packard, M. G. Competition among multiple memory systems: converging evidence from animal and human brain studies. Neuropsychologia 41, 245–251 (2003).

Bayley, P. J., Frascino, J. C. & Squire, L. R. Robust habit learning in the absence of awareness and independent of the medial temporal lobe. Nature 436, 550–553 (2005).

Knowlton, B. J., Mangels, J. A. & Squire, L. R. A neostriatal habit learning system in humans. Science 273, 1399–1402 (1996).

Moody, T. D., Bookheimer, S. Y., Vanek, Z. & Knowlton, B. J. An implicit learning task activates medial temporal lobe in patients with Parkinson's disease. Behav. Neurosci. 118, 438–442 (2004).

Kawagoe, R., Takikawa, Y. & Hikosaka, O. Expectation of reward modulates cognitive signals in the basal ganglia. Nature Neurosci. 1, 411–416 (1998).

Lauwereyns, J. et al. Feature-based anticipation of cues that predict reward in monkey caudate nucleus. Neuron 33, 463–473 (2002).

Lauwereyns, J., Watanabe, K., Coe, B. & Hikosaka, O. A neural correlate of response bias in monkey caudate nucleus. Nature 418, 413–417 (2002).

Pasupathy, A. & Miller, E. K. Different time courses of learning-related activity in the prefrontal cortex and striatum. Nature 433, 873–876 (2005).

Joel, D. & Weiner, I. The connections of the dopaminergic system with the striatum in rats and primates: an analysis with respect to the functional and compartmental organization of the striatum. Neuroscience 96, 451–474 (2000).

West, M. O. et al. A region in the dorsolateral striatum of the rat exhibiting single-unit correlations with specific locomotor limb movements. J. Neurophysiol. 64, 1233–1246 (1990).

Partridge, J. G., Tang, K. C. & Lovinger, D. M. Regional and postnatal heterogeneity of activity-dependent long-term changes in synaptic efficacy in the dorsal striatum. J. Neurophysiol. 84, 1422–1429 (2000). The first study to demonstrate regional variations in the types and mechanisms of striatal synaptic plasticity.

Whishaw, I. Q., Mittleman, G., Bunch, S. T. & Dunnett, S. B. Impairments in the acquisition, retention and selection of spatial navigation strategies after medial caudate-putamen lesions in rats. Behav. Brain Res. 24, 125–138 (1987).

Devan, B. D., McDonald, R. J. & White, N. M. Effects of medial and lateral caudate-putamen lesions on place- and cue-guided behaviors in the water maze: relation to thigmotaxis. Behav. Brain Res. 100, 5–14 (1999).

Devan, B. D. & White, N. M. Parallel information processing in the dorsal striatum: relation to hippocampal function. J. Neurosci. 19, 2789–2798 (1999).

Yin, H. H., Knowlton, B. J. & Balleine, B. W. Blockade of NMDA receptors in the dorsomedial striatum prevents action–outcome learning in instrumental conditioning. Eur. J. Neurosci. 22, 505–512 (2005).

Yin, H. H., Ostlund, S. B., Knowlton, B. J. & Balleine, B. W. The role of the dorsomedial striatum in instrumental conditioning. Eur. J. Neurosci. 22, 513–523 (2005).

Yin, H. H. & Knowlton, B. J. Contributions of striatal subregions to place and response learning. Learn. Mem. 11, 459–463 (2004). References 47–49 present a series of studies that established for the first time a dissociation between S–R learning in the DLS and A–O learning in the pDMS.

Ragozzino, M. E. Acetylcholine actions in the dorsomedial striatum support the flexible shifting of response patterns. Neurobiol. Learn. Mem. 80, 257–267 (2003).

Ragozzino, M. E., Jih, J. & Tzavos, A. Involvement of the dorsomedial striatum in behavioral flexibility: role of muscarinic cholinergic receptors. Brain Res. 953, 205–214 (2002).

Ragozzino, K. E., Leutgeb, S. & Mizumori, S. J. Dorsal striatal head direction and hippocampal place representations during spatial navigation. Exp. Brain Res. 139, 372–376 (2001).

Mulder, A. B., Tabuchi, E. & Wiener, S. I. Neurons in hippocampal afferent zones of rat striatum parse routes into multi-pace segments during maze navigation. Eur. J. Neurosci. 19, 1923–1932 (2004).

Delgado, M. R., Locke, H. M., Stenger, V. A. & Fiez, J. A. Dorsal striatum responses to reward and punishment: effects of valence and magnitude manipulations. Cogn. Affect. Behav. Neurosci. 3, 27–38 (2003).

Delgado, M. R., Stenger, V. A. & Fiez, J. A. Motivation-dependent responses in the human caudate nucleus. Cereb. Cortex 14, 1022–1030 (2004).

Zink, C. F., Pagnoni, G., Martin-Skurski, M. E., Chappelow, J. C. & Berns, G. S. Human striatal responses to monetary reward depend on saliency. Neuron 42, 509–517 (2004).

Tricomi, E. M., Delgado, M. R. & Fiez, J. A. Modulation of caudate activity by action contingency. Neuron 41, 281–292 (2004). An interesting human imaging study that provided strong evidence for the role of the caudate in encoding A–O contingencies.

Williams, Z. M. & Eskandar, E. N. Selective enhancement of associative learning by microstimulation of the anterior caudate. Nature Neurosci. 9, 562–568 (2006).

Dickinson, A., Balleine, B., Watt, A. & Gonzalez, F. Motivational control after extended instrumental training. Anim. Learn. Behav. 23, 197–206 (1995).

Balleine, B. W. & Dickinson, A. Goal-directed instrumental action: contingency and incentive learning and their cortical substrates. Neuropharmacology 37, 407–419 (1998).

Corbit, L. H. & Balleine, B. W. The role of prelimbic cortex in instrumental conditioning. Behav. Brain Res. 146, 145–157 (2003).

Leon, M. I. & Shadlen, M. N. Effect of expected reward magnitude on the response of neurons in the dorsolateral prefrontal cortex of the macaque. Neuron 24, 415–425 (1999).

Tsujimoto, S. & Sawaguchi, T. Properties of delay-period neuronal activity in the primate prefrontal cortex during memory- and sensory-guided saccade tasks. Eur. J. Neurosci. 19, 447–457 (2004).

Tsujimoto, S. & Sawaguchi, T. Neuronal representation of response–outcome in the primate prefrontal cortex. Cereb. Cortex 14, 47–55 (2004).

Tsujimoto, S. & Sawaguchi, T. Working memory of action: a comparative study of ability to selecting response based on previous action in New World monkeys (Saimiri sciureus and Callithrix jacchus). Behav. Processes 58, 149–155 (2002).

Corbit, L. H., Muir, J. L. & Balleine, B. W. Lesions of mediodorsal thalamus and anterior thalamic nuclei produce dissociable effects on instrumental conditioning in rats. Eur. J. Neurosci. 18, 1286–1294 (2003).

Ostlund, S. B. & Balleine, B. W. Lesions of medial prefrontal cortex disrupt the acquisition but not the expression of goal-directed learning. J. Neurosci. 25, 7763–7770 (2005).

Corbit, L. H., Ostlund, S. B. & Balleine, B. W. Sensitivity to instrumental contingency degradation is mediated by the entorhinal cortex and its efferents via the dorsal hippocampus. J. Neurosci. 22, 10976–10984 (2002).

Packard, M. G. & McGaugh, J. L. Double dissociation of fornix and caudate nucleus lesions on acquisition of two water maze tasks: further evidence for multiple memory systems. Behav. Neurosci. 106, 439–446 (1992).

Alexander, G. E., DeLong, M. R. & Strick, P. L. Parallel organization of functionally segregated circuits linking basal ganglia and cortex. Annu. Rev. Neurosci. 9, 357–381 (1986).

Reep, R. L., Cheatwood, J. L. & Corwin, J. V. The associative striatum: organization of cortical projections to the dorsocentral striatum in rats. J. Comp. Neurol. 467, 271–292 (2003).

Dalley, J. W., Cardinal, R. N. & Robbins, T. W. Prefrontal executive and cognitive functions in rodents: neural and neurochemical substrates. Neurosci. Biobehav. Rev. 28, 771–784 (2004).

Divac, I., Markowitsch, H. J. & Pritzel, M. Behavioral and anatomical consequences of small intrastriatal injections of kainic acid in the rat. Brain Res. 151, 523–532 (1978).

Levy, R., Friedman, H. R., Davachi, L. & Goldman-Rakic, P. S. Differential activation of the caudate nucleus in primates performing spatial and nonspatial working memory tasks. J. Neurosci. 17, 3870–3882 (1997).

Hassani, O. K., Cromwell, H. C. & Schultz, W. Influence of expectation of different rewards on behavior-related neuronal activity in the striatum. J. Neurophysiol. 85, 2477–2489 (2001).

Kimura, M., Aosaki, T. & Ishida, A. Neurophysiological aspects of the differential roles of the putamen and caudate nucleus in voluntary movement. Adv. Neurol. 60, 62–70 (1993).

Kanazawa, I., Murata, M. & Kimura, M. Roles of dopamine and its receptors in generation of choreic movements. Adv. Neurol. 60, 107–112 (1993).

Joel, D. & Weiner, I. The organization of the basal ganglia-thalamocortical circuits: open interconnected rather than closed segregated. Neuroscience 63, 363–379. (1994). An important review in a series by the same authors arguing for interactions between cortico-basal ganglia networks.

Joel, D. & Weiner, I. The connections of the primate subthalamic nucleus: indirect pathways and the open-interconnected scheme of basal ganglia-thalamocortical circuitry. Brain Res. Brain Res. Rev. 23, 62–78 (1997).

Haber, S. N., Fudge, J. L. & McFarland, N. R. Striatonigrostriatal pathways in primates form an ascending spiral from the shell to the dorsolateral striatum. J. Neurosci. 20, 2369–2382 (2000).

Redgrave, P., Prescott, T. J. & Gurney, K. The basal ganglia: a vertebrate solution to the selection problem? Neuroscience 89, 1009–1023 (1999).

Haber, S. N. The primate basal ganglia: parallel and integrative networks. J. Chem. Neuroanat. 26, 317–330 (2003).

Middleton, F. A. & Strick, P. L. Basal ganglia and cerebellar loops: motor and cognitive circuits. Brain Res. Brain Res. Rev. 31, 236–250 (2000).

Rand, M. K. et al. Characteristics of sequential movements during early learning period in monkeys. Exp. Brain Res. 131, 293–304 (2000).

McGeorge, A. J. & Faull, R. L. The organization of the projection from the cerebral cortex to the striatum in the rat. Neuroscience 29, 503–537 (1989).

Jueptner, M., Frith, C. D., Brooks, D. J., Frackowiak, R. S. & Passingham, R. E. Anatomy of motor learning. II. Subcortical structures and learning by trial and error. J. Neurophysiol. 77, 1325–1337 (1997).

Jueptner, M. et al. Anatomy of motor learning. I. Frontal cortex and attention to action. J. Neurophysiol. 77, 1313–1324 (1997).

Poldrack, R. A. et al. The neural correlates of motor skill automaticity. J. Neurosci. 25, 5356–5364 (2005). References 85–88 show shifts in activation patterns of cortico-basal ganglia networks in the course of skill learning.

Daw, N. D., Niv, Y. & Dayan, P. Uncertainty-based competition between prefrontal and dorsolateral striatal systems for behavioral control. Nature Neurosci. 8, 1704–1711 (2005).

Everitt, B. J. & Wolf, M. E. Psychomotor stimulant addiction: a neural systems perspective. J. Neurosci. 22, 3312–3320 (2002).

Altman, J. et al. The biological, social and clinical bases of drug addiction: commentary and debate. Psychopharmacology (Berl.) 125, 285–345 (1996).

Porrino, L. J., Lyons, D., Smith, H. R., Daunais, J. B. & Nader, M. A. Cocaine self-administration produces a progressive involvement of limbic, association, and sensorimotor striatal domains. J. Neurosci. 24, 3554–3562 (2004).

Williams, D. R. & Williams, H. Automaintenance in the pigeon: sustained pecking despite contingent non-reinforcement. J. Exp. Anal. Behav. 12, 511–520 (1969).

Robinson, T. E. & Berridge, K. C. Addiction. Annu. Rev. Psychol. 54, 25–53 (2003).

Berridge, K. C. & Robinson, T. E. What is the role of dopamine in reward: hedonic impact, reward learning, or incentive salience? Brain Res. Brain Res. Rev. 28, 309–369 (1998).

Tiffany, S. T. A cognitive model of drug urges and drug-use behavior: role of automatic and nonautomatic processes. Psychol. Rev. 97, 147–168 (1990).

Rescorla, R. A. & Solomon, R. L. Two-process learning theory: relationships between Pavlovian conditioning and instrumental learning. Psychol. Rev. 74, 151–182 (1967).

Corbit, L. H. & Balleine, B. W. Double dissociation of basolateral and central amygdala lesions on the general and outcome-specific forms of Pavlovian–instrumental transfer. J. Neurosci. 25, 962–970 (2005).

Holland, P. C. Relations between Pavlovian–instrumental transfer and reinforcer devaluation. J. Exp. Psychol. Anim. Behav. Process. 30, 104–117 (2004).

Corbit, L. H., Muir, J. L. & Balleine, B. W. The role of the nucleus accumbens in instrumental conditioning: evidence of a functional dissociation between accumbens core and shell. J. Neurosci. 21, 3251–3260 (2001).

Canales, J. J. & Graybiel, A. M. A measure of striatal function predicts motor stereotypy. Nature Neurosci. 3, 377–383 (2000).

Rhodes, S. E. & Killcross, S. Lesions of rat infralimbic cortex enhance recovery and reinstatement of an appetitive Pavlovian response. Learn. Mem. 11, 611–616 (2004).

Coutureau, E. & Killcross, S. Inactivation of the infralimbic prefrontal cortex reinstates goal-directed responding in overtrained rats. Behav. Brain Res. 146, 167–174 (2003).

Killcross, S. & Coutureau, E. Coordination of actions and habits in the medial prefrontal cortex of rats. Cereb. Cortex 13, 400–408 (2003).

Everitt, B. J. & Robbins, T. W. Neural systems of reinforcement for drug addiction: from actions to habits to compulsion. Nature Neurosci. 8, 1481–1489 (2005).

Baum, W. M. The correlation-based law of effect. J. Exp. Anal. Behav. 20, 137–153 (1973).

Dickinson, A. Actions and habits: the development of behavioural autonomy. Phil. Trans. R. Soc. Lond. B 308, 67–78 (1985).

Kerr, J. N. & Wickens, J. R. Dopamine D-1/D-5 receptor activation is required for long-term potentiation in the rat neostriatum in vitro. J. Neurophysiol. 85, 117–124 (2001).

Reynolds, J. N., Hyland, B. I. & Wickens, J. R. A cellular mechanism of reward-related learning. Nature 413, 67–70 (2001).

Gerdeman, G. L., Partridge, J. G., Lupica, C. R. & Lovinger, D. M. It could be habit forming: drugs of abuse and striatal synaptic plasticity. Trends Neurosci. 26, 184–192 (2003).

Gerdeman, G. L., Ronesi, J. & Lovinger, D. M. Postsynaptic endocannabinoid release is critical to long-term depression in the striatum. Nature Neurosci. 5, 446–451 (2002).

Packard, M. G. & White, N. M. Dissociation of hippocampus and caudate nucleus memory systems by posttraining intracerebral injection of dopamine agonists. Behav. Neurosci. 105, 295–306 (1991).

Sage, J. R. & Knowlton, B. J. Effects of US devaluation on win-stay and win-shift radial maze performance in rats. Behav. Neurosci. 114, 295–306 (2000).

Packard, M. G., Hirsh, R. & White, N. M. Differential effects of fornix and caudate nucleus lesions on two radial maze tasks: evidence for multiple memory systems. J. Neurosci. 9, 1465–1472 (1989).