Neural systems of reinforcement for drug addiction: from actions to habits to compulsion

Haber, S.N., Fudge, J.L. & McFarland, N.R. Striatonigral pathways in primates form an ascending spiral from the shell to the dorsolateral striatum. J. Neurosci. 20, 2369–2382 (2000).

Dickinson, A. & Balleine, B. Motivational control of goal-directed action. Anim. Learn. Behav. 22, 1–18 (1994).

White, N.M. & McDonald, R.J. Multiple parallel memory systems in the brain of the rat. Neurobiol. Learn. Mem. 77, 125–184 (2002).

O'Brien, C.P. & McLellan, A.T. Myths about the treatment of addiction. Lancet 347, 237–240 (1996).

Leshner, A.I. Addiction is a brain disease, and it matters. Science 278, 45–47 (1997).

Schultz, W. & Dickinson, A. Neuronal coding of prediction errors. Annu. Rev. Neurosci. 23, 473–500 (2000).

Fiorillo, C.D., Tobler, P.N. & Schultz, W. Discrete coding of reward probability and uncertainty by dopamine neurons. Science 299, 1898–1902 (2003).

Robbins, T.W. & Everitt, B.J. Functions of dopamine in the dorsal and ventral striatum. Seminars in the Neurosciences 4, 119–128 (1992).

Ito, R., Dalley, J.W., Howes, S.R., Robbins, T.W. & Everitt, B.J. Dissociation in conditioned dopamine release in the nucleus accumbens core and shell in response to cocaine cues and during cocaine-seeking behavior in rats. J. Neurosci. 20, 7489–7495 (2000).

Parkinson, J.A., Olmstead, M.C., Burns, L.H., Robbins, T.W. & Everitt, B.J. Dissociation in effects of lesions of the nucleus accumbens core and shell on appetitive Pavlovian approach behavior and the potentiation of conditioned reinforcement and locomotor activity by D-amphetamine. J. Neurosci. 19, 2401–2411 (1999).

Di Ciano, P., Cardinal, R.N., Cowell, R.A., Little, S.J. & Everitt, B.J. Differential involvement of NMDA, AMPA/kainate, and dopamine receptors in the nucleus accumbens core in the acquisition and performance of Pavlovian approach behavior. J. Neurosci. 21, 9471–9477 (2001).

Dalley, J.W. et al. Time-limited modulation of appetitive Pavlovian memory by D1 and NMDA receptors in the nucleus accumbens. Proc. Natl. Acad. Sci. USA 102, 6189–6194 (2005).

Hall, J., Parkinson, J.A., Connor, T.M., Dickinson, A. & Everitt, B.J. Involvement of the central nucleus of the amygdala and nucleus accumbens core in mediating Pavlovian influences on instrumental behaviour. Eur. J. Neurosci. 13, 1984–1992 (2001).

Wyvell, C.L. & Berridge, K.C. Intra-accumbens amphetamine increases the conditioned incentive salience of sucrose reward: enhancement of reward “wanting” without enhanced “liking” or response reinforcement. J. Neurosci. 20, 8122–8130 (2000).

Robinson, T.E. & Berridge, K.C. The neural basis of drug craving: an incentive-sensitization theory of addiction. Brain Res. Brain Res. Rev. 18, 247–291 (1993).

Kearns, D.N. & Weiss, S.J. Sign-tracking (autoshaping) in rats: A comparison of cocaine and food as unconditioned stimuli. Learn. Behav. 32, 463–476 (2004).

Grimm, J.W., Kruzich, P.J. & See, R.E. Contingent access to stimuli associated with cocaine self- administration is required for reinstatement of drug-seeking behavior. Psychobiology 28, 383–386 (2000).

Di Ciano, P. & Everitt, B.J. Differential control over drug-seeking behavior by drug-associated conditioned reinforcers and discriminative stimuli predictive of drug availability. Behav. Neurosci. 117, 952–960 (2003).

Cardinal, R.N. & Everitt, B.J. Neural and psychological mechanisms underlying appetitive learning: links to drug addiction. Curr. Opin. Neurobiol. 14, 156–162 (2004).

Everitt, B.J., Dickinson, A. & Robbins, T.W. The neuropsychological basis of addictive behaviour. Brain Res. Brain Res. Rev. 36, 129–138 (2001).

Cardinal, R.N., Robbins, T.W. & Everitt, B.J. The effects of d-amphetamine, chlordiazepoxide, alpha- flupenthixol and behavioural manipulations on choice of signalled and unsignalled delayed reinforcement in rats. Psychopharmacology (Berl.) 152, 362–375 (2000).

Taylor, J.R. & Robbins, T.W. Enhanced behavioural control by conditioned reinforcers following microinjections of d-amphetamine into the nucleus accumbens. Psychopharmacology (Berl.) 84, 405–412 (1984).

Di Ciano, P. & Everitt, B.J. Conditioned reinforcing properties of stimuli paired with self-administered cocaine, heroin or sucrose: implications for the persistence of addictive behaviour. Neuropharmacology 47 (Suppl.) 202–213 (2004).

Everitt, B.J. & Robbins, T.W. Second-order schedules of drug reinforcement in rats and monkeys: measurement of reinforcing efficacy and drug-seeking behaviour. Psychopharmacology (Berl.) 153, 17–30 (2000).

Robbins, T.W. & Everitt, B.J. Drug addiction: bad habits add up. Nature 398, 567–570 (1999).

Groenewegen, H.J., Wright, C.I. & Beijer, A.V.J. The nucleus accumbens: Gateway for limbic structures to reach the motor system? Prog. Brain Res. 107, 485–511 (1996).

Kelley, A.E. & Berridge, K.C. The neuroscience of natural rewards: Relevance to addictive drugs. J. Neurosci. 22, 3306–3311 (2002).

Ikemoto, S., Qin, M. & Liu, Z.H. The functional divide for primary reinforcement of D-amphetamine lies between the medial and lateral ventral striatum: Is the division of the accumbens core, shell, and olfactory tubercle valid? J. Neurosci. 25, 5061–5065 (2005).

Fenu, S., Bassareo, V. & Di Chiara, G. A role for dopamine D1 receptors of the nucleus accumbens shell in conditioned taste aversion learning. J. Neurosci. 21, 6897–6904 (2001).

Yin, H.H., Ostlund, S.B., Knowlton, B.J. & Balleine, B.W. The role of the dorsomedial striatum in instrumental conditioning. Eur. J. Neurosci. 22, 513–523 (2005).

Ostlund, S.B. & Balleine, B.W. Lesions of medial prefrontal cortex disrupt the acquisition but not the expression of goal-directed learning. J. Neurosci. 25, 7763–7770 (2005).

Yin, H.H., Knowlton, B.J. & Balleine, B.W. Lesions of dorsolateral striatum preserve outcome expectancy but disrupt habit formation in instrumental learning. Eur. J. Neurosci. 19, 181–189 (2004).

Wolffgramm, J. & Heyne, A. From controlled drug intake to loss of control - the irreversible development of drug-addiction in the rat. Behav. Brain Res. 70, 77–94 (1995).

Deroche-Gamonet, V., Belin, D. & Piazza, P.V. Evidence for addiction-like behavior in the rat. Science 305, 1014–1017 (2004).

Vanderschuren, L.J. & Everitt, B.J. Drug seeking becomes compulsive after prolonged cocaine self-administration. Science 305, 1017–1019 (2004).

Ito, R., Dalley, J.W., Robbins, T.W. & Everitt, B.J. Dopamine release in the dorsal striatum during cocaine-seeking behavior under the control of a drug-associated cue. J. Neurosci. 22, 6247–6253 (2002).

Jakes, I. Theoretical Approaches to Obsessive-Compulsive Disorder (Cambridge Univ. Press, Cambridge, 1996).

Barrett, J.E., Katz, J.L. & Glowa, J.R. Effects of D-amphetamine on responding of squirrel-monkeys maintained under 2nd-order schedules of food presentation, electric-shock presentation or stimulus-shock termination. J. Pharmacol. Exp. Ther. 218, 692–700 (1981).

Dickinson, A., Nicholas, D.J. & Adams, C.D. The effect of instrumental training contingency on susceptibility to reinforcer devaluation. Q. J. Exp. Psychol. 35B, 35–51 (1983).

Faure, A., Haberland, U., Conde, F. & El Massioui, N. Lesion to the nigrostriatal dopamine system disrupts stimulus-response habit formation. J. Neurosci. 25, 2771–2780 (2005).

Dickinson, A., Wood, N. & Smith, J.W. Alcohol seeking by rats: Action or habit? Q. J. Exp. Psychol. B 55, 331–348 (2002).

Miles, F.J., Everitt, B.J. & Dickinson, A. Oral cocaine seeking by rats: action or habit? Behav. Neurosci. 117, 927–938 (2003).

Di Ciano, P. & Everitt, B.J. Direct interactions between the basolateral amygdala and nucleus accumbens core underlie cocaine-seeking behavior by rats. J. Neurosci. 24, 7167–7173 (2004).

Vanderschuren, L.M.J., Di Ciano, P. & Everitt, B.J. Involvement of the dorsal striatum in cue-controlled cocaine seeking. J. Neurosci. 25, 8665–8770 (2005).

Goldstein, R.Z. & Volkow, N.D. Drug addiction and its underlying neurobiological basis: Neuroimaging evidence for the involvement of the frontal cortex. Am. J. Psychiatry 159, 1642–1652 (2002).

Letchworth, S.R., Nader, M.A., Smith, H.R., Friedman, D.P. & Porrino, L.J. Progression of changes in dopamine transporter binding site density as a result of cocaine self-administration in rhesus monkeys. J. Neurosci. 21, 2799–2807 (2001).

Porrino, L.J., Lyons, D., Smith, H.R., Daunais, J.B. & Nader, M.A. Cocaine self-administration produces a progressive involvement of limbic, association, and sensorimotor striatal domains. J. Neurosci. 24, 3554–3562 (2004).

Nader, M.A. et al. Effects of cocaine self-administration on striatal dopamine systems in rhesus monkeys: Initial and chronic exposure. Neuropsychopharmacology 27, 35–46 (2002).

O'Doherty, J. et al. Dissociable roles of ventral and dorsal striatum in instrumental conditioning. Science 304, 452–454 (2004).

Whitelaw, R.B., Markou, A., Robbins, T.W. & Everitt, B.J. Excitotoxic lesions of the basolateral amygdala impair the acquisition of cocaine-seeking behaviour under a second-order schedule of reinforcement. Psychopharmacology (Berl.) 127, 213–224 (1996).

Alderson, H.L., Robbins, T.W. & Everitt, B.J. The effects of excitotoxic lesions of the basolateral amygdala on the acquisition of heroin-seeking behaviour in rats. Psychopharmacology (Berl.) 153, 111–119 (2000).

Hutcheson, D.M., Parkinson, J.A., Robbins, T.W. & Everitt, B.J. The effects of nucleus accumbens core and shell lesions on intravenous heroin self-administration and the acquisition of drug-seeking behaviour under a second-order schedule of heroin reinforcement. Psychopharmacology (Berl.) 153, 464–472 (2001).

Ito, R., Robbins, T.W. & Everitt, B.J. Differential control over cocaine-seeking behavior by nucleus accumbens core and shell. Nat. Neurosci. 7, 389–397 (2004).

Winstanley, C.A., Theobald, D.E.H., Cardinal, R.N. & Robbins, T.W. Contrasting roles of basolateral amygdala and orbitofrontal cortex in impulsive choice. J. Neurosci. 24, 4718–4722 (2004).

Cardinal, R.N. & Cheung, T.H. Nucleus accumbens core lesions retard instrumental learning and performance with delayed reinforcement in the rat. BMC Neurosci. 6, 9 (2005).

Hutcheson, D.M. & Everitt, B.J. The effects of selective orbitofrontal cortex lesions on the acquisition and performance of cue-controlled cocaine seeking in rats. Ann. NY Acad. Sci. 1003, 410–411 (2003).

Pears, A., Parkinson, J.A., Hopewell, L., Everitt, B.J. & Roberts, A.C. Lesions of the orbitofrontal but not medial prefrontal cortex disrupt conditioned reinforcement in primates. J. Neurosci. 23, 11189–11201 (2003).

Schoenbaum, G., Setlow, B., Saddoris, M.P. & Gallagher, M. Encoding predicted outcome and acquired value in orbitofrontal cortex during cue sampling depends upon input from basolateral amygdala. Neuron 39, 855–867 (2003).

Kalivas, P.W. & McFarland, K. Brain circuitry and the reinstatement of cocaine-seeking behavior. Psychopharmacology (Berl.) 168, 44–56 (2003).

Shaham, Y., Shalev, U., Lu, L., de Wit, H. & Stewart, J. The reinstatement model of drug relapse: history, methodology and major findings. Psychopharmacology (Berl.) 168, 3–20 (2003).

Meil, W.M. & See, R.E. Lesions of the basolateral amygdala abolish the ability of drug associated cues to reinstate responding during withdrawal from self-administered cocaine. Behav. Brain Res. 87, 139–148 (1997).

Fuchs, R.A., Evans, K.A., Parker, M.P. & See, R.E. Differential involvement of orbitofrontal cortex subregions in conditioned cue-induced and cocaine-primed reinstatement of cocaine seeking in rats. J. Neurosci. 24, 6600–6610 (2004).

McFarland, K. & Kalivas, P.W. The circuitry mediating cocaine-induced reinstatement of drug seeking behavior. J. Neurosci. 21, 8655–8663 (2001).

McFarland, K., Lapish, C.C. & Kalivas, P.W. Prefrontal glutamate release into the core of the nucleus accumbens mediates cocaine-induced reinstatement of drug seeking behavior. J. Neurosci. 23, 3531–3537 (2003).

See, R.E., Kruzich, P.J. & Grimm, J.W. Dopamine, but not glutamate, receptor blockade in the basolateral amygdala attenuates conditioned reward in a rat model of relapse to cocaine-seeking behavior. Psychopharmacology (Berl.) 154, 301–310 (2001).

Fuchs, R.A. et al. The role of the dorsomedial prefrontal cortex, basolateral amygdala, and dorsal hippocampus in contextual reinstatement of cocaine seeking in rats. Neuropsychopharmacology 30, 296–309 (2005).

Vorel, S.R., Liu, X., Hayes, R.J., Spector, J.A. & Gardner, E.L. Relapse to cocaine-seeking after hippocampal theta burst stimulation. Science 292, 1175–1178 (2001).

Selden, N.R.W., Everitt, B.J., Jarrard, L.E. & Robbins, T.W. Complementary roles for the amygdala and hippocampus in aversive conditioning to explicit and contextual cues. Neuroscience 42, 335–350 (1991).

Pennartz, C.M.A., Groenewegan, H.J. & Lopas da Silva, F.H. The nucleus accumbens as a complex of functionally distinct neuronal ensembles: an integration of behavioural, electrophysiological and anatomical data. Prog. Neurobiol. 42, 719–761 (1994).

Goto, Y. & Grace, A.A. Dopaminergic modulation of limbic and cortical drive of nucleus accumbens in goal-directed behavior. Nat. Neurosci. 8, 805–812 (2005).

Grace, A.A., Floresco, S.B., West, A.R. & Goto, Y. Dissociation of tonic and phasic dopamine neuron activity by afferent pathway activation: Relationship to patterns of dopamine release. Int. J. Neuropsychopharmacol. 7, S15–S15 (2004).

O'Donnell, P. Dopamine gating of forebrain neural ensembles. Eur. J. Neurosci. 17, 429–435 (2003).

Floresco, S.B., Blaha, C.D., Yang, C.R. & Phillips, A.G. Modulation of hippocampal and amygdalar-evoked activity of nucleus accumbens neurons by dopamine: cellular mechanisms of input selection. J. Neurosci. 21, 2851–2860 (2001).

Caine, S.B., Humby, T., Robbins, T.W. & Everitt, B.J. Behavioral effects of psychomotor stimulants in rats with dorsal or ventral subiculum lesions: Locomotion, cocaine self- administration, and prepulse inhibition of startle. Behav. Neurosci. 115, 880–894 (2001).

Burns, L.H., Robbins, T.W. & Everitt, B.J. Differential effects of excitotoxic lesions of the basolateral amygdala, ventral subiculum and medial prefrontal cortex on responding with conditioned reinforcement and locomotor activity potentiated by intra-accumbens infusions of D-amphetamine. Behav. Brain Res. 55, 167–183 (1993).

Weissenborn, R., Robbins, T.W. & Everitt, B.J. Effects of medial prefrontal or anterior cingulate cortex lesions on responding for cocaine under fixed-ratio and second-order schedules of reinforcement in rats. Psychopharmacology (Berl.) 134, 242–257 (1997).

Dalley, J.W., Cardinal, R.N. & Robbins, T.W. Prefrontal executive and cognitive functions in rodents: neural and neurochemical substrates. Neurosci. Biobehav. Rev. 28, 771–784 (2004).

Rogers, R.D. & Robbins, T.W. Investigating the neurocognitive deficits associated with chronic drug misuse. Curr. Opin. Neurobiol. 11, 250–257 (2001).

Bolla, K.I. et al. Orbitofrontal cortex dysfunction in abstinent cocaine abusers performing a decision-making task. Neuroimage 19, 1085–1094 (2003).

Hester, R. & Garavan, H. Executive dysfunction in cocaine addiction: evidence for discordant frontal, cingulate, and cerebellar activity. J. Neurosci. 24, 11017–11022 (2004).

Volkow, N.D., Fowler, J.S. & Wang, G.J. The addicted human brain viewed in the light of imaging studies: brain circuits and treatment strategies. Neuropharmacology 47, 3–13 (2004).

Crombag, H.S., Gorny, G., Li, Y.L., Kolb, B. & Robinson, T.E. Opposite effects of amphetamine self-administration experience on dendritic spines in the medial and orbital prefrontal cortex. Cereb. Cortex 15, 341–348 (2005).

Killcross, S. & Coutureau, E. Coordination of actions and habits in the medial prefrontal cortex of rats. Cereb. Cortex 13, 400–408 (2003).

Corbit, L.H. & Balleine, B.W. The role of prelimbic cortex in instrumental conditioning. Behav. Brain Res. 146, 145–157 (2003).

Robinson, T.E. & Berridge, K.C. Addiction. Annu. Rev. Psychol. 54, 25–53 (2003).

Taylor, J.R. & Horger, B.A. Enhanced responding for conditioned reward produced by intra-accumbens amphetamine is potentiated after cocaine sensitization. Psychopharmacology (Berl.) 142, 31–40 (1999).

Vezina, P. Sensitization of midbrain dopamine neuron reactivity and the self-administration of psychomotor stimulant drugs. Neurosci. Biobehav. Rev. 27, 827–839 (2004).

Koob, G.F. & Le Moal, M. Drug addiction, dysregulation of reward, and allostasis. Neuropsychopharmacology 24, 97–129 (2001).

Koob, G.F. et al. Neurobiological mechanisms in the transition from drug use to drug dependence. Neurosci. Biobehav. Rev. 27, 739–749 (2004).

The Frontal Lobes and Neuropsychiatric Illness (Salloway, S.P., Malloy, P.E. & Duffy, J.D., eds.) (American Psychiatric Press, Washington, D.C., 2001).

Altman, J. et al. The biological, social and clinical bases of drug addiction: Commentary and debate. Psychopharmacology (Berl.) 125, 285–345 (1996).

Everitt, B.J. & Wolf, M.E. Psychomotor stimulant addiction: A neural systems perspective. J. Neurosci. 22, 3312–3320 (2002).

Stewart, J., de Wit, H. & Eikelboom, R. The role of unconditioned and conditioned drug effects in the self administration of opiates and stimulants. Psychol. Rev. 91, 251–268 (1984).

Di Chiara, G. A motivational learning hypothesis of the role of mesolimbic dopamine in compulsive drug use. J. Psychopharmacol. 12, 54–67 (1998).

Wise, R.A. Dopamine, learning and motivation. Nat. Rev. Neurosci. 5, 483–494 (2004).

Berridge, K.C. Pleasures of the brain. Brain Cogn. 52, 106–128 (2003).

Kadohisa, M., Rolls, E.T. & Verhagen, J.V. Orbitofrontal cortex: Neuronal representation of oral temperature and capsaicin in addition to taste and texture. Neuroscience 127, 207–221 (2004).

Critchley, H.D., Wiens, S., Rotshtein, P., Ohman, A. & Dolan, R.J. Neural systems supporting interoceptive awareness. Nat. Neurosci. 7, 189–195 (2004).

Stewart, J. & de Wit, H. in Methods of Assessing the Reinforcing Properties of Abused Drugs (ed. Bozarth, M.A.) 211–227 (Springer-Verlag, New York, 1987).

Lu, L., Grimm, J.W., Hope, B.T. & Shaham, Y. Incubation of cocaine craving after withdrawal: a review of preclinical data. Neuropharmacology 47, 214–226 (2004).