Exploration versus exploitation in space, mind, and society

Cohen, 2007, Should I stay or should I go? How the human brain manages the trade-off between exploitation and exploration, Philos. Trans. R. Soc. Lond. B: Biol. Sci., 362, 933, 10.1098/rstb.2007.2098 March, 1991, Exploration and exploitation in organizational learning, Organ. Sci., 2, 71, 10.1287/orsc.2.1.71 Treisman, 1980, A feature-integration theory of attention, Cogn. Psychol., 12, 97, 10.1016/0010-0285(80)90005-5 Wolfe, 1989, Guided search: an alternative to the feature integration model for visual search, J. Exp. Psychol. Hum. Percept. Perform., 15, 419, 10.1037/0096-1523.15.3.419 Benhamou, 2007, How many animals really do the Lévy walk?, Ecology, 88, 1962, 10.1890/06-1769.1 Hills, 2012, Optimal foraging in semantic memory, Psychol. Rev., 119, 431, 10.1037/a0027373 Korf, 1985, Depth-first iterative-deepening: an optimal admissible tree search, Artif. Intell., 27, 97, 10.1016/0004-3702(85)90084-0 Sutton, 1998 Hills, 2006, Animal foraging and the evolution of goal-directed cognition, Cogn. Sci., 30, 3, 10.1207/s15516709cog0000_50 Stephens, 2007 Bonabeau, 1999 Deneubourg, 1986, Random behaviour, amplification processes and number of participants: how they contribute to the foraging properties of ants, Physica D, 22, 176, 10.1016/0167-2789(86)90239-3 King, 2009, Is the true “wisdom of the crowd” to copy successful individuals?, Trends Cogn. Sci., 13, 197 Hills, 2012, The evolution of cognitive search, 11 Charnov, 1976, Optimal foraging, the marginal value theorem, Theor. Popul. Biol., 9, 129, 10.1016/0040-5809(76)90040-X Stephens, 1987 Chun, 1996, Just say no: how are visual searches terminated when there is no target present?, Cogn. Psychol., 30, 39, 10.1006/cogp.1996.0002 Wolfe, 2005, Rare items often missed in visual searches, Nature, 435, 439, 10.1038/435439a Kristjansson, 2010, Fortune and reversals of fortune in visual search: reward contingencies for pop-out targets affect search efficiency and target repetition effects, Atten. Percept. Psychophys., 72, 1229, 10.3758/APP.72.5.1229 Cousineau, 2004, Termination of a visual search with large display size effects, Spat. Vis., 17, 327, 10.1163/1568568041920104 Hills, 2008, Population heterogeneity and individual differences in an assortative agent-based marriage and divorce model (MADAM) using search with relaxing expectations, J. Artif. Soc. Social Simul., 11, 5 Pirolli, 2005, Rational analyses of information foraging on the web, Cogn. Sci., 29, 343, 10.1207/s15516709cog0000_20 Gigerenzer, 2012, Efficient cognition through limited search, 608 Wilke, 2009, Fishing for the right words: decision rules for human foraging behavior in internal search tasks, Cogn. Sci., 33, 497, 10.1111/j.1551-6709.2009.01020.x Hutchinson, 2008, Patch leaving in humans: can a generalist adapt its rules to dispersal of items across patches?, Anim. Behav., 75, 1331, 10.1016/j.anbehav.2007.09.006 Scheibehenne, 2011, Expectations of clumpy resources influence predictions of sequential events, Evol. Hum. Behav., 32, 326, 10.1016/j.evolhumbehav.2010.11.003 Dayan, 2014, Model-based and model-free Pavlovian reward learning: revaluation, revision, and revelation, Cogn. Affect. Behav. Neurosci., 14, 473, 10.3758/s13415-014-0277-8 Blanco, 2013, The influence of depression symptoms on exploratory decision-making, Cognition, 129, 563, 10.1016/j.cognition.2013.08.018 Otto, 2014, Physiological and behavioral signatures of reflective exploratory choice, Cogn. Affect. Behav. Neurosci., 14, 1167, 10.3758/s13415-014-0260-4 Markant, 2014, Is it better to select or to receive? Learning via active and passive hypothesis testing, J. Exp. Psychol. Gen., 143, 94, 10.1037/a0032108 Sims, 2013, Melioration as rational choice: sequential decision making in uncertain environments, Psychol. Rev., 120, 139, 10.1037/a0030850 Metcalfe, 2010, People's study time allocation and its relation to animal foraging, Behav. Processes, 83, 213, 10.1016/j.beproc.2009.12.011 Hills, 2010, The central executive as a search process: priming exploration and exploitation across domains, J. Exp. Psychol. Gen., 139, 590, 10.1037/a0020666 Hills, 2012, Dynamic search and working memory in social recall, J. Exp. Psychol. Learn. Mem. Cogn., 38, 218, 10.1037/a0025161 Rakow, 2008, Biased samples not mode of presentation: re-examining the apparent underweighting of rare events in experience-based choice, Organ. Behav. Hum. Decis. Process., 106, 168, 10.1016/j.obhdp.2008.02.001 Unsworth, 2007, Individual differences in working memory capacity and episodic retrieval: examining the dynamics of delayed and continuous distractor free recall, J. Exp. Psychol. Learn. Mem. Cogn., 33, 1020, 10.1037/0278-7393.33.6.1020 Hills, 2013, Mechanisms of age-related decline in memory search across the adult life span, Dev. Psychol., 49, 2396, 10.1037/a0032272 Sobel, 2007, Individual differences in working memory capacity and visual search: the roles of top-down and bottom-up processing, Psychon. Bull. Rev., 14, 840, 10.3758/BF03194109 Anderson, 2013, A common discrete resource for visual working memory and visual search, Psychol. Sci., 24, 929, 10.1177/0956797612464380 Wolfe, 2012, Visual foraging behavior: when are the berries riper on the other side of the screen?, J. Vis., 12, 265, 10.1167/12.9.265 Fuster, 2008 Goldman-Rakic, 1995, Cellular basis of working memory, Neuron, 14, 477, 10.1016/0896-6273(95)90304-6 Jones, 2005, Theta rhythms coordinate hippocampal–prefrontal interactions in a spatial memory task, PLoS Biol., 3, e402, 10.1371/journal.pbio.0030402 Hyman, 2010, Working memory performance correlates with prefrontal–hippocampal theta interactions but not with prefrontal neuron firing rates, Front. Integr. Neurosci., 4, 1 Benchenane, 2010, Coherent theta oscillations and reorganization of spike timing in the hippocampal–prefrontal network upon learning, Neuron, 66, 921, 10.1016/j.neuron.2010.05.013 Voss, 2010, Hippocampal brain-network coordination during volitional exploratory behavior enhances learning, Nat. Neurosci., 14, 115, 10.1038/nn.2693 Schacter, 2009, On the nature of medial temporal lobe contributions to the constructive simulation of future events, Philos. Trans. R. Soc. Lond. B: Biol. Sci., 364, 1245, 10.1098/rstb.2008.0308 Barnes, 2005, Activity of striatal neurons reflects dynamic encoding and recoding of procedural memories, Nature, 437, 1158, 10.1038/nature04053 Smith, 2013, A dual operator view of habitual behavior reflecting cortical and striatal dynamics, Neuron, 79, 361, 10.1016/j.neuron.2013.05.038 Dezfouli, 2012, Habits, action sequences and reinforcement learning, Eur. J. Neurosci., 35, 1036, 10.1111/j.1460-9568.2012.08050.x Robbins, 2007, Differential regulation of fronto-executive function by the monoamines and acetylcholine, Cereb. Cortex, 17, i151, 10.1093/cercor/bhm066 Cools, 2011, Inverted-U-shaped dopamine actions on human working memory and cognitive control, Biol. Psychiatry, 69, e113, 10.1016/j.biopsych.2011.03.028 Schultz, 1998, Predictive reward signal of dopamine neurons, J. Neurophysiol., 80, 1, 10.1152/jn.1998.80.1.1 Barron, 2010, The roles of dopamine and related compounds in reward-seeking behavior across animal phyla, Front. Behav. Neurosci., 4, 1, 10.3389/fnbeh.2010.00163 Redish, 2007, Reconciling reinforcement learning models with behavioral extinction and renewal: implications for addiction, relapse, and problem gambling, Psychol. Rev., 114, 784, 10.1037/0033-295X.114.3.784 Beeler, 2010, Tonic dopamine modulates exploitation of reward learning, Front. Behav. Neurosci., 4, 1, 10.3389/fnbeh.2010.00170 Servan-Schreiber, 1990, A network model of catecholamine effects: gain, signal-to-noise ratio, and behavior, Science, 249, 892, 10.1126/science.2392679 Durstewitz, 2008, The dual-state theory of prefrontal cortex dopamine function with relevance to catechol-O-methyltransferase genotypes and schizophrenia, Biol. Psychiatry, 64, 739, 10.1016/j.biopsych.2008.05.015 Yu, 2005, Uncertainty, neuromodulation, and attention, Neuron, 46, 681, 10.1016/j.neuron.2005.04.026 Aston-Jones, 2005, An integrative theory of locus coeruleus–norepinephrine function: adaptive gain and optimal performance, Annu. Rev. Neurosci., 28, 403, 10.1146/annurev.neuro.28.061604.135709 Kehagia, 2010, Learning and cognitive flexibility: frontostriatal function and monoaminergic modulation, Curr. Opin. Neurobiol., 20, 199, 10.1016/j.conb.2010.01.007 Huys, 2012, Bonsai trees in your head: how the Pavlovian system sculpts goal-directed choices by pruning decision trees, PLoS Comput. Biol., 8, e1002410, 10.1371/journal.pcbi.1002410 Winstanley, 2012, Search, goals, and the brain, 125 Frisch von, 1967 Liang, 2012, Molecular determinants of scouting behavior in honey bees, Science, 335, 1225, 10.1126/science.1213962 Beekman, 2007, What makes a honeybee scout?, Behav. Ecol. Sociobiol., 61, 985, 10.1007/s00265-006-0331-9 Mason, 2008, Propagation of innovations in networked groups, J. Exp. Psychol. Gen., 137, 422, 10.1037/a0012798 Lazer, 2007, The network structure of exploration and exploitation, Admin. Sci. Quart., 52, 667, 10.2189/asqu.52.4.667 Kollman, 2000, Decentralization and the search for policy solutions, J. Law Econ. Organ., 16, 102, 10.1093/jleo/16.1.102 Sorenson, 2001, Finding the right mix: franchising, organizational learning, and chain performance, Strat. Mgmt J., 22, 713, 10.1002/smj.185 He, 2004, Exploration vs. exploitation: an empirical test of the ambidexterity hypothesis, Organ. Sci., 15, 481, 10.1287/orsc.1040.0078 Rendell, 2010, Why copy others? Insights from the social learning strategies tournament, Science, 328, 208, 10.1126/science.1184719 Wisdom, 2013, Social learning strategies in networked groups, Cogn. Sci., 37, 1383, 10.1111/cogs.12052 Berdahl, 2013, Emergent sensing of complex environments by mobile animal groups, Science, 339, 574, 10.1126/science.1225883 Roberts, 2011, Adaptive group coordination and role differentiation, PLoS ONE, 6, e22377, 10.1371/journal.pone.0022377 Page, 2007 Krause, 2011, Swarm intelligence in humans: diversity can trump ability, Anim. Behav., 81, 941, 10.1016/j.anbehav.2010.12.018 Nemeth, 1983, Creative problem solving as a result of majority vs minority influence, Eur. J. Soc. Psychol., 13, 45, 10.1002/ejsp.2420130103 Couzin, 2011, Uninformed individuals promote democratic consensus in animal groups, Science, 334, 1578, 10.1126/science.1210280 Habermas, 1985 2012 Goldstone, 2009, Collective behavior, Top. Cogn. Sci., 1, 412, 10.1111/j.1756-8765.2009.01038.x Gavrilets, 1997, Evolution and speciation on holey adaptive landscapes, Trends Ecol. Evol., 12, 307, 10.1016/S0169-5347(97)01098-7 Gureckis, 2009, Learning in noise: dynamic decision-making in a variable environment, J. Math. Psychol., 53, 180, 10.1016/j.jmp.2009.02.004 Otto, 2009, Navigating through abstract decision spaces: evaluating the role of state generalization in a dynamic decision-making task, Psychon. Bull. Rev., 16, 957, 10.3758/PBR.16.5.957 Borge-Holthoefer, 2011, Modeling abnormal priming in Alzheimer's patients with a free association network, PLoS ONE, 6, e22651, 10.1371/journal.pone.0022651 Ramscar, 2014, The myth of cognitive decline: non-linear dynamics of lifelong learning, Top. Cogn. Sci., 6, 5, 10.1111/tops.12078 Nonacs, 2001, State dependent behavior and the marginal value theorem, Behav. Ecol., 12, 71, 10.1093/oxfordjournals.beheco.a000381 Beachly, 1995, On the economics of sit-and-wait foraging: site selection and assessment, Behav. Ecol., 6, 258, 10.1093/beheco/6.3.258 Kareiva, 1987, Swarms of predators exhibit ‘preytaxis’ if individual predators use area-restricted search, Am. Nat., 130, 233, 10.1086/284707 Viswanathan, 1999, Optimizing the success of random searches, Nature, 401, 911, 10.1038/44831 Hills, 2013, Adaptive Lévy processes and area-restricted search in human foraging, PLoS ONE, 8, e60488, 10.1371/journal.pone.0060488 Ferreira, 2012, The influence of the environment on Lévy random search efficiency: fractality and memory effects, Physica A, 391, 3234, 10.1016/j.physa.2012.01.028 Klein, 1988, Inhibitory tagging system facilitates visual search, Nature, 334, 430, 10.1038/334430a0 Peterson, 2001, Visual search has memory, Psychol. Sci., 12, 287, 10.1111/1467-9280.00353 Wolfe, 2000, Attention is fast but volition is slow, Nature, 406, 691, 10.1038/35021132 Buckner, 2007, Self-projection and the brain, Trends Cogn. Sci., 11, 49, 10.1016/j.tics.2006.11.004 Redish, 2013 Kurth-Nelson, 2012, A theoretical account of cognitive effects in delay discounting, Eur. J. Neurosci., 35, 1052, 10.1111/j.1460-9568.2012.08058.x Otto, 2013, Working-memory capacity protects model-based learning from stress, Proc. Natl. Acad. Sci. U.S.A., 110, 20941, 10.1073/pnas.1312011110 Genovesio, 2014, Prefrontal–parietal function: from foraging to foresight, Trends Cogn. Sci., 18, 72, 10.1016/j.tics.2013.11.007 Tolman, 1938, The determiners of behavior at a choice point, Psychol. Rev., 45, 1, 10.1037/h0062733 Johnson, 2007, Neural ensembles in CA3 transiently encode paths forward of the animal at a decision point, J. Neurosci., 27, 12176, 10.1523/JNEUROSCI.3761-07.2007 Schacter, 2007, The cognitive neuroscience of constructive memory: remembering the past and imagining the future, Philos. Trans. R. Soc. Lond. B: Biol. Sci., 362, 773, 10.1098/rstb.2007.2087 Hassabis, 2007, Patients with hippocampal amnesia cannot imagine new experiences, Proc. Natl. Acad. Sci. U.S.A., 104, 1726, 10.1073/pnas.0610561104 Keramati, 2011, Speed/accuracy trade-off between the habitual and the goal-directed processes, PLoS Comput. Biol., 7, e1002055, 10.1371/journal.pcbi.1002055 Johnson, 2012, The hippocampus and exploration: dynamically evolving behavior and neural representations, Front. Hum. Neurosci., 6, 216, 10.3389/fnhum.2012.00216 van der Meer, 2012, Information processing in decision-making systems, Neuroscientist, 18, 342, 10.1177/1073858411435128 Daw, 2005, Uncertainty-based competition between prefrontal and dorsolateral striatal systems for behavioral control, Nat. Neurosci., 8, 1704, 10.1038/nn1560 van Bergen, 2004, Nine-spined sticklebacks exploit the most reliable source when public and private information conflict, Proc. Biol. Sci., 271, 957, 10.1098/rspb.2004.2684 Barnard, 1981, Producers and scroungers: a general model and its application to captive flocks of house sparrows, Anim. Behav., 29, 543, 10.1016/S0003-3472(81)80117-0 Vickery, 1991, Producers, scroungers, and group foraging, Am. Nat., 137, 847, 10.1086/285197 Giraldeau, 2000 Giraldeau, 2002, Potential disadvantages of using socially acquired information, Philos. Trans. R. Soc. Lond. B: Biol. Sci., 357, 1559, 10.1098/rstb.2002.1065 Simons, 2004, Many wrongs: the advantage of group navigation, Trends Ecol. Evol., 19, 453, 10.1016/j.tree.2004.07.001 Guttal, 2010, Social interactions, information use, and the evolution of collective migration, Proc. Natl. Acad. Sci. U.S.A., 107, 16172, 10.1073/pnas.1006874107 Dussutour, 2009, The role of multiple pheromones in food recruitment by ants, J. Exp. Biol., 212, 2337, 10.1242/jeb.029827 Deneubourg, 1983, Probabilistic behaviour in ants: a strategy of errors?, J. Theor. Biol., 105, 259, 10.1016/S0022-5193(83)80007-1 Weidenmüller, 1999, Imprecision in waggle dances of the honeybee (Apis mellifera) for nearby food sources: error or adaptation?, Behav. Ecol. Sociobiol., 46, 190, 10.1007/s002650050609 Galton, 1907, Vox populi, Nature, 75, 450, 10.1038/075450a0