Context-oriented model validation of individual-based models in ecology: A hierarchically structured approach to validate qualitative, compositional and quantitative characteristics

Anscombe, 1973, Graphs in statistical analysis, Am. Stat., 27, 17 Augusiak, 2014, Merging validation and evaluation of ecological models to ‘evaludation’: a review of terminology and a practical approach, Ecol. Model., 280, 117, 10.1016/j.ecolmodel.2013.11.009 Aumann, 2006, How transient patches affect population dynamics: the case of hypoxia and blue crabs, Ecol. Monogr., 76, 415, 10.1890/0012-9615(2006)076[0415:HTPAPD]2.0.CO;2 Aumann, 2007, A methodology for developing simulation models of complex systems, Ecol. Model., 202, 385, 10.1016/j.ecolmodel.2006.11.005 Aumann, 2011, Constructing model credibility in the context of policy appraisal, Environ. Model. Softw., 26, 258, 10.1016/j.envsoft.2009.09.006 Arii, 2008, A selection harvesting algorithm for use in spatially explicit individual-based forest simulation models, Ecol. Model., 211, 251, 10.1016/j.ecolmodel.2007.09.007 Babin-Fenske, 2011, Agent-based simulation of effects of stress on forest tent caterpillar (Malacosoma disstria Hübner) population dynamics, Ecol. Model., 222, 2561, 10.1016/j.ecolmodel.2011.04.014 Bennett, 2013, Characterising performance of environmental models, Environ. Model. Softw., 40, 1, 10.1016/j.envsoft.2012.09.011 Berger, 1999, Virtual biologists observe virtual grasshoppers: an assessment of different mobility parameters for the analysis of movement patterns, Ecol. Model., 115, 119, 10.1016/S0304-3800(98)00187-2 Bragg, 2004, A hierarchical approach for simulating northern forest dynamics, Ecol. Model., 173, 31, 10.1016/j.ecolmodel.2003.08.017 Breckling, 2006, Individual-based models as tools for ecological theory and application: understanding the emergence of organisational properties in ecological systems, Ecol. Model., 194, 102, 10.1016/j.ecolmodel.2005.10.005 Breckling, 2011, System analysis and context assessment Carrasco, 2010, Unveiling human-assisted dispersal mechanisms in invasive alien insects: Integration of spatial stochastic simulation and phenology models, Ecol. Model., 221, 2068, 10.1016/j.ecolmodel.2010.05.012 Caron-Lormier, 2009, How might we model an ecosystem, Ecol. Model., 202, 1935, 10.1016/j.ecolmodel.2009.04.021 Castella, 2007, Combination of process-oriented and pattern-oriented models of land-use change in a mountain area of Vietnam, Ecol. Model., 202, 410, 10.1016/j.ecolmodel.2006.11.011 Charles, 2008, An individual-based model to describe a bullhead population dynamics including temperature variations, Ecol. Model., 215, 377, 10.1016/j.ecolmodel.2008.04.005 DeAngelis, 1993, Fish cohort dynamics: application of complementary modeling approaches, Am. Nat., 142, 604, 10.1086/285560 DeAngelis, 2003, In praise of mechanistically rich models, 63 Deygout, 2009, Modeling the impact of feeding stations on vulture scavenging service efficiency, Ecol. Model., 220, 1826, 10.1016/j.ecolmodel.2009.04.030 Dion, 2011, The landscape epidemiology of foot-and-mouth disease in South Africa: a spatially e0plicit multi-agent simulation, Ecol. Model., 222, 2059, 10.1016/j.ecolmodel.2011.03.026 Fei, 2012, Predicting micro thermal habitat of lizards in a dynamic thermal environment, Ecol. Model., 231, 126, 10.1016/j.ecolmodel.2012.02.012 Filatova, 2013, Spatial agent-based models for socio-ecological systems: challenges and prospects, Environ. Model. Softw., 45, 1, 10.1016/j.envsoft.2013.03.017 Fyllas, 2010, Simulating regeneration and vegetation dynamics in Mediterranean coniferous forests, Ecol. Model., 221, 1494, 10.1016/j.ecolmodel.2010.03.003 Gardner, 2003, Validation and Testing: Past lessons, present concerns, future prospects, 184 Grajetzky, 2000 van Griensven, 2006, A global sensitivity analysis tool for the parameters of multi-variable watershed models, J. Hydrol., 324, 10, 10.1016/j.jhydrol.2005.09.008 Grimm, 2002, Visual debugging: a way of analyzing, understanding and communicating bottom-up simulation models in ecology, Nat. Resour. Model., 15, 23, 10.1111/j.1939-7445.2002.tb00078.x Grimm, 1996, Pattern-oriented modelling in population ecology, Sci. Total Environ., 183, 151, 10.1016/0048-9697(95)04966-5 Grimm, 2005 Grimm, 2005, Pattern-oriented modeling of agent-based complex systems: lessons from ecology, Science, 310, 987, 10.1126/science.1116681 Grosman, 2011, Trade-off between road avoidance and attraction by roadside salt pools in moose: an agent-based model to assess measures for reducing moose-vehicle collisions, Ecol. Model., 222, 1423, 10.1016/j.ecolmodel.2011.01.022 Hamilton, 1991, Model validation: an annotated bibliography, Commun. Stat. Theory Methods, 20, 2207, 10.1080/03610929108830628 Hawking, 1988 Henttonen, 1985, Comparison of amplitudes and frequences (spectral analysis) of density variations in long-term data sets of Chlethrionomys species, Ann. Zool. Fenn., 22, 221 Herben, 2012, Community-level effects of plant traits in a grassland community e0amined by multispecies model of clonal plant growth, Ecol. Model., 234, 60, 10.1016/j.ecolmodel.2011.06.012 Hölker, 2005, A spatiotemporal individual-based fish model to investigate emergent properties at the organismal and the population level, Ecol. Model., 186, 406, 10.1016/j.ecolmodel.2005.02.010 Huitu, 2003, Landscape effects on temporal and spatial properties of vole population fluctuations, Oecologia, 135, 209, 10.1007/s00442-002-1171-6 Huston, 1988, New computer models unify ecological theory, Bioscience, 38, 682, 10.2307/1310870 Jager, 2004, Spatial uncertainty and ecological models, Ecosystems, 7, 841, 10.1007/s10021-004-0025-y Jakeman, 2006, Ten iterative steps in development and evaluation of environmental models, Environ. Model. Softw., 21, 602, 10.1016/j.envsoft.2006.01.004 Janssen, 1995, Calibration of process-oriented models, Ecol. Model., 83, 55, 10.1016/0304-3800(95)00084-9 Judson, 1994, The rise of the individual-based models in ecology, Trends Ecol. Evol., 9, 9, 10.1016/0169-5347(94)90225-9 Klepper, 1997, Multivariate aspects of model uncertainty analysis: tools für sensitvity analyis and calibration, Ecol. Model., 83, 55 Kubicek, 2012, Simulations of long-term community dynamics in coral reefs—how perturbations shape trajectories, PLoS Comput. Biol., 8, e1002791, 10.1371/journal.pcbi.1002791 Küppers, 2005, Validation of simulation: patterns in the social and natural sciences, J. Artif. Soc. Soc. Simul., 8 Krebs, 1996, Population cycles revisited, J. Mammal., 77, 8, 10.2307/1382705 Krueger, 2012, The role of expert opinion in environmental modelling, Environ. Model. Softw., 36, 4, 10.1016/j.envsoft.2012.01.011 Lammens, 2004, Effects of commercial fishery on the bream population and the e0pansion of Chara aspera in Lake Veluwe, Ecol. Model., 177, 233, 10.1016/j.ecolmodel.2003.10.002 Laperriére, 2009, Structural validation of an individual based model plague epidemic simulation, Ecol. Complex., 6, 102, 10.1016/j.ecocom.2008.08.001 Larocque, 2011, Development of the gap model ZELIG-CFS to predict the dynamics of North American mixed forest types with complex structures, Ecol. Model., 222, 2570, 10.1016/j.ecolmodel.2010.08.035 Latombe, 2011, Levels of emergence in individual based models: coping with scarctiy of data and pattern redundancy, Ecol. Model., 222, 1557, 10.1016/j.ecolmodel.2011.02.020 Lotka, 1925, Elements of physical biology Ma, 2009, Incorporating behaviour into simple models of dispersal using the biological control agent Dicyphus hesperus, Ecol. Model., 220, 3271, 10.1016/j.ecolmodel.2009.08.022 Makler-Pick, 2010, Sensitivity analysis for complex ecological models—a new approach, Environ. Model. Softw., 26, 124, 10.1016/j.envsoft.2010.06.010 Manson, 2003, Validation and verification of multi-agent models for ecosystem management, 63 Marks, 2007, Validating simulation models: a general framework and four applied examples, Comput. Econ., 30, 265, 10.1007/s10614-007-9101-7 McIntosh, 2011, Preface: thematic issue on the assessment and evaluation of environmental models and software, Environ. Model. Softw., 26, 245, 10.1016/j.envsoft.2010.08.008 Mitro, 2001, Ecological model testing: verification, validation, or neither, Bull. Ecol. Soc. Am., 82, 235 Moss, 2008, Alternative approaches to the empirical validation of agent-based models, J. Artif. Soc. Soc. Simul., 11, 5 Müller, 2011, What are the general conditions, under which ecological models can be applied Muthiga, 2008, Status of coral reefs in East Africa: Kenya, Tanzania, Mozambique and South Africa, 91 Nisbet, 1997, Primary-productivity gradients and short-term population dynamics in open systems, Ecol. Monogr., 67, 535, 10.1890/0012-9615(1997)067[0535:PPGAST]2.0.CO;2 Nyström, 2000, Coral reef disturbance and resilience in a human-dominated environment, Trends Ecol. Evol., 15, 413, 10.1016/S0169-5347(00)01948-0 Oreskes, 1994, Verification, validation, and confirmation of numerical models in the earth sciences, Science, 263, 641, 10.1126/science.263.5147.641 Pereira, A., Duarte, P., Reis, L., 2008. Agent-based ecological model calibration—on the edge of a new approach. In: Proceedings of the International Conference on Knowledge Engineering and Decision Support. ISEP, Porto, Portugal, 107–113. Petrovskii, 2012, Computational ecology as an emerging science, Interface Focus, 2, 241, 10.1098/rsfs.2011.0083 Popp, 2007, Simulating the impacts of vegetation structure on the occurrence of a small mammalian carnivore in semi-arid savanna rangelands, Ecol. Model., 209, 136, 10.1016/j.ecolmodel.2007.06.017 Power, 1993, The predictive validation of ecological and environmental models, Ecol. Model., 68, 33, 10.1016/0304-3800(93)90106-3 Qudrat-Ullah, H., 2005. Structural validation of system dynamics and agent- based simulation models. In: Merkuryev, Y., Zobel, R., Kerckhoffs, E. (Eds.), Proceedings of the 19th European Conference on Modelling and Simulation. ECMS. Railsback, 2011, Pattern-oriented modeling of bird foraging and pest control in coffee farms, Ecol. Model., 222, 3305, 10.1016/j.ecolmodel.2011.07.009 Ratz, 1995, Long-term spatial patterns created by fire: a model oriented towards boreal forests, Int. J. Wildland Fire, 5, 23, 10.1071/WF9950025 Reineking, 2006, Environmental variability and allocation trade-offs maintain species diversity in a process-based model of succulent plant communities, Ecol. Model., 199, 486, 10.1016/j.ecolmodel.2006.03.038 Reuter, 2005, Community processes as emergent properties: modelling multilevel interaction in small mammals communities, Ecol. Model., 186, 427, 10.1016/j.ecolmodel.2005.02.011 Reuter, 1999, Emerging properties on the individual level: modelling the reproduction phase of the European robin, Ecol. Model., 121, 199, 10.1016/S0304-3800(99)00078-2 Reuter, 2005, The concepts of emergent and collective properties individual-based models—summary and outlook of the Bornhöved case studies, Ecol. Model., 186, 489, 10.1016/j.ecolmodel.2005.02.014 Reuter, 2008, The ecological effect of phenotypic plasticity—analyzing complex interaction networks (COIN) with agent-based models, Ecol. Inform., 3, 35, 10.1016/j.ecoinf.2007.03.010 Reuter, 2010, Ecological hierarchies and self-organisation—pattern analysis, modelling and process integration across scales, Basic Appl. Ecol., 11, 572, 10.1016/j.baae.2010.08.002 Reuter, 2011, Individual-based models Rupp, 2010, Development of an individual-based model to evaluate elk (Cervus elaphus nelsoni) movement and distribution patterns following the Cerro Grande Fire in north central New Mexico, USA, Ecol. Model., 221, 1605, 10.1016/j.ecolmodel.2010.03.014 Rykiel, 1996, Testing ecological models: the meaning of validation, Ecol. Model., 90, 229, 10.1016/0304-3800(95)00152-2 Saltelli, 2010, How to avoid a perfunctory sensitivity analysis, Environ. Model. Softw., 25, 1508, 10.1016/j.envsoft.2010.04.012 Sargent, R.G., 1998. Verification and validation of simulation models. In: Proceedings of the 1998 Winter Simulation Conference, pp. 121–130. Sargent, R. G., 2003. Verification and validation of simulation models. In: Chick, S., Sanchez, P.J., Ferrin, E., Morrice, D.J. (Eds.), Proceedings of the Winter Simulation Conference, pp. 37–48. Sargent, 2013, Verification and validation of simulation models, J. Simul., 7, 12, 10.1057/jos.2012.20 Srinivasan, 2010, Behavioral games involving a clever prey avoiding a clever predator: An individual-based model of dusky dolphins and killer whales, Ecol. Model., 221, 2687, 10.1016/j.ecolmodel.2010.07.010 Stenseth, 1999, Population cycles in voles and lemmings: density dependence and phase dependence in a stochastic world, Oikos, 87, 427, 10.2307/3546809 Sundell, 2004, Large-scale spatial dynamics of vole populations in Finland revealed by the breeding success of vole-eating avian predators, J. Anim. Ecol., 73, 167, 10.1111/j.1365-2656.2004.00795.x Tang, 2012, Reprint of: parallel agent-based modeling of spatial opinion diffusion accelerated using graphics processing units, Ecol. Model., 229, 108, 10.1016/j.ecolmodel.2012.02.003 Tedeschi, 2006, Assessment of the adequacy of mathematical models, Agric. Syst., 89, 225, 10.1016/j.agsy.2005.11.004 Topping, 2012, Post-hoc pattern-oriented testing and tuning of an existing large model: lessons from the field vole, PLoS ONE, 7, e45872, 10.1371/journal.pone.0045872 Troitzsch, K., 2004. Validating simulation models. In: Horton, G. (Ed.), Proceedings of the 18th European Simulation Multiconference, SCS Europe. Turing, 1950, Computing machinery and intelligence, Mind, 59, 433, 10.1093/mind/LIX.236.433 Urban, 1990 Volterra, V., 1926. Variazioni e fluttuazioni del numero d’individui in specie animali conviventi. Mem. Accad. Lincei ser.6, 2(3), 31–112. Translation by Mary Evelyn Wells: variations and fluctuations of the number of individuals in animal species living together, see 〈http://icesjms.oxfordjournals.org/cgi/reprint/3/1/3.pdf〉. Wainwright, 2004 Wiegand, 2003, Using pattern-oriented modeling for revealing hidden information: a key for reconciling ecological theory and application, Oikos, 100, 209, 10.1034/j.1600-0706.2003.12027.x Wiegand, 2004, Dealing with uncertainty in spatially explicit population models, Biodivers. Conserv., 13, 53, 10.1023/B:BIOC.0000004313.86836.ab Windrum, 2007, Empirical validation of agent-based models: alternatives and prospects, J. Artif. Soc. Soc. Simul., 10 Yniguez, 2008, Allowing macroalgae growth forms to emerge: use of an agent-based model to understand the growth and spread of macroalgae in Florida coral reefs, with emphasis on Halimeda tuna, Ecol. Model., 216, 60, 10.1016/j.ecolmodel.2008.04.016 Zadereev, 2003, Zadereev individual-based model of the reproduction cycle of Moina macrocopa (Crustacea: Cladocera), Ecol. Model., 162, 15, 10.1016/S0304-3800(02)00348-4