Survey of sampling-based methods for uncertainty and sensitivity analysis

Reliability Engineering & System Safety - Tập 91 Số 10-11 - Trang 1175-1209 - 2006
J.C. Helton1, Jay Johnson2, Cedric Jean-Marie Sallaberry3, Curtis B. Storlie4
1Department of Mathematics and Statistics, Arizona State University, Tempe, AZ 85287-1804, USA
2ProStat, Mesa, AZ 85204-5326, USA
3Sandia National Laboratories, Albuquerque, NM 87185-0776, USA
4Department of Statistics, Colorado State University, Fort Collins, CO 80523-1877, USA

Tóm tắt

Từ khóa


Tài liệu tham khảo

Christie, 2005, Error analysis and simulations of complex phenomena, Los Alamos Sci, 29, 6

2004

Sharp, 2003, QMU and nuclear weapons certification: what's under the hood?, Los Alamos Sci, 28, 47

Wagner, 2003, Science, uncertainty and risk: the problem of complex phenomena, APS News, 12, 8

Oberkampf, 2002, Error and uncertainty in modeling and simulation, Reliab Eng Syst Safety, 75, 333, 10.1016/S0951-8320(01)00120-X

Roache, 1998

1997

1997

1996

1994

1993

1993

1989

Beck, 1987, Water-quality modeling: a review of the analysis of uncertainty, Water Resour Res, 23, 1393, 10.1029/WR023i008p01393

Helton, 1997, Uncertainty and sensitivity analysis in the presence of stochastic and subjective uncertainty, J Stat Comput Simul, 57, 3, 10.1080/00949659708811803

1996, Reliab Eng Syst Safety, 54, 91

Paté-Cornell, 1996, Uncertainties in Risk Anal: six levels of treatment, Reliab Eng Syst Safety, 54, 95, 10.1016/S0951-8320(96)00067-1

Winkler, 1996, Uncertainty in probabilistic risk assessment, Reliab Eng Syst Safety, 54, 127, 10.1016/S0951-8320(96)00070-1

Hoffman, 1994, Propagation of uncertainty in risk assessments: the need to distinguish between uncertainty due to lack of knowledge and uncertainty due to variability, Risk Anal, 14, 707, 10.1111/j.1539-6924.1994.tb00281.x

Helton, 1994, Treatment of uncertainty in performance assessments for complex systems, Risk Anal, 14, 483, 10.1111/j.1539-6924.1994.tb00266.x

Apostolakis, 1990, The concept of probability in safety assessments of technological systems, Science, 250, 1359, 10.1126/science.2255906

Haan, 1989, Parametric uncertainty in hydrologic modeling, Trans ASAE, 32, 137, 10.13031/2013.30973

Parry, 1981, Characterization and evaluation of uncertainty in probabilistic Risk Anal, Nucl Safety, 22, 28

Kaplan, 1981, On the quantitative definition of risk, Risk Anal, 1, 11, 10.1111/j.1539-6924.1981.tb01350.x

Cacuci, 2003

Griewank, 2000

Berz, 1996

Cacuci, 1994, On the application of the adjoint method of sensitivity analysis to problems in the atmospheric sciences, Atmósfera, 7, 47

Turányi, 1990, Sensitivity analysis of complex kinetic systems, tools and applications, J Math Chem, 5, 203, 10.1007/BF01166355

Rabitz, 1983, Sensitivity analysis in chemical kinetics, vol. 34, 419

1982, vol. 14

Frank, 1978

Tomovic, 1972

Myers, 2004, Response surface methodology: a retrospective and literature review, J Quality Technol, 36, 53, 10.1080/00224065.2004.11980252

Myers, 1999, Response surface methodology—current status and future directions, J Quality Technol, 31, 30, 10.1080/00224065.1999.11979891

Andres, 1997, Sampling methods and sensitivity analysis for large parameter sets, J Stat Comput Simul, 57, 77, 10.1080/00949659708811804

Kleijnen, 1997, Sensitivity analysis and related analyses: a review of some statistical techniques, J Stat Comput Simul, 57, 111, 10.1080/00949659708811805

Kleijnen, 1992, Sensitivity analysis of simulation experiments: regression analysis and statistical design, Math Comput Simul, 34, 297, 10.1016/0378-4754(92)90007-4

Myers, 1989, Response surface methodology: 1966–1988, Technometrics, 31, 137, 10.2307/1268813

Sacks, 1989, Design and analysis of computer experiments, Stat Sci, 4, 409, 10.1214/ss/1177012413

Morton, 1983, Response surface methodology, Math Sci, 8, 31

Mead, 1975, A review of response surface methodology from a biometric viewpoint, Biometrics, 31, 803, 10.2307/2529809

Myers, 1971

Helton, 2003, Latin hypercube sampling and the propagation of uncertainty in analyses of complex systems, Reliab Eng Syst Safety, 81, 23, 10.1016/S0951-8320(03)00058-9

Helton, 2002, Illustration of sampling-based methods for uncertainty and sensitivity analysis, Risk Anal, 22, 591, 10.1111/0272-4332.00041

Helton, 2000, Sampling-based methods, 101

Kleijnen, 1999, Statistical analyses of scatterplots to identify important factors in large-scale simulations, 1: review and comparison of techniques, Reliab Eng Syst Safety, 65, 147, 10.1016/S0951-8320(98)00091-X

Blower, 1994, Sensitivity and uncertainty analysis of complex models of disease transmission: an HIV model, as an example, Int Stat Rev, 62, 229, 10.2307/1403510

Saltelli, 1993, Sensitivity analysis of model output. An investigation of new techniques, Comput Stat Data Anal, 15, 445, 10.1016/0167-9473(93)90193-W

Iman, 1992, Uncertainty and sensitivity analysis for computer modeling applications, vol. 28, 153

Saltelli, 1990, Non-parametric statistics in sensitivity analysis for model output. A comparison of selected techniques, Reliab Eng Syst Safety, 28, 229, 10.1016/0951-8320(90)90065-U

Iman, 1981, An approach to sensitivity analysis of computer models, Part 1. Introduction, input variable selection and preliminary variable assessment, J Quality Technol, 13, 174, 10.1080/00224065.1981.11978748

Iman, 1981, An approach to sensitivity analysis of computer models, Part 2. Ranking of input variables, response surface validation, distribution effect and technique synopsis, J Quality Technol, 13, 232, 10.1080/00224065.1981.11978763

Iman, 1980, Small sample sensitivity analysis techniques for computer models, with an application to risk assessment, Commun Stat: Theory Methods A, 9, 1749, 10.1080/03610928008827996

McKay, 1979, A comparison of three methods for selecting values of input variables in the analysis of output from a computer code, Technometrics, 21, 239, 10.2307/1268522

Li, 2001, High-dimensional model representations, J Phys Chem, 105, 7765, 10.1021/jp010450t

Rabitz, 1999, General foundations of high-dimensional model representations, J Math Chem, 25, 197, 10.1023/A:1019188517934

Saltelli, 1999, A quantitative model-independent method for global sensitivity analysis of model output, Technometrics, 41, 39, 10.2307/1270993

Sobol’, 1993, Sensitivity estimates for nonlinear mathematical models, Math Modeling Comput Exp, 1, 407

Cukier, 1978, Nonlinear sensitivity analysis of multiparameter model systems, J Comput Phys, 26, 1, 10.1016/0021-9991(78)90097-9

Ionescu-Bujor, 2004, A comparative review of sensitivity and uncertainty analysis of large-scale systems–I: deterministic methods, Nucl Sci Eng, 147, 189, 10.13182/NSE03-105CR

Cacuci, 2004, A comparative review of sensitivity and uncertainty analysis of large-scale systems–II: statistical methods, Nucl Sci Eng, 147, 204, 10.13182/04-54CR

Frey, 2002, Identification and review of sensitivity analysis methods, Risk Anal, 22, 553, 10.1111/0272-4332.00039

2000

Hamby, 1994, A review of techniques for parameter sensitivity analysis of environmental models, Environ Monitor Assess, 32, 135, 10.1007/BF00547132

Helton, 1993, Uncertainty and sensitivity analysis techniques for use in performance assessment for radioactive waste disposal, Reliab Eng Syst Safety, 42, 327, 10.1016/0951-8320(93)90097-I

Ronen, 1988

Iman, 1988, An investigation of uncertainty and sensitivity analysis techniques for computer models, Risk Anal, 8, 71, 10.1111/j.1539-6924.1988.tb01155.x

Blower, 2000, A tale of two futures: HIV and antiretroviral therapy in San Francisco, Science, 287, 650, 10.1126/science.287.5453.650

Cohen, 2000, A discrete-event computer model of feline herpes virus within cat populations, Prevent Vet Med, 45, 163, 10.1016/S0167-5877(00)00132-X

Hofer, 1999, Sensitivity analysis in the context of uncertainty analysis for computationally intensive models, Comput Phys Commun, 117, 21, 10.1016/S0010-4655(98)00153-2

Caswell, 1998, Harbor porpoise and fisheries: an uncertainty analysis of incidental mortality, Ecol Appl, 8, 1226, 10.1890/1051-0761(1998)008[1226:HPAFAU]2.0.CO;2

Sanchez, 1997, Uncertainty and sensitivity analysis of the basic reproductive rate: tuberculosis as an example, Am J Epidemiol, 145, 1127, 10.1093/oxfordjournals.aje.a009076

Chan, 1996, The consequences of uncertainty for the prediction of the effects of schistosomiasis control programmes, Epidemiol Infect, 117, 537, 10.1017/S0950268800059239

Gwo, 1996, Subsurface stormflow modeling with sensitivity analysis using a latin-hypercube sampling technique, Ground Water, 34, 811, 10.1111/j.1745-6584.1996.tb02075.x

Helton, 1996, Uncertainty and sensitivity analysis results obtained in the 1992 performance assessment for the waste isolation pilot plant, Reliab Eng Syst Safety, 51, 53, 10.1016/0951-8320(95)00089-5

Kolev, 1996, Uncertainty and sensitivity analysis of a postexperiment simulation of nonexplosive melt-water interaction, Exp Thermal Fluid Sci, 13, 98, 10.1016/0894-1777(96)00034-9

Whiting, 1993, Effect of uncertainties in thermodynamic data and model parameters on calculated process performance, Ind Eng Chem Res, 32, 1367, 10.1021/ie00019a011

Ma, 1993, Parameter sensitivity of a model of viral epidemics simulated with Monte Carlo techniques. I. Illness attack rates, Int J Biomed Comput, 32, 237, 10.1016/0020-7101(93)90017-Z

Ma, 1993, Parameter sensitivity of a model of viral epidemics simulated with Monte Carlo techniques. II. Durations and peaks, Int J Biomed Comput, 32, 255, 10.1016/0020-7101(93)90018-2

Breshears, 1992, Contaminant transport through agroecosystems: assessing relative importance of environmental, physiological, and management factors, Ecol Appl, 2, 285, 10.2307/1941862

Breeding, 1992, Summary description of the methods used in the probabilistic risk assessments for NUREG-1150, Nucl Eng Des, 135, 1, 10.1016/0029-5493(92)90300-K

MacDonald, 1986, Valuation of supplemental and enhanced oil recovery projects with risk analysis, J Petroleum Technol, 38, 57, 10.2118/11303-PA

Helton, 2004, An exploration of alternative approaches to the representation of uncertainty in model predictions, Reliab Eng Syst Safety, 85, 39, 10.1016/j.ress.2004.03.025

Klir, 2004, Generalized information theory: aims, results, and open problems, Reliab Eng Syst Safety, 85, 21, 10.1016/j.ress.2004.03.003

Ross, 2004

Halpern, 2003

2002

Jaulin, 2001

Wolkenhauer, 2001

Klir, 1999

1994

Cooke, 2004, Expert judgement elicitation for risk assessment of critical infrastructures, J Risk Res, 7, 643, 10.1080/1366987042000192237

Ayyub, 2001

McKay, 2000, Critique of and limitations on the use of expert judgements in accident consequence uncertainty analysis, Radiat Protect Dosim, 90, 325, 10.1093/oxfordjournals.rpd.a033155

Budnitz, 1998, Use of technical expert panels: applications to probabilistic seismic hazard analysis, Risk Anal, 18, 463, 10.1111/j.1539-6924.1998.tb00361.x

Thorne, 1992, A review of expert judgement techniques with reference to nuclear safety, Progr Nucl Safety, 27, 83, 10.1016/0149-1970(92)90001-J

Cooke, 1991

Meyer, 1991

Hora, 1989, Expert opinion in risk analysis: the NUREG-1150 methodology, Nucl Sci Eng, 102, 323, 10.13182/NSE89-A23645

Helton, 2000

Vaughn, 2000, Representation of two-phase flow in the vicinity of the repository in the 1996 performance assessment for the waste isolation pilot plant, Reliab Eng Syst Safety, 69, 205, 10.1016/S0951-8320(00)00033-8

Helton, 2000, Uncertainty and sensitivity analysis for two-phase flow in the vicinity of the repository in the 1996 performance assessment for the waste isolation pilot plant: undisturbed conditions, Reliab Eng Syst Safety, 69, 227, 10.1016/S0951-8320(00)00034-X

Helton, 2000, Uncertainty and sensitivity analysis for two-phase flow in the vicinity of the repository in the 1996 performance assessment for the waste isolation pilot plant: disturbed conditions, Reliab Eng Syst Safety, 69, 263, 10.1016/S0951-8320(00)00035-1

1996

Helton, 1998

Helton, 2000, Special issue: the 1996 performance assessment for the waste isolation pilot plant, Reliab Eng Syst Safety, 69, 1, 10.1016/S0951-8320(00)00022-3

Helton, 2000, Characterization of subjective uncertainty in the 1996 performance assessment for the waste isolation pilot plant, Reliab Eng Syst Safety, 69, 191, 10.1016/S0951-8320(00)00032-6

Goossens, 2000, Expert judgement for a probabilistic accident consequence uncertainty analysis, Radiat Protect Dosim, 90, 295, 10.1093/oxfordjournals.rpd.a033151

Goossens, 1998, Joint EC/USNRC expert judgement driven radiological protection uncertainty analysis, J Radiol Protect, 18, 249, 10.1088/0952-4746/18/4/003

Siu, 1998, Bayesian parameter estimation in probabilistic risk assessment, Reliab Eng Syst Safety, 62, 89, 10.1016/S0951-8320(97)00159-2

Evans, 1994, Use of probabilistic expert judgement in uncertainty analysis of carcinogenic potency, Regul Toxicol Pharmacol, 20, 15, 10.1006/rtph.1994.1034

Chhibber, 1992, A taxonomy of issues related to the use of expert judgments in probabilistic safety studies, Reliab Eng Syst Safety, 38, 27, 10.1016/0951-8320(92)90103-R

Kaplan, 1992, Expert information versus expert opinions: another approach to the problem of eliciting combining using expert knowledge in PRA, Reliab Eng Syst Safety, 35, 61, 10.1016/0951-8320(92)90023-E

Otway, 1992, Expert judgement in risk analysis and management: process, context, and pitfalls, Risk Anal, 12, 83, 10.1111/j.1539-6924.1992.tb01310.x

Bonano, 1991, Theoretical foundations and practical issues for using expert judgments in uncertainty analysis of high-level radioactive waste disposal, Radioactive Waste Manage Nucl Fuel Cycle, 16, 137

Keeney, 1991, Eliciting probabilities from experts in complex technical problems, IEEE Trans Eng Manage, 38, 191, 10.1109/17.83752

Svenson, 1989, on expert judgments in safety analyses in the process industries, Reliab Eng Syst Safety, 25, 219, 10.1016/0951-8320(89)90095-1

Mosleh, 1988, A critique of current practice for the use of expert opinions in probabilistic risk assessment, Reliab Eng Syst Safety, 20, 63, 10.1016/0951-8320(88)90006-3

Breeding, 1992, The NUREG-1150 probabilistic risk assessment for the surry nuclear power station, Nucl Eng Des, 135, 29, 10.1016/0029-5493(92)90301-B

Payne, 1992, The NUREG-1150 probabilistic risk assessment for the peach bottom atomic power station, Nucl Eng Des, 135, 61, 10.1016/0029-5493(92)90302-C

Gregory, 1992, The NUREG-1150 probabilistic risk assessment for the sequoyah nuclear plant, Nucl Eng Des, 135, 92, 10.1016/0029-5493(92)90303-D

Brown, 1992, The NUREG-1150 probabilistic risk assessment for the grand gulf nuclear station, Nucl Eng Des, 135, 117, 10.1016/0029-5493(92)90304-E

Helton, 1993, Calculation of reactor accident safety goals, Reliab Eng Syst Safety, 39, 129, 10.1016/0951-8320(93)90038-Z

Morris, 2000, Three technometrics experimental design classics, Technometrics, 42, 26, 10.2307/1271429

Evans, 2000

Hurtado, 1998, Monte Carlo techniques in computational stochastic mechanics, Arch Comput Methods Eng, 5, 3, 10.1007/BF02736747

Nicola, 2001, Techniques for fast simulation of models of highly dependable systems, IEEE Trans Reliab, 50, 246, 10.1109/24.974122

Owen, 2000, Safe and effective importance sampling, J Am Stat Assoc, 95, 135, 10.2307/2669533

Heidelberger, 1995, Fast simulation of rare events in queueing and reliability models, ACM Trans Modeling Comput Simul, 5, 43, 10.1145/203091.203094

Shahabuddin, 1994, Importance sampling for the simulation of highly reliable Markovian systems, Manage Sci, 40, 333, 10.1287/mnsc.40.3.333

Goyal, 1992, A unified framework for simulating Markovian models of highly dependable systems, IEEE Trans Comput, 41, 36, 10.1109/12.123381

Melchers, 1990, Search-based importance sampling, Struct Safety, 9, 117, 10.1016/0167-4730(90)90003-8

Glynn, 1989, Importance sampling for stochastic simulations, Manage Sci, 35, 1367, 10.1287/mnsc.35.11.1367

Iman, 1982, A distribution-free approach to inducing rank correlation among input variables, Commun Stat: Simul Comput B, 11, 311, 10.1080/03610918208812265

Iman, 1982, Rank correlation plots for use with correlated input variables, Commun Stat: Simul Comput B, 11, 335, 10.1080/03610918208812266

Helton, 2002

Helton, 2005, A comparison of uncertainty and sensitivity analysis results obtained with random and Latin hypercube sampling, Reliab Eng Syst Safety, 89, 305, 10.1016/j.ress.2004.09.006

Kleinjen, 2005, An overview of the design and analysis of simulation experiments for sensitivity analysis, Europ J Operat Res, 164, 287, 10.1016/j.ejor.2004.02.005

2000

2000

1998

Ibrekk, 1987, Graphical communication of uncertain quantities to nontechnical people, Risk Anal, 7, 519, 10.1111/j.1539-6924.1987.tb00488.x

Tufte, 1983

Berglund, 2000, Direct releases to the surface and associated complementary cumulative distribution functions in the 1996 performance assessment for the waste isolation pilot plant: cuttings, cavings and spallings, Reliab Eng Syst Safety, 69, 305, 10.1016/S0951-8320(00)00036-3

Cooke, 2000, Graphical methods, 245

Myers, 1989

Draper, 1981

Daniel, 1980

Seber, 1977

Neter, 1974

Iman, 1985

Iman, 1979, The use of the rank transform in regression, Technometrics, 21, 499, 10.2307/1268289

Scheffé, 1959

Conover, 1980

Barnard, 1963, Contribution to the discussion of professor Bartlett's paper, J R Stat Soc Ser B, 26, 294

Press, 1992

Kleijnen, 1999, Statistical analyses of scatterplots to identify important factors in large-scale simulations, 2: robustness of techniques, Reliab Eng Syst Safety, 65, 187, 10.1016/S0951-8320(98)00090-8

Kleijnen, 1999

Wagner, 1995, Global sensitivity analysis, Oper Res, 43, 948, 10.1287/opre.43.6.948

Granger, 1994, Using the mutual information coefficient to identify lags in nonlinear models, J Time Ser Anal, 15, 371, 10.1111/j.1467-9892.1994.tb00200.x

Mishra S, Knowlton RG. Testing for input–output dependence in performance assessment models. In: Proceedings of the 10th international high-level radioactive waste management conference (IHLRWM), March 30–April 2, 2003. Las Vegas, Nevada. La Grange Park, IL: American Nuclear Society; p. 882–7.

Bonnlander BV, Weigend AS. Selecting input variables using mutual information and nonparametric density estimation. In: Proceedings of the international symposium on artificial neural networks (ISANN’94), Tainan, Taiwan, 1994. p. 42–50.

Moddemeijer, 1989, On the estimation of entropy and mutual information of continuous distributions, Signal Process, 16, 233, 10.1016/0165-1684(89)90132-1

Simonoff, 1996

Hastie, 1990

Chambers, 1992

Storlie CB, Helton JC. Multiple Predictor smoothing methods for sensitivity analysis, SAND2005-???? Albuquerque, NM: Sandia National Laboratories; to appear.

Ruppert, 2003

Breiman, 1984

Mishra, 2003, Application of classification trees in the sensitivity analysis of probabilistic model results, Reliab Eng Syst Safety, 79, 123, 10.1016/S0951-8320(02)00222-3

Hora, 2003, A distribution-free test for the relationship between model input and output when using Latin hypercube sampling, Reliab Eng Syst Safety, 79, 333, 10.1016/S0951-8320(02)00240-5

Peacock, 1983, Two-dimensional goodness-of-fit testing in astronomy, Monthly Notices R Astronom Soc, 202, 615, 10.1093/mnras/202.3.615

Fasano, 1987, A multidimensional version of the Kolmogorov–Smirnov test, Monthly Notices R Astronom Soc, 225, 155, 10.1093/mnras/225.1.155

Garvey, 1998, From star charts to stoneflies: detecting relationships in continuous bivariate data, Ecology, 79, 442, 10.1890/0012-9658(1998)079[0442:FSCTSD]2.0.CO;2

Ripley, 1987, Spatial point pattern analysis in ecology, vol. 14, 407

Zeng, 1985, A comparison of tests for randomness, Pattern Recogn, 18, 191, 10.1016/0031-3203(85)90043-3

Diggle, 1983

Diggle, 1983, Some distance-based tests of independence for sparsely sampled multivariate spatial point patterns, Int Stat Rev, 51, 11, 10.2307/1402729

Byth, 1982, On robust distance-based intensity estimators, Biometrics, 38, 127, 10.2307/2530295

Byth, 1980, On sampling spatial patterns by distance methods, Biometrics, 36, 279, 10.2307/2529979

Diggle, 1979, On parameter estimation and goodness-of-fit testing for spatial point patterns, Biometrics, 35, 87, 10.2307/2529938

Diggle, 1979, Statistical methods for spatial point patterns in ecology, 95

Ripley, 1979, Tests of randomness for spatial point patterns, J R Stat Soc, 41, 368

Besag, 1977, Simple Monte Carlo tests for spatial pattern, Appl Stat, 26, 327, 10.2307/2346974

Diggle, 1976, Statistical analysis of spatial point patterns by means of distance methods, Biometrics, 32, 659, 10.2307/2529754

Cox, 1976, A conditioned distance ratio method for analyzing spatial patterns, Biometrika, 63, 483, 10.1093/biomet/63.3.483

Holgate, 1972, The use of distance methods for the analysis of spatial distribution of points, 122

Holgate, 1965, Tests of randomness based on distance methods, Biometrika, 52, 345, 10.1093/biomet/52.3-4.345

Clark, 1954, Distance to nearest neighbour as a measure of spatial relationships in populations, Ecology, 35, 23, 10.2307/1931034

Hopkins, 1954, A new method for determining the type of distribution of plant individuals, Ann Botany, 18, 213, 10.1093/oxfordjournals.aob.a083391

Cressie, 1993

Iman, 1987, A measure of top-down correlation, Technometrics, 29, 351, 10.2307/1269344

Campolongo, 2000, Hitchhiker's guide to sensitivity analysis, 15

Saltelli, 2004

Cukier, 1973, Study of the sensitivity of coupled reaction systems to uncertainties in rate coefficients, I. Theory, J Chem Phys, 59, 3873, 10.1063/1.1680571

Schaibly, 1973, Study of the sensitivity of coupled reaction systems to uncertainties in rate coefficients, II. Applications, J Chem Phys, 59, 3879, 10.1063/1.1680572

McRae, 1981, Global sensitivity analysis—a computational implementation of the Fourier amplitude sensitivity test (FAST), Comput Chem Eng, 6, 15, 10.1016/0098-1354(82)80003-3

Saltelli, 1995, About the use of rank transformation in sensitivity analysis of model output, Reliab Eng Syst Safety, 50, 225, 10.1016/0951-8320(95)00099-2

Homma, 1996, Importance measures in global sensitivity analysis of nonlinear models, Reliab Eng Syst Safety, 52, 1, 10.1016/0951-8320(96)00002-6

Archer, 1997, Sensitivity measures, ANOVA-like techniques and the use of bootstrap, J Stat Comput Simul, 58, 99, 10.1080/00949659708811825

Saltelli, 1998, An alternative way to compute Fourier amplitude sensitivity test (FAST), Comput Stat Data Anal, 26, 267, 10.1016/S0167-9473(97)00043-1

Saltelli, 2000, Sensitivity analysis as an ingredient of modeling, Stat Sci, 15, 377, 10.1214/ss/1009213004

Chan, 2000, Winding stairs: a sampling tool to compute sensitivity indices, Stat Comput, 10, 187, 10.1023/A:1008950625967

Rabitz, 1999, Efficient input–output model representations, Comput Phys Commun, 117, 11, 10.1016/S0010-4655(98)00152-0

Jansen, 1994, Monte Carlo estimation of uncertainty contributions from several independent multivariate sources, 334

Jansen, 1999, Analysis of variance designs for model output, Comput Phys Commun, 117, 35, 10.1016/S0010-4655(98)00154-4

McKay, 1995

McKay, 1997, Nonparametric variance-based methods of assessing uncertainty importance, Reliab Eng Syst Safety, 57, 267, 10.1016/S0951-8320(97)00039-2

McKay, 1999, Evaluating prediction uncertainty in simulation models, Comput Phys Commun, 117, 44, 10.1016/S0010-4655(98)00155-6

Helton, 2003, Mathematical and numerical approaches in performance assessment for radioactive waste disposal: dealing with uncertainty, 353, 10.1016/S1569-4860(03)80068-2

Helton, 2000, Conceptual structure of the 1996 performance assessment for the waste isolation pilot plant, Reliab Eng Syst Safety, 69, 151, 10.1016/S0951-8320(00)00030-2

Diaz NJ. Realism and conservatism, remarks by chairman diaz at the 2003 nuclear safety research conference. NRC News. No. S-03-023, Washington, DC: US Nuclear Regulatory Commission; October 20, 2003.

Paté-Cornell, 2002, Risk and uncertainty analysis in government safety decisions, Risk Anal, 22, 633, 10.1111/0272-4332.00043

Caruso, 1999, An approach for using risk assessment in risk-informed decisions on plant-specific changes to the licensing basis, Reliab Eng Syst Safety, 63, 231, 10.1016/S0951-8320(98)00038-6

Sielken, 1995, Challenges to default assumptions stimulate comprehensive realism as a new tier in quantitative cancer risk assessment, Regul Toxicol Pharmacol, 21, 270, 10.1006/rtph.1995.1041

Nichols, 1988, The perils of prudence: how conservative risk assessments distort regulation, Regul Toxicol Pharmacol, 8, 61, 10.1016/0273-2300(88)90007-4