An Approach to Perform Expert Elicitation for Engineering Design Risk Analysis: Methodology and Experimental Results

Journal of the Royal Statistical Society. Series A: Statistics in Society - Tập 177 Số 2 - Trang 475-497 - 2014
Alessandra Babuscia1, Kar‐Ming Cheung2
1Massachusetts Institute of Technology, Cambridge, USA
2Jet Propulsion Laboratory and California Institute of Technology , Pasadena , USA

Tóm tắt

SummaryExpert elicitation is increasingly applied to different research areas. Multiple approaches have been implemented, but the development of methods to quantify experts' biases and calibration represents a challenge. As a result, the integration of multiple and often conflicting opinions can be demanding, owing to the complexity of properly weighting experts' contributions. We propose an approach to address this problem when probability densities for seed calibration variables are not available. The methodology generates an expert score that is employed to aggregate multiple-expert assessments. The approach has been experimentally applied to engineering design risk analysis. Results indicate that the approach improves the quality of the estimations. The weighted aggregations of experts' estimates based on the experts' scores achieve better results than the corresponding aggregations based on experts' opinions equally weighted.

Từ khóa


Tài liệu tham khảo

Carlson, 1993, In defense of a constitutional theory of experts, Schol. Wrks, 1, 453

Clemen, 1999, Combining probability distributions from experts in risk analysis, Risk Anal., 19, 187, 10.1111/j.1539-6924.1999.tb00399.x

Cooke, 1991

Cooke, 2008, On the performance of social network and likelihood-based expert weighting schemes, Reliab. Engng Syst. Safty, 93, 745, 10.1016/j.ress.2007.03.017

Fiske, 1997, The conjunction fallacy: the case for the existence of competing heuristic strategies, Br. J. Psychol., 88, 1, 10.1111/j.2044-8295.1997.tb02617.x

Garthwaite, 2005, Statistical methods for eliciting probability distributions, J. Am. Statist. Ass., 470, 680, 10.1198/016214505000000105

Jacobs, 1995, Methods for combining experts' probability assessment, Neur. Computn, 7, 867, 10.1162/neco.1995.7.5.867

Jacobson, 1969, The maximum variance of restricted unimodal distributions, Ann. Math. Statist., 5, 1746, 10.1214/aoms/1177697386

King, 2004, Enhancing the validity and cross-cultural comparability of measurement in survey research, Am. Polit. Sci. Rev., 98, 191, 10.1017/S000305540400108X

Koheler, 1996, The base rate fallacy reconsidered: descriptive, normative, and methodological challenges, Behav. Brain Sci., 19, 1, 10.1017/S0140525X00041157

Lichtenstein, 1981, Technical Report

Lindley, 1983, Reconciliation of probability distributions, Ops Res., 31, 866, 10.1287/opre.31.5.866

Lindley, 1979, On the reconciliation of probability assessments (with discussion), J. R. Statist. Soc. A, 142, 146, 10.2307/2345078

Melnick, 2008

Morris, 1974, Decision analysis expert use, Mangmnt Sci., 20, 1233, 10.1287/mnsc.20.9.1233

O’Hagan, 1998, Eliciting expert beliefs in substantial practical applications, Statistician, 47, 21

O’Hagan, 2006, Uncertain Judgments: Eliciting Experts Probabilities, 10.1002/0470033312

Sackman, 1974, Delphi Critique; Expert Opinion, Forecasting, and Group Process

Seaver, 1973, Eliciting subjective probability distributions on continuous variable, Organiznl Behav. Hum. Perform., 1, 379

Tversky, 1971, Belief in the law of small numbers, Psychol. Bull., 76, 105, 10.1037/h0031322

Tversky, 1974, Judgments under uncertainty: heuristics and biases, Science, 185, 1124, 10.1126/science.185.4157.1124

Tversky, 1990, Extensional vs. intuitive reasoning: the conjunction fallacy in probability judgment, Psychol. Rev., 90, 293, 10.1037/0033-295X.90.4.293

Wright, 1989, Effects of situation familiarity and financial incentives on use of the anchoring and adjustment heuristic for probability assessment, Hum. Decsn Process., 44, 68, 10.1016/0749-5978(89)90035-6

Yuen, 1983, Deep Space Telecommunication Systems Engineering, 10.1007/978-1-4757-4923-6

Thông tin
Thông tin xuất bản

Journal of the Royal Statistical Society. Series A: Statistics in Society

Tập 177 Số 2

475-497

Thông tin tác giả