An Approach to Perform Expert Elicitation for Engineering Design Risk Analysis: Methodology and Experimental Results
Tóm tắt
Expert elicitation is increasingly applied to different research areas. Multiple approaches have been implemented, but the development of methods to quantify experts' biases and calibration represents a challenge. As a result, the integration of multiple and often conflicting opinions can be demanding, owing to the complexity of properly weighting experts' contributions. We propose an approach to address this problem when probability densities for seed calibration variables are not available. The methodology generates an expert score that is employed to aggregate multiple-expert assessments. The approach has been experimentally applied to engineering design risk analysis. Results indicate that the approach improves the quality of the estimations. The weighted aggregations of experts' estimates based on the experts' scores achieve better results than the corresponding aggregations based on experts' opinions equally weighted.
Từ khóa
Tài liệu tham khảo
Carlson, 1993, In defense of a constitutional theory of experts, Schol. Wrks, 1, 453
Clemen, 1999, Combining probability distributions from experts in risk analysis, Risk Anal., 19, 187, 10.1111/j.1539-6924.1999.tb00399.x
Cooke, 1991
Cooke, 2008, On the performance of social network and likelihood-based expert weighting schemes, Reliab. Engng Syst. Safty, 93, 745, 10.1016/j.ress.2007.03.017
Fiske, 1997, The conjunction fallacy: the case for the existence of competing heuristic strategies, Br. J. Psychol., 88, 1, 10.1111/j.2044-8295.1997.tb02617.x
Garthwaite, 2005, Statistical methods for eliciting probability distributions, J. Am. Statist. Ass., 470, 680, 10.1198/016214505000000105
Jacobs, 1995, Methods for combining experts' probability assessment, Neur. Computn, 7, 867, 10.1162/neco.1995.7.5.867
Jacobson, 1969, The maximum variance of restricted unimodal distributions, Ann. Math. Statist., 5, 1746, 10.1214/aoms/1177697386
King, 2004, Enhancing the validity and cross-cultural comparability of measurement in survey research, Am. Polit. Sci. Rev., 98, 191, 10.1017/S000305540400108X
Koheler, 1996, The base rate fallacy reconsidered: descriptive, normative, and methodological challenges, Behav. Brain Sci., 19, 1, 10.1017/S0140525X00041157
Lichtenstein, 1981, Technical Report
Lindley, 1983, Reconciliation of probability distributions, Ops Res., 31, 866, 10.1287/opre.31.5.866
Lindley, 1979, On the reconciliation of probability assessments (with discussion), J. R. Statist. Soc. A, 142, 146, 10.2307/2345078
Melnick, 2008
O’Hagan, 1998, Eliciting expert beliefs in substantial practical applications, Statistician, 47, 21
Sackman, 1974, Delphi Critique; Expert Opinion, Forecasting, and Group Process
Seaver, 1973, Eliciting subjective probability distributions on continuous variable, Organiznl Behav. Hum. Perform., 1, 379
Tversky, 1974, Judgments under uncertainty: heuristics and biases, Science, 185, 1124, 10.1126/science.185.4157.1124
Tversky, 1990, Extensional vs. intuitive reasoning: the conjunction fallacy in probability judgment, Psychol. Rev., 90, 293, 10.1037/0033-295X.90.4.293
Wright, 1989, Effects of situation familiarity and financial incentives on use of the anchoring and adjustment heuristic for probability assessment, Hum. Decsn Process., 44, 68, 10.1016/0749-5978(89)90035-6