Use of Expert Judgement Across NICE Guidance-Making Programmes: A Review of Current Processes and Suitability of Existing Tools to Support the Use of Expert Elicitation
Tóm tắt
This study aimed to review current use of experts within National Institute for Health and Care Excellence (NICE) guidance-making programmes, identify improvements in use of expert judgement, and to assess tools and protocols to support the elicitation of information from experts for use by NICE. The study comprised a review of NICE process guides; semi-structured interviews with individuals representing each NICE guidance-making programme and a comparison of the suitability of currently available tools and protocols for expert elicitation to the requirements of NICE programmes. Information elicited from experts and the way in which it is used varies across NICE guidance-making programmes. Experts’ involvement can be as intensive as being a member of a committee and thus having direct influence on recommendations or limited one-off consultations on specific parameters. We identified 16 tools for expert elicitation that were potentially relevant. None fully met the requirements of NICE, although an existing tool could be potentially adapted. Ongoing research to develop a reference protocol for expert elicitation in healthcare decision making may be of use in future. NICE uses expert judgement across all its guidance-making programmes, but its uses vary considerably. There is no currently available tool for expert elicitation suitable for use by NICE. However, adaptation of an existing tool or ongoing research in the area could address this in the future.
Tài liệu tham khảo
Iglesias CP, Thompson A, Rogowski WH, Payne K. Reporting guidelines for the use of expert judgement in model-based economic evaluations. PharmacoEconomics. 2016;34(11):1161–72.
Evans D. Hierarchy of evidence: a framework for ranking evidence evaluating healthcare interventions. J Clin Nurs. 2003;12(1):77–84.
O’Hagan A, Oakley J. SHELF: the Sheffield Elicitation Framework v3.0. 2016. Available from: http://www.tonyohagan.co.uk/shelf/.
O’Hagan A, Buck CE, Daneshkhah AJ, Eiser R, Garthwaite PH, Jenkinson DJ, et al. Uncertain judgements: eliciting experts’ probabilities. New York: Wiley; 2006.
Garthwaite PH, Kadane JB, O’Hagan A. Statistical methods for eliciting probability distributions. J Am Stat Assoc. 2005;100(470):680–701.
Cooke RM. Experts in uncertainty: opinion and subjective probability in science. Oxford: Oxford University Press; 1991.
Devilee J, Knol A. Software to support expert elicitation: an exploratory study of existing software packages. RIVM Letter report 630003001/2011. National Institute for Public Health and the Environment of the Netherlands. 2012. Available from: https://www.rivm.nl/bibliotheek/rapporten/630003001.pdf.
Sharma T, Choudhury M, Kaur B, Naidoo B, Garner S, Littlejohns P, et al. Evidence informed decision making: the use of “colloquial evidence” at NICE. Int J Technol Assess Health Care. 2015;31(3):138–46.
National Institute for Health and Care Excellence. NICE Guidance. 2017. [cited 17th July 2017]. Available from: https://www.nice.org.uk/about/what-we-do/our-programmes/nice-guidance.
Speight P, Palmer S, Moles D, Downer M, Smith D. The cost-effectiveness of screening for oral cancer in primary care. Health Technol Assess. 2006;10(14):1–144, iii–iv.
Ababei D. EXCALIBUR. Available from: http://www.lighttwist.net/wp/excalibur.
Bigagli L. UncertWeb. 2013. Available from: https://wiki.aston.ac.uk/foswiki/bin/view/UncertWeb/UncertWebSoftware.
Fenton N, Neil M. AgenaRisk 7.0. 2016. Available from: http://www.agenarisk.com/.
Fisher R, O’Leary R. ElicitN. 2011. Available from: http://www.downloadcollection.com/elicitn.htm.
James A, Low Choy S, Mengerson K. Elicitor (v. 2010). 2010.
Middleton M. RiskSim. 2013. Available from: http://www.decisiontoolworks.com/.
Morris D, Oakley J, Crowe J. MATCH. 2014. Available from: http://optics.eee.nottingham.ac.uk/match/uncertainty.php.
Palisade. @Risk 7.5. [program] UK; 2016. Available from: http://www.palisade.com/.
RAMAS. RAMAS Constuctor. 2016. Available from: http://www.ramas.com/constructor-1.
RAMAS. RAMAS Risk Calc. 2016. Available from: http://www.ramas.com/riskcalc.
RAND. ExpertLens. 2011. Available from: http://www.rand.org/pubs/tools/expertlens.html.
Technical University of Delft. Unicorn. 2007. Available from: http://www.lighttwist.net/wp/unicorn-download.
Truong P, Heuvelink G, Gosling J. VARIOGRAM. 2013. Available from: http://www.variogramelicitation.org/.
Open University. PEGS (Prior elicitation of graphical software). Available from: http://statistics.open.ac.uk/elicitation.
Vose Software. ModelRisk 3.0. 2008. Available from: http://vosesoftware.com/knowledgebase/newsletters/newsletters/06/index.html.
Durbach I, Merven B, McCall B. Expert elicitation of autocorrelated time series with application to e3 (energy-environment-economic) forecasting models. Environ Model Softw. 2017;88:93–105.
Expert Elicitation Task Force. Expert Elicitation Task Force White Paper. 2009. Available from: https://yosemite.epa.gov/sab/sabproduct.nsf/fedrgstr_activites/F4ACE05D0975F8C68525719200598BC7/$File/Expert_Elicitation_White_Paper-January_06_2009.pdf.
Hoffmann S, Fischbeck P, Krupnick A, McWilliams M. Elicitation from large, heterogeneous expert panels: using multiple uncertainty measures to characterize information quality for decision analysis. Decis Anal. 2007;4(2):91–109.
Sperber D, Mortimer D, Lorgelly P, Berlowitz D. An expert on every street corner? Methods for eliciting distributions in geographically dispersed opinion pools. Value Health. 2013;16(2):434–7.
Medical Research Council. Successfully funded proposals. 2017. [cited 24th July 2017]. Available from: https://www.mrc.ac.uk/funding/how-we-fund-research/highlight-notices/methodology-for-eliciting-expert-opinion/successfully-funded-proposals/.
Grigore B, Peters J, Hyde C, Stein K. EXPLICIT: a feasibility study of remote expert elicitation in health technology assessment. BMC Med Inform Decis Mak. 2017;17:131.
O’Hagan A, Oakley J. SHELF: the Sheffield Elicitation Framework v3.0. [program]. 2016. http://www.tonyohagan.co.uk/shelf/.