Evaluating the added predictive ability of a new marker: From area under the ROC curve to reclassification and beyond

Statistics in Medicine - Tập 27 Số 2 - Trang 157-172 - 2008
Michael Preuß1, Ralph B. D' Agostino1, Ramachandran S. Vasan2
1Department of Mathematics and Statistics, Framingham Heart Study, Boston University, 111 Cummington St., Boston, MA 02215, U.S.A.
2Framingham Heart Study, Boston University School of Medicine, 73 Mount Wayte Avenue, Suite 2, Framingham, MA 01702-5803, U.S.A.

Tóm tắt

AbstractIdentification of key factors associated with the risk of developing cardiovascular disease and quantification of this risk using multivariable prediction algorithms are among the major advances made in preventive cardiology and cardiovascular epidemiology in the 20th century. The ongoing discovery of new risk markers by scientists presents opportunities and challenges for statisticians and clinicians to evaluate these biomarkers and to develop new risk formulations that incorporate them. One of the key questions is how best to assess and quantify the improvement in risk prediction offered by these new models. Demonstration of a statistically significant association of a new biomarker with cardiovascular risk is not enough. Some researchers have advanced that the improvement in the area under the receiver‐operating‐characteristic curve (AUC) should be the main criterion, whereas others argue that better measures of performance of prediction models are needed. In this paper, we address this question by introducing two new measures, one based on integrated sensitivity and specificity and the other on reclassification tables. These new measures offer incremental information over the AUC. We discuss the properties of these new measures and contrast them with the AUC. We also develop simple asymptotic tests of significance. We illustrate the use of these measures with an example from the Framingham Heart Study. We propose that scientists consider these types of measures in addition to the AUC when assessing the performance of newer biomarkers. Copyright © 2007 John Wiley & Sons, Ltd.

Từ khóa


Tài liệu tham khảo

10.1016/0002-9149(76)90061-8

10.1161/01.CIR.83.1.356

10.1161/01.CIR.97.18.1837

10.1016/S0002-8703(00)90236-9

10.1001/jama.285.19.2486

10.1016/S0195-668X(03)00114-3

10.1001/jama.286.2.180

10.1001/jama.291.21.2591

10.1136/jech.57.8.634

10.1001/jama.297.6.611

10.1161/01.ATV.0000251993.20372.40

10.1056/NEJMoa055373

10.1097/01.hjh.0000217845.57466.cc

D'Agostino RB, 1997, Proceedings of the Biometrics Section, 253

10.1016/0022-2496(75)90001-2

10.1148/radiology.143.1.7063747

10.1002/(SICI)1097-0258(19960229)15:4<361::AID-SIM168>3.0.CO;2-4

10.1002/sim.1802

10.1093/aje/kwh101

10.1001/archinte.165.21.2454

10.1056/NEJMp068249

10.7326/0003-4819-145-1-200607040-00128

10.1161/CIRCULATIONAHA.106.672402

PepeMS FengZ HuangY LongtonGM PrenticeR ThompsonIM ZhengY. Integrating the predictiveness of a marker with its performance as a classifier. UW Biostatistics Working Paper Series #289.2006. Available athttp://www.bepress.com/uwbiostat/paper289(accessed 9 March 2007).

10.1016/0030-5073(82)90237-9

Schmid CH, 1998, Encyclopedia of Biostatistics

10.1007/BF02295996

10.1002/1097-0142(1950)3:1<32::AID-CNCR2820030106>3.0.CO;2-3

D'Agostino RB, 1989, Proceedings of the American Statistical Association Sesquicentennial Invited Paper Sessions

10.2307/2531595

10.1109/TAC.1974.1100705

D'Agostino RB, 2004, Handbook of Statistics

Hosmer DW, 1989, Applied Logistic Regression

10.1002/sim.2299

Thông tin
Thông tin xuất bản

Statistics in Medicine

Tập 27 Số 2

157-172

Thông tin tác giả