Extensions of net reclassification improvement calculations to measure usefulness of new biomarkers

Statistics in Medicine - Tập 30 Số 1 - Trang 11-21 - 2011
Michael Pencina1,2, Ralph B. D’Agostino3, Ewout W. Steyerberg4
1Department of Biostatistics, Boston University, 801 Massachusetts Ave, Boston, MA 02118, U.S.A.
2Harvard Clinical Research Institute, 930 Commonwealth Ave, Boston, MA 02215, U.S.A.
3Department of Mathematics and Statistics, Boston University, 111 Cummington Street, Boston, MA 02215, U.S.A.
4Erasmus MC, Public Health, P.O. Box 1738, Rotterdam, 3000 DR, The Netherlands

Tóm tắt

AbstractAppropriate quantification of added usefulness offered by new markers included in risk prediction algorithms is a problem of active research and debate. Standard methods, including statistical significance and c statistic are useful but not sufficient. Net reclassification improvement (NRI) offers a simple intuitive way of quantifying improvement offered by new markers and has been gaining popularity among researchers. However, several aspects of the NRI have not been studied in sufficient detail.In this paper we propose a prospective formulation for the NRI which offers immediate application to survival and competing risk data as well as allows for easy weighting with observed or perceived costs. We address the issue of the number and choice of categories and their impact on NRI. We contrast category‐based NRI with one which is category‐free and conclude that NRIs cannot be compared across studies unless they are defined in the same manner. We discuss the impact of differing event rates when models are applied to different samples or definitions of events and durations of follow‐up vary between studies. We also show how NRI can be applied to case–control data. The concepts presented in the paper are illustrated in a Framingham Heart Study example.In conclusion, NRI can be readily calculated for survival, competing risk, and case–control data, is more objective and comparable across studies using the category‐free version, and can include relative costs for classifications. We recommend that researchers clearly define and justify the choices they make when choosing NRI for their application. Copyright © 2010 John Wiley & Sons, Ltd.

Từ khóa


Tài liệu tham khảo

10.1161/CIRCULATIONAHA.107.699579

10.1161/01.CIR.97.18.1837

10.1161/01.STR.25.1.40

10.1016/S0140-6736(09)60443-8

10.1093/jnci/81.24.1879

10.7326/0003-4819-148-2-200801150-00005

10.1056/NEJMoa0804742

10.1161/CIRCULATIONAHA.106.672402

10.1002/sim.2929

10.1002/sim.2991

UnoH TianL CaiT KohaneIS WeiLJ.Comparing risk scoring systems beyond the ROC paradigm in survival analysis. Harvard University Biostatistics Working Paper Series 2009; paper 107; accessed online on November 26 2009.

10.1002/sim.3106

10.1001/jama.298.7.776

Cook NR, 2009, Advances in measuring the effect of individual predictors of cardiovascular risk: the role of reclassification measures, Annals of Internal Medicine, 150, 795, 10.7326/0003-4819-150-11-200906020-00007

Steyerberg EW, 2010, Reclassification calculations with incomplete follow‐up, Annals of Internal Medicine, 152, 195, 10.7326/0003-4819-152-3-201002020-00019

10.1161/CIRCULATIONAHA.108.816694

10.1002/sim.2995

10.1002/sim.3087

10.1080/01621459.1993.10476289

10.1126/science.ns-4.93.453-a

10.1177/0272989X06295361

10.1161/CIRCULATIONAHA.108.814251

HarrellFE.ImproveProb() routine in R statistical software. Accessed November 26 2009.

10.1002/1097-0142(1950)3:1<32::AID-CNCR2820030106>3.0.CO;2-3

10.1002/sim.2300

10.1002/sim.2987

10.1097/EDE.0b013e3181c30fb2

10.1093/biomet/asp040

10.1080/01621459.1996.10476660

10.1002/sim.1802

10.1002/sim.4780141909

Thông tin
Thông tin xuất bản

Statistics in Medicine

Tập 30 Số 1

11-21

Thông tin tác giả