Sum of ranking differences for method discrimination and its validation: comparison of ranks with random numbers

Journal of Chemometrics - Tập 25 Số 4 - Trang 151-158 - 2011
Károly Héberger1, K. Kollár‐Hunek2
1Chemical Research Center, Hungarian Academy of Sciences, H‐1025 Budapest, Pusztaszeri út 59/67, Hungary
2Department of Inorganic and Analytical Chemistry Budapest University of Technology and Economics H‐1111 Budapest, Szt. Gellért tér 4. Hungary

Tóm tắt

AbstractThis paper describes the theoretical background, algorithm and validation of a recently developed novel method of ranking based on the sum of ranking differences [TrAC Trends Anal. Chem. 2010; 29: 101–109]. The ranking is intended to compare models, methods, analytical techniques, panel members, etc. and it is entirely general. First, the objects to be ranked are arranged in the rows and the variables (for example model results) in the columns of an input matrix. Then, the results of each model for each object are ranked in the order of increasing magnitude. The difference between the rank of the model results and the rank of the known, reference or standard results is then computed. (If the golden standard ranking is known the rank differences can be completed easily.) In the end, the absolute values of the differences are summed together for all models to be compared. The sum of ranking differences (SRD) arranges the models in a unique and unambiguous way. The closer the SRD value to zero (i.e. the closer the ranking to the golden standard), the better is the model. The proximity of SRD values shows similarity of the models, whereas large variation will imply dissimilarity. Generally, the average can be accepted as the golden standard in the absence of known or reference results, even if bias is also present in the model results in addition to random error. Validation of the SRD method can be carried out by using simulated random numbers for comparison (permutation test). A recursive algorithm calculates the discrete distribution for a small number of objects (n < 14), whereas the normal distribution is used as a reasonable approximation if the number of objects is large. The theoretical distribution is visualized for random numbers and can be used to identify SRD values for models that are far from being random. The ranking and validation procedures are called Sum of Ranking differences (SRD) and Comparison of Ranks by Random Numbers (CRNN), respectively. Copyright © 2010 John Wiley & Sons, Ltd.

Từ khóa


Tài liệu tham khảo

10.1016/j.trac.2009.09.009

10.1016/0169-7439(94)00035-2

GeladiP.The regression model comparison plot (REMOCOP) Proc. Spectroscopy Across the Spectrum IV Norwich UK 11‐14 July 1994; In Frontiers in Analytical Spectroscopy Andrews D Davies A (eds). The Royal Society of Chemistry.

10.1016/S0003-2670(97)00290-0

10.1016/j.chemolab.2005.11.001

10.1002/cem.1025

10.1021/ci049827t

10.1016/j.chroma.2008.05.019

10.1021/ci034001x

10.1002/sim.3124

10.1016/j.matdes.2009.05.016

HongY KwongS ChangY RenQ.Unsupervised feature selection using clustering ensembles and population based incremental learning algorithmPattern Recogn.2008;41:2742–2756.

10.1016/j.neucom.2007.04.012

10.1016/j.jimonfin.2008.06.005

10.1016/j.ipm.2005.03.024

10.1016/j.tourman.2007.04.001

10.1016/j.jpba.2010.02.004

Conover WJ, 1980, Practical Nonparametric Statistics, 213

10.1016/j.chemolab.2006.03.001

10.1002/cem.1135

10.1016/j.chemolab.2004.01.008

10.1016/j.chroma.2010.02.037

Thông tin
Thông tin xuất bản

Journal of Chemometrics

Tập 25 Số 4

151-158

Thông tin tác giả