Fifty Years of Classification and Regression Trees

International Statistical Review - Tập 82 Số 3 - Trang 329-348 - 2014
Wei‐Yin Loh1
1Department of statistics, University of Wisconsin, Madison, WI 53706, USA

Tóm tắt

AbstractFifty years have passed since the publication of the first regression tree algorithm. New techniques have added capabilities that far surpass those of the early methods. Modern classification trees can partition the data with linear splits on subsets of variables and fit nearest neighbor, kernel density, and other models in the partitions. Regression trees can fit almost every kind of traditional statistical model, including least‐squares, quantile, logistic, Poisson, and proportional hazards models, as well as models for longitudinal and multiresponse data. Greater availability and affordability of software (much of which is free) have played a significant role in helping the techniques gain acceptance and popularity in the broader scientific community. This article surveys the developments and briefly reviews the key ideas behind some of the major algorithms.

Từ khóa


Tài liệu tham khảo

10.1214/aos/1176344247

10.1002/sim.1266

10.1002/bimj.4710360106

10.1080/03610929408831245

10.1080/03610929608831696

10.1016/0167-9473(95)00023-2

10.2307/2533948

10.2307/2533389

10.1002/widm.51

Alexander W. P., 1996, Treed regression, J. Comput. Graph. Stat., 5, 156, 10.1080/10618600.1996.10474702

10.1007/BF00985256

Bishop Y. M. M., 1975, Discrete Multivariate Analysis

Breiman L., 1984, Classification and Regression Trees

10.1007/BF00058655

10.1023/A:1010933404324

10.1016/0167-8655(96)00033-5

10.1093/biomet/66.3.429

10.1016/S0167-9473(01)00098-6

10.1198/106186004X13064

Chaudhuri P., 1994, Piecewise‐polynomial regression trees, Stat. Sinica, 4, 143

Chaudhuri P., 1995, Generalized regression trees, Stat. Sinica, 5, 641

Chaudhuri P., 2002, Nonparametric estimation of conditional quantiles using quantile regression trees, Bernoulli, 8, 561

10.1080/01621459.1998.10473750

10.1023/A:1013916107446

10.1109/TSMCA.2008.918598

10.1016/j.csda.2004.06.011

10.1109/34.88569

10.1016/0167-9473(91)90103-9

10.1002/sim.1106

10.1016/0169-2607(88)90004-1

10.1016/0167-9473(86)90033-2

10.1016/j.stamet.2007.09.003

10.1214/lnms/1215452213

10.1002/sim.4780080806

10.1890/0012-9658(2002)083[1105:MRTANT]2.0.CO;2

10.1890/0012-9658(2000)081[3178:CARTAP]2.0.CO;2

10.1093/biomet/85.2.363

10.1007/3-540-45014-9_1

Ding Y., 2010, An investigation of missing data methods for classification trees applied to binary response data, J. Mach. Learn. Res., 11, 131

10.1145/775047.775117

10.1057/jors.1973.81

10.1198/jcgs.2010.06089

10.1007/BF02295641

10.1086/268019

10.1198/106186005X37210

10.1198/016214506000000438

10.1111/j.1469-1809.1936.tb02137.x

10.1006/jcss.1997.1504

10.1214/aos/1016218223

10.1023/B:MACH.0000027782.67192.13

10.1016/S0167-9473(03)00089-6

Garg L., 2011, Phase‐type survival trees and mixed distribution survival trees for clustering patients’ hospital length of stay, Informatica, 22, 57, 10.15388/Informatica.2011.314

10.1109/ICMLA.2008.154

Gehrke J., 2009, Encyclopedia of Database Systems, 2469, 10.1007/978-0-387-39940-9_555

Gordon L., 1985, Tree‐structured survival analysis, Cancer Treat. Rep., 69, 1065

10.1016/j.csda.2007.03.014

10.1007/s10994-006-6226-1

10.1145/1656274.1656278

10.1080/00949650410001729472

10.1016/j.healthpol.2004.05.002

10.1109/T-C.1969.222728

Hoaglin D. C., 1995, A critical look at some analyses of major league baseball salaries, Am. Stat., 49, 277, 10.1080/00031305.1995.10476165

10.1093/biostatistics/kxj011

10.1198/106186006X133933

10.1016/j.csda.2004.06.019

10.1002/sim.1593

10.1016/j.spl.2006.08.014

10.1198/016214504000000638

10.1214/08-AOAS169

10.1177/0272989X04271048

10.2307/2986296

10.1198/016214501753168271

10.1198/1061860032049

10.1080/07408170600897502

Kononenko I., 1995, 14th International Joint Conference on Artificial Intelligence

Kooperberg C., 1997, Polychotomous regression, J. Amer. Statist. Assoc., 92, 117, 10.1080/01621459.1997.10473608

10.2307/2529876

10.1007/s10994-005-0466-3

10.1111/j.0006-341X.2004.00202.x

10.2307/2532300

10.1080/01621459.1993.10476296

10.1016/j.csda.2010.01.027

10.1016/j.csda.2004.07.003

10.1207/S15324796ABM2603_02

Light R. J., 1971, An analysis of variance for categorical data, J. Amer. Statist. Assoc., 66, 534, 10.1080/01621459.1971.10482297

10.1023/A:1007608224229

10.1016/0167-9473(91)90113-G

Loh W.‐Y., 2002, Regression trees with unbiased variable selection and interaction detection, Stat. Sinica, 12, 361

10.1007/978-1-84628-288-1_29

10.1214/074921706000000464

10.1002/9780470061572.eqr492

10.1007/978-3-540-33037-0_18

10.1214/09-AOAS260

10.1002/widm.8

Loh W.‐Y., 2012, Probability approximations and beyond, 133

Loh W.‐Y., 2007, Extrapolation errors in linear model trees, ACM Trans. On Knowledge Disc. From Data, 1

Loh W.‐Y., 1997, Split selection methods for classification trees, Stat. Sinica, 7, 815

10.1080/01621459.1988.10478652

10.1214/12-AOAS596

10.1109/T-C.1973.223603

10.1348/000711010X503129

Messenger R., 1972, A modal search technique for predictive nominal scale multivariate analysis, J. Amer. Statist. Assoc., 67, 768

10.1016/j.jmva.2004.02.003

10.1080/01621459.1963.10500855

10.1023/A:1009744630224

10.1109/TC.1977.1674938

Perlich C., 2004, Tree induction vs. logistic regression: A learning‐curve analysis, J. Mach. Learn. Res., 4, 211

10.2307/2344317

10.1007/s10994-005-1121-8

10.1007/BF00116251

Quinlan J. R., 1992, Proceedings of the 5th Australian Joint Conference on Artificial Intelligence, 343

Quinlan J. R., 1993, C4.5: Programs for Machine Learning

R Core Team, 2014, R: A language and environment for statistical computing

10.1080/00949655.2012.658804

10.1198/106186008X344522

10.2307/2531894

10.1080/01621459.1992.10475220

Sela R. J., 2011, Reemtree: Regression trees with random effects

10.1007/s10994-011-5258-3

10.1016/0031-3203(77)90004-8

10.1016/S0167-9473(03)00064-1

10.1016/S0167-9473(03)00036-7

Smyth P., 1995, Proceedings of the 12th International Conference on Machine Learning, 506

10.1016/j.csda.2006.12.030

10.1186/1471-2105-9-307

10.1186/1471-2105-8-25

StroblC. KopfJ.&ZeileisA.(2010).A new method for detecting differential item functioning in the Rasch model. Tech. Rep. 92 Department of Statistics Ludwig‐Maximilians‐Universitaet Muenchen.

10.1037/a0016973

10.3102/1076998609359791

10.1111/j.0006-341X.2004.00139.x

10.1198/106186004X2165

10.1198/jasa.2011.ap09769

Therneau T. M., 2011, rpart: Recursive partitioning

Torgo L., 1997, Proceedings of the Fourteenth International Conference on Machine Learning, 385

Wang Y., 1996, Induction of model trees for predicting continuous classes

10.1007/BF00993349

10.1109/72.963795

10.1142/S0218001405004125

10.1007/11564096_45

10.2307/1390825

10.1198/106186008X319331

10.1080/01621459.1998.10474100

10.1007/978-1-4419-6824-1

10.4310/SII.2008.v1.n1.a14