Regularization and Variable Selection Via the Elastic Net

Journal of the Royal Statistical Society. Series B: Statistical Methodology - Tập 67 Số 2 - Trang 301-320 - 2005
Hui Zou1, Trevor Hastie1
1Stanford University USA

Tóm tắt

SummaryWe propose the elastic net, a new regularization and variable selection method. Real world data and a simulation study show that the elastic net often outperforms the lasso, while enjoying a similar sparsity of representation. In addition, the elastic net encourages a grouping effect, where strongly correlated predictors tend to be in or out of the model together. The elastic net is particularly useful when the number of predictors (p) is much bigger than the number of observations (n). By contrast, the lasso is not a very satisfactory variable selection method in the p≫n case. An algorithm called LARS-EN is proposed for computing elastic net regularization paths efficiently, much like algorithm LARS does for the lasso.

Từ khóa


Tài liệu tham khảo

Breiman, 1996, Heuristics of instability and stabilization in model selection, Ann. Statist., 24, 2350, 10.1214/aos/1032181158

Dettling, 2004, Finding predictive gene groups from microarray data, J. Multiv. Anal., 90, 106, 10.1016/j.jmva.2004.02.012

Díaz-Uriarte, 2003, Tech-nical Report

Donoho, 1995, Wavelet shrinkage: asymptopia (with discussion)?, J. R. Statist. Soc., 57, 301

Efron, 2004, Least angle regression, Ann. Statist., 32, 407, 10.1214/009053604000000067

Fan, 2001, Variable selection via nonconcave penalized likelihood and its oracle properties, J. Am. Statist. Ass., 96, 1348, 10.1198/016214501753382273

Frank, 1993, A statistical view of some chemometrics regression tools, Technometrics, 35, 109, 10.1080/00401706.1993.10485033

Friedman, 1989, Regularized discriminant analysis, J. Am. Statist. Ass., 84, 249, 10.1080/01621459.1989.10478752

Friedman, 2004, Discussion of boosting papers, Ann. Statist., 32, 102

Fu, 1998, Penalized regression: the bridge versus the lasso, J. Computnl Graph. Statist., 7, 397

Golub, 1983, Matrix Computations

Golub, 1999, Molecular classification of cancer: class discovery and class prediction by gene expression monitoring, Science, 286, 513, 10.1126/science.286.5439.531

Guyon, 2002, Gene selection for cancer classification using support vector machines, Mach. Learn., 46, 389, 10.1023/A:1012487302797

Hastie, 2003, Supervised harvesting of expression trees, Genome Biol., 2, 0003.1

Hastie, 2000, ‘Gene shaving’ as a method for identifying distinct sets of genes with similar expression patterns, Genome Biol., 1, 1, 10.1186/gb-2000-1-2-research0003

Hastie, 2001, The Elements of Statistical Learning; Data Mining, Inference and Prediction

Hoerl, 1988, Encyclopedia of Statistical Sciences, 129

Rosset, 2004, Boosting as a regularized path to a maximum margin classifier, J. Mach. Learn. Res., 5, 941

Segal, 2003, Regression approach for microarray data analysis, J. Computnl Biol., 10, 961, 10.1089/106652703322756177

Stamey, 1989, Prostate specific antigen in the diagnosis and treatment of adenocarcinoma of the prostate ii: radical prostatectomy treated patients, J. Urol., 16, 1076, 10.1016/S0022-5347(17)41175-X

Tibshirani, 1996, Regression shrinkage and selection via the lasso, J. R. Statist. Soc., 58, 267

Tibshirani, 2002, Diagnosis of multiple cancer types by shrunken centroids of gene expression, Proc. Natn. Acad. Sci. USA, 99, 6567, 10.1073/pnas.082099299

Tusher, 2001, Significance analysis of microarrays applied to transcriptional responses to ionizing radiation, Proc. Natn. Acad. Sci. USA, 98, 5116, 10.1073/pnas.091062498

West, 2001, Predicting the clinical status of human breast cancer using gene expression profiles, Proc. Natn. Acad. Sci. USA, 98, 11462, 10.1073/pnas.201162998

Zhang, 2004, Statistical behavior and consistency of classification methods based on convex risk minimization, Ann. Statist., 32, 469, 10.1214/aos/1079120130

Zhu, 2004, Classification of gene microarrays by penalized logistic regression, Biostatistics, 5, 427, 10.1093/biostatistics/kxg046

Thông tin
Thông tin xuất bản

Journal of the Royal Statistical Society. Series B: Statistical Methodology

Tập 67 Số 2

301-320

Thông tin tác giả