Regression Shrinkage and Selection via The Lasso: A Retrospective

Robert Tibshirani1
1Stanford University - USA > > > >

Tóm tắt

Summary In the paper I give a brief review of the basic idea and some history and then discuss some developments since the original paper on regression shrinkage and selection via the lasso.

Từ khóa


Tài liệu tham khảo

Barlow, 1972, Statistical Inference under Order Restrictions; the Theory and Applications of Isotonic Regression

Breiman, 1995, Better subset selection using the non-negative garotte, Technometrics, 37, 738, 10.1080/00401706.1995.10484371

Candes, 2006, Compressive sampling, Proc. Int. Congr. Mathematicians, Madrid.

Candes, 2007, The dantzig selector statistical estimation when p is much larger than n, Ann. Statist., 35, 2313

Candès, 2009, The power of convex relaxation: near-optimal matrix completion

Chen, 1998, Atomic decomposition by basis pursuit, SIAM J. Scient. Comput., 43, 33, 10.1137/S1064827596304010

Donoho, 2004, Technical Report

Efron, 2002, Technical Report.

Frank, 1993, A statistical view of some chemometrics regression tools (with discussion), Technometrics, 35, 109, 10.1080/00401706.1993.10485033

Friedman, 2007, Pathwise coordinate optimization, Ann. Appl. Statist., 2, 302

Friedman, 2010, Regularization paths for generalized linear models via coordinate descent, J. Statist. Sofwr., 33

Fu, 1998, Penalized regressions: the bridge vs. the lasso, J. Computnl Graph. Statist., 7, 397

Genkin, 2007, Large-scale Bayesian logistic regression for text categorization, Technometrics, 49, 291, 10.1198/004017007000000245

Hastie, 2008, The Elements of Statistical Learning; Data Mining, Inference and Prediction

Jolliffe, 2003, A modified principal.component technique based on the lasso, J. Computnl Graph. Statist., 12, 531, 10.1198/1061860032148

Mazumder, 2010, Spectral regularization algorithms for learning large incomplete matrices, J.Mach. Learn. Res., 11, 2287

Osborne, 2000, On the lasso and its dual, J. Computnl Graph. Statist., 9, 319

Tibshirani, 2011, Nearly isotonic regression, Technometrics, 53, 54, 10.1198/TECH.2010.10111

Tibshirani, 2005, Sparsity and smoothness via the fused lasso, J. R. Statist. Soc. B, 67, 91, 10.1111/j.1467-9868.2005.00490.x

Tibshirani, 2011, The solution path of the generalized lasso, Ann. Statist., 10.1214/11-AOS878

Witten, 2009, A penalized matrix decomposition, with applications to sparse principal components and canonical correlation analysis, Biometrika, 10, 515

Wu, 2008, Coordinate descent procedures for lasso penalized regression, Ann. Appl. Statist., 2, 224, 10.1214/07-AOAS147

Yuan, 2007, Model selection and estimation in regression with grouped variables, J. R. Statist. Soc. B, 68, 49, 10.1111/j.1467-9868.2005.00532.x

Yuan, 2007, Model selection and estimation in the Gaussian graphical model, Biometrika, 94, 19, 10.1093/biomet/asm018

Zou, 2006, The adaptive lasso and its oracle properties, J. Am. Statist. Ass., 101, 1418, 10.1198/016214506000000735

Zou, 2005, Regularization and variable selection via the elastic net, J. R. Statist. Soc. B, 67, 301, 10.1111/j.1467-9868.2005.00503.x

Allen, 2010, Transposable regularized covariance models with an application to missing data imputation, Ann. Appl. Statist., 4, 764, 10.1214/09-AOAS314

Benjamini, 1995, Controlling the false discovery rate: a practical and powerful approach to multiple testing, J. R. Statist. Soc. B, 57, 289

Bernardo, 1994, Bayesian Theory, 10.1002/9780470316870

Bickel, 2009, Simultaneous analysis of Lasso and Dantzig selector, Ann. Statist., 37, 1705, 10.1214/08-AOS620

Bondell, 2010, Joint variable selection of fixed and random effects in linear mixed-effects models, Biometrics, 66, 10.1111/j.1541-0420.2010.01391.x

Bühlmann, 2011, Statistics for High-dimensional Data: Methods, Theory and Applications, 10.1007/978-3-642-20192-9

Bunea, 2007, Sparsity oracle inequalities for the Lasso, Electron. J. Statist., 1, 169, 10.1214/07-EJS008

Candès, 2008, Enhancing sparsity by reweighted l1 minimization, J. Four. Anal. Appl., 14, 877, 10.1007/s00041-008-9045-x

Clyde, 2004, Model uncertainty, Statist. Sci., 19, 81, 10.1214/088342304000000035

Donoho, 2006, Stable recovery of sparse overcomplete representations in the presence of noise, IEEE Trans. Inform. Theor., 52, 6, 10.1109/TIT.2005.860430

Donoho, 1994, Ideal spatial adaptation by wavelet shrinkage, Biometrika, 81, 425, 10.1093/biomet/81.3.425

Efron, 1979, Bootstrap methods: another look at the Jackknife, Ann. Statist., 7, 1, 10.1214/aos/1176344552

Fan, 2005, Variable selection via nonconcave penalized likelihood and its oracle properties, J. Am. Statist. Ass., 96, 1348, 10.1198/016214501753382273

Fu, 1998, Penalized regressions: the Bridge versus the Lasso, J. Computnl Graph. Statist., 7, 397

van de Geer, 2007, Proc. Jt Statist. Meet., 140

van de Geer, 2008, High-dimensional generalized linear models and the Lasso, Ann. Statist., 36, 614, 10.1214/009053607000000929

van de Geer, 2009, On the conditions used to prove oracle results for the Lasso, Electron. J. Statist., 3, 1360, 10.1214/09-EJS506

George, 1993, Variable selection via gibbs sampling, J. Am. Statist. Ass., 88, 884, 10.1080/01621459.1993.10476353

Green, 1995, Reversible jump Markov chain Monte Carlo computation and Bayesian model determination, Biometrika, 82, 711, 10.1093/biomet/82.4.711

Greenshtein, 2004, Persistence in high-dimensional predictor selection and the virtue of over-parametrization, Bernoulli, 10, 971, 10.3150/bj/1106314846

Holmes, 2007, Bayesian Statistics 8

Khalili, 2007, Variable selection in finite mixture of regression models, J. Am. Statist. Ass., 102, 1025, 10.1198/016214507000000590

Mazumder, 2010, Sparsenet: coordinate descent with non-convex penalties

Meier, 2008, The group lasso for logistic regression, J. R. Statist. Soc. B, 70, 53, 10.1111/j.1467-9868.2007.00627.x

Meinshausen, 2007, Relaxed Lasso, Computnl Statist. Data Anal., 52, 374, 10.1016/j.csda.2006.12.019

Meinshausen, 2006, High-dimensional graphs and variable selection with the lasso, Ann. Statist., 34, 1436, 10.1214/009053606000000281

Meinshausen, 2010, Stability selection (with discussion), J. R. Statist. Soc. B, 72, 417, 10.1111/j.1467-9868.2010.00740.x

Meinshausen, 2009, P-values for high-dimensional regression, J. Am. Statist. Ass., 104, 1671, 10.1198/jasa.2009.tm08647

Ming, 2006, Model selection and estimation in regression with grouped variables, J. R. Statist. Soc. B, 68, 49, 10.1111/j.1467-9868.2005.00532.x

Park, 2008, The Bayesian Lasso, J. Am. Statist. Ass., 103, 681, 10.1198/016214508000000337

Sardy, 2000, Block coordinate relaxation methods for nonparametric wavelet denoising, J. Computnl Graph. Statist., 9, 361

Sardy, 2004, On the statistical analysis of smoothing by maximizing dirty Markov random field posterior distributions, J. Am. Statist. Ass., 99, 191, 10.1198/016214504000000188

Schelldorfer, 2011, Estimation for high-dimensional linear mixed-effects models using l1-penalization, Scand. J. Statist., 10.1111/j.1467-9469.2011.00740.x

Städler, 2011, Missing values: sparse inverse covariance estimation and an extension to sparse regression, Statist. Comput.

Städler, 2010, l1-penalization for mixture regression models (with discussion), Test, 19, 209, 10.1007/s11749-010-0197-z

Tibshirani, 1996, Regression shrinkage and selection via the lasso, J. R. Statist. Soc. B, 58, 267

Tseng, 2001, Convergence of a block coordinate descent method for nonsmooth separable minimization, J. Optimzn Theor. Appl., 109, 475, 10.1023/A:1017501703105

Tseng, 2009, A coordinate gradient descent method for nonsmooth separable minimization, Math. Programing B, 117, 387, 10.1007/s10107-007-0170-0

Witten, 2009, Covariance-regularized regression and classification for high dimensional problems, J. R. Statist. Soc. B, 71, 615, 10.1111/j.1467-9868.2009.00699.x

Yuan, 2006, Model selection and estimation in regression with grouped variables, J. R. Statist. Soc. B, 68, 49, 10.1111/j.1467-9868.2005.00532.x

Zhang, 2010, Nearly unbiased variable selection under minimax concave penalty, Ann. Statist., 38, 894, 10.1214/09-AOS729

Zhao, 2006, On model selection consistency of Lasso, J. Mach. Learn. Res., 7, 2541

Zou, 2006, The adaptive Lasso and its oracle properties, J. Am. Statist. Ass., 101, 1418, 10.1198/016214506000000735

Zou, 2005, Regularization and variable selection via the elastic net, J. R. Statist. Soc. B, 67, 301, 10.1111/j.1467-9868.2005.00503.x

Zou, 2008, One-step sparse estimates in nonconcave penalized likelihood models, Ann. Statist., 36, 1509