The Group Lasso for Logistic Regression

Journal of the Royal Statistical Society. Series B: Statistical Methodology - Tập 70 Số 1 - Trang 53-71 - 2008
Lukas Meier1, Sara van de Geer1, Peter Bühlmann1
1Eidgenössische Technische Hochschule, Zürich, Switzerland

Tóm tắt

SummaryThe group lasso is an extension of the lasso to do variable selection on (predefined) groups of variables in linear regression models. The estimates have the attractive property of being invariant under groupwise orthogonal reparameterizations. We extend the group lasso to logistic regression models and present an efficient algorithm, that is especially suitable for high dimensional problems, which can also be applied to generalized linear models to solve the corresponding convex optimization problem. The group lasso estimator for logistic regression is shown to be statistically consistent even if the number of predictors is much larger than sample size but with sparse true underlying structure. We further use a two-stage procedure which aims for sparser models than the group lasso, leading to improved prediction performance for some cases. Moreover, owing to the two-stage nature, the estimates can be constructed to be hierarchical. The methods are used on simulated and real data sets about splice site detection in DNA sequences.

Từ khóa


Tài liệu tham khảo

Antoniadis, 2001, Regularization of wavelet approximations (with discussion), J. Am. Statist. Ass., 96, 939, 10.1198/016214501753208942

Bakin, 1999, Adaptive regression and model selection in data mining problems

Balakrishnan, 2006, Algorithms for sparse linear classifiers in the massive data setting

Bertsekas, 2003, Nonlinear Programming

Burge, 1998, Computational Methods in Molecular Biology, 129, 10.1016/S0167-7306(08)60465-2

Burge, 1997, Prediction of complete gene structures in human genomic DNA, J. Molec. Biol., 268, 78, 10.1006/jmbi.1997.0951

Cai, 2001, Discussion of ‘‘Regularization of wavelet approximations’’ (by A. Antoniadis and J. Fan), J. Am. Statist. Ass., 96, 960

Efron, 2004, Least angle regression, Ann. Statist., 32, 407, 10.1214/009053604000000067

Van De Geer, 2003, Recent Advances and Trends in Nonparametric Statistics, 235, 10.1016/B978-044451378-6/50016-8

Van De Geer, 2007, High-dimensional generalized linear models and the lasso, Ann. Statist.

Genkin, 2007, Large-scale bayesian logistic regression for text categorization, 291

Kim, 2006, Blockwise sparse regression, Statist. Sin., 16, 375

King, 2001, Logistic regression in rare events data, Polit. Anal., 9, 137, 10.1093/oxfordjournals.pan.a004868

Krishnapuram, 2005, Sparse multinomial logistic regression: fast algorithms and generalization bounds, IEEE Trans. Pattn Anal. Mach. Intell., 27, 957, 10.1109/TPAMI.2005.127

Lokhorst, 1999, The lasso and generalised linear models

Meinshausen, 2007, Lasso with relaxation, Computnl Statist. Data Anal., 52, 374, 10.1016/j.csda.2006.12.019

Osborne, 2000, A new approach to variable selection in least squares problems, IMA J. Numer. Anal., 20, 389, 10.1093/imanum/20.3.389

Park, 2006, Regularization path algorithms for detecting gene interactions

Park, 2007, L1-regularization path algorithm for generalized linear models, J. R. Statist. Soc. B, 69, 659

Rosset, 2005, Advances in Neural Information Processing Systems, 1153

Roth, 2004, The generalized lasso, IEEE Trans. Neur. Netwrks, 15, 16

Shevade, 2003, A simple and efficient algorithm for gene selection using sparse logistic regression, Bioinformatics, 19, 2246

Tarigan, 2006, Classifiers of support vector machine type with l1 complexity regularization, Bernoulli, 12, 1045

Tibshirani, 1996, Regression shrinkage and selection via the lasso, J. R. Statist. Soc. B, 58, 267

Tibshirani, 1997, The lasso method for variable selection in the cox model, Statist. Med., 16, 385

Tseng, 2001, Convergence of a block coordinate descent method for nondifferentiable minimization, J. Optimizn Theory Appl., 109, 475

Tseng, 2007, A coordinate gradient descent method for nonsmooth separable minimization

Yeo, 2004, Maximum entropy modeling of short sequence motifs with applications to RNA splicing signals, J. Computnl Biol., 11, 475

Yuan, 2006, Model selection and estimation in regression with grouped variables, J. R. Statist. Soc. B, 68, 49

Zhao, 2007, Stagewise lasso

Thông tin
Thông tin xuất bản

Journal of the Royal Statistical Society. Series B: Statistical Methodology

Tập 70 Số 1

53-71

Thông tin tác giả