Sparse Partial Least Squares Regression for Simultaneous Dimension Reduction and Variable Selection

Journal of the Royal Statistical Society. Series B: Statistical Methodology - Tập 72 Số 1 - Trang 3-25 - 2010
Hyonho Chun1, Sündüz Keleş1
1University of Wisconsin, Madison, USA

Tóm tắt

SummaryPartial least squares regression has been an alternative to ordinary least squares for handling multicollinearity in several areas of scientific research since the 1960s. It has recently gained much attention in the analysis of high dimensional genomic data. We show that known asymptotic consistency of the partial least squares estimator for a univariate response does not hold with the very large p and small n paradigm. We derive a similar result for a multivariate response regression with partial least squares. We then propose a sparse partial least squares formulation which aims simultaneously to achieve good predictive performance and variable selection by producing sparse linear combinations of the original predictors. We provide an efficient implementation of sparse partial least squares regression and compare it with well-known variable selection and dimension reduction approaches via simulation experiments. We illustrate the practical utility of sparse partial least squares regression in a joint analysis of gene expression and genomewide binding data.

Từ khóa


Tài liệu tham khảo

Abramovich, 2006, Adapting to unknown sparsity by controlling the false discovery rate, Ann. Statist., 34, 584, 10.1214/009053606000000074

D’Aspremont, 2007, A direct formulation for sparse pca using semidefinite programming, SIAM Rev., 49, 434, 10.1137/050645506

Bair, 2006, Prediction by supervised principal components, J. Am. Statist. Ass., 101, 119, 10.1198/016214505000000628

Bendel, 1976, A criterion for stepwise regression, Am. Statistn, 30, 85, 10.1080/00031305.1976.10479146

Benjamini, 1995, Controlling the false discovery rate: a practical and powerful approach to multiple testing, J. R. Statist. Soc. B, 57, 289

Boulesteix, 2005, Predicting transcription factor activities from combined analysis of microarray and chip data: a partial least squares approach, Theor. Biol. Med. Modllng, 2

Boulesteix, 2006, Partial least squares: a versatile tool for the analysis of high-dimensional genomic data, Brief. Bioinform., 7, 32, 10.1093/bib/bbl016

Ter Braak, 1998, The objective function of partial least squares regression, J. Chemometr., 12, 41, 10.1002/(SICI)1099-128X(199801/02)12:1<41::AID-CEM500>3.0.CO;2-F

Butler, 2000, The peculiar shrinkage properties of partial least squares regression, J. R. Statist. Soc B, 62, 585, 10.1111/1467-9868.00252

Chun, 2009, Expression quantitative loci mapping with multivariate sparse partial least squares, Genetics, 182, 79, 10.1534/genetics.109.100362

Efron, 2004, Least angle regression, Ann. Statist., 32, 407, 10.1214/009053604000000067

Frank, 1993, A statistical view of some chemometrics regression tools, Technometrics, 35, 109, 10.1080/00401706.1993.10485033

Friedman, 2004, Gradient directed regularization for linear regression and classification

Geman, 1980, A limit theorem for the norm of random matrices, Ann. Probab., 8, 252, 10.1214/aop/1176994775

Golub, 1987, Matrix Computations

Goutis, 1996, Partial least squares algorithm yields shrinkage estimators, Ann. Statist., 24, 816, 10.1214/aos/1032894467

Hastie, 2000, Identifying distinct sets of genes with similar expression patterns via ‘‘gene shaving’’, Genome Biol., 1, 1, 10.1186/gb-2000-1-2-research0003

Helland, 1990, Partial least squares regression and statistical models, Scand. J. Statist., 17, 97

Helland, 2000, Model reduction for prediction in regression models, Scand. J. Statist., 27, 1, 10.1111/1467-9469.00174

Helland, 1994, Comparison of prediction methods when only a few components are relevant, J. Am. Statist. Ass., 89, 583, 10.1080/01621459.1994.10476783

Huang, 2004, Modeling the relationship between lvad support time and gene expression changes in the human heart by penalized partial least squares, Bioinformatics, 20, 888, 10.1093/bioinformatics/btg499

Johnstone, 2004, Sparse principal component analysis. Technical Report

Jolliffe, 2003, A modified principal component technique based on the lasso, J. Computnl Graph. Statist., 12, 531, 10.1198/1061860032148

De Jong, 1993, SIMPLS: an alternative approach to partial least squares regression, Chemometr. Intell. Lab. Syst., 18, 251, 10.1016/0169-7439(93)85002-X

Kosorok, 2007, Marginal asymptotics for the ‘‘large p, small n’’ paradigm: with applications to microarray data, Ann. Statist., 35, 1456, 10.1214/009053606000001433

Krämer, 2007, An overview on the shrinkage properties of partial least squares regression, Computnl Statist., 22, 249, 10.1007/s00180-007-0038-z

Lee, 2002, Transcriptional regulatory networks in saccharomyces cerevisiae, Science, 298, 799, 10.1126/science.1075090

Nadler, 2005, The prediction error in cls and pls: the importance of feature selection prior to multivariate calibration, J. Chemometr., 19, 107, 10.1002/cem.915

Naik, 2000, Partial least squares estimator for single-index models, J. R. Statist. Soc. B, 62, 763, 10.1111/1467-9868.00262

Pratt, 1960, On interchanging limits and integrals, Ann. Math. Statist., 31, 74, 10.1214/aoms/1177705988

Rosipal, 2006, Subspace, Latent Structure and Feature Selection Techniques, 34, 10.1007/11752790_2

Spellman, 1998, Comprehensive identification of cell cycle-regulated genes of the yeast saccharomyces cerevisiae by microarray hybridization, Molec. Biol. Cell, 9, 3273, 10.1091/mbc.9.12.3273

Stoica, 1998, Partial least squares: a first-order analysis, Scand. J. Statist., 25, 17, 10.1111/1467-9469.00085

Tibshirani, 1996, Regression shrinkage and selection via the lasso, J. R. Statist. Soc. B, 58, 267

Wang, 2007, Group scad regression analysis for microarray time course gene expression data, Bioinformatics, 23, 1486, 10.1093/bioinformatics/btm125

Wold, 1966, Estimation of Principal Components and Related Models by Iterative Least Squares

Zou, 2005, Regularization and variable selection via the elastic net, J. R. Statist. Soc. B, 67, 301, 10.1111/j.1467-9868.2005.00503.x

Zou, 2006, Sparse principal component analysis, J. Computnl Graph. Statist., 15, 265, 10.1198/106186006X113430

Thông tin
Thông tin xuất bản

Journal of the Royal Statistical Society. Series B: Statistical Methodology

Tập 72 Số 1

3-25

Thông tin tác giả