OPIT: A Simple but Effective Method for Sparse Subspace Tracking in High-Dimension and Low-Sample-Size Context

IEEE Transactions on Signal Processing - Tập 72 - Trang 521-534 - 2024
Thanh Trung Le1,2, Karim Abed-Meraim2,3, Nguyen Linh Trung1, Adel Hafiane2
1VNU University of Engineering and Technology, Hanoi, Vietnam
2PRISME, INSA CVL, University of Orléans, Orléans, France
3Academic Institute of France, Paris, France

Tóm tắt

In recent years, sparse subspace tracking has attracted increasing attention in the signal processing community. In this paper, we propose a new provable effective method called OPIT (which stands for Online Power Iteration via Thresholding) for tracking the sparse principal subspace of data streams over time. Particularly, OPIT introduces a new adaptive variant of power iteration with space and computational complexity linear to the data dimension. In addition, a new column-based thresholding operator is developed to regularize the subspace sparsity. Utilizing both advantages of power iteration and thresholding operation, OPIT is capable of tracking the underlying subspace in both the classical regime and high dimensional regime. We also present a theoretical result on its convergence to verify its consistency in high dimensions. Several experiments are carried out on both synthetic and real data to demonstrate the performance of OPIT.

Từ khóa

#Sparse subspace tracking #data streams #high dimensions #thresholding #power iteration

Tài liệu tham khảo

10.1002/9780470575758.ch4

10.1109/MSP.2018.2826566

10.1109/JPROC.2018.2844126

10.21553/rev-jec.270

10.1214/07-AOS581

10.1109/TSP.2008.929662

10.1007/s10959-010-0338-z

10.1214/07-AOS559

10.1214/08-aos600

10.1198/jasa.2009.0121

10.1016/j.jmva.2012.10.007

10.1214/08-AOS664

Journée, 2010, Generalized power method for sparse principal component analysis, J. Mach. Learn. Res., 11, 517

10.1214/13-AOS1097

10.1214/13-AOS1178

10.1214/15-EJS1081

10.1109/JPROC.2018.2846588

10.1109/ICASSP43922.2022.9746546

10.23919/EUSIPCO.2017.8081439

10.1109/IEEECONF44664.2019.9048883

10.1016/j.sigpro.2020.107522

10.1109/78.365290

10.1109/TSP.2005.850378

10.1109/TSP.2016.2569471

10.1109/ALLERTON.2016.7852242

10.1016/j.sigpro.2017.02.003

10.1109/ITW.2016.7606821

Yang, 2015, Streaming sparse principal component analysis, Proc. Int. Conf. Mach. Learn., 494

10.1109/97.841157

10.1109/FOCS.2017.51

10.1006/dspr.1999.0348

Abed-Meraim, 2002, On a class of orthonormal algorithms for principal and minor subspace tracking, J. VLSI Signal Process. Syst., 31, 57, 10.1023/A:1014445221814

10.1109/TSP.2007.909335

10.1109/TSP.2011.2169406

10.1109/ACCESS.2018.2863557

10.1109/CVPR.2012.6247848

10.1109/TSP.2013.2282910

10.1109/JSTSP.2018.2876626

10.1109/TSP.2021.3066795

10.1109/TSP.2019.2924595

10.1109/TIT.2018.2872023

10.1109/TSP.2005.861072

10.1016/j.dsp.2016.11.011

10.1109/ISCAS.2006.1692539

10.1109/TCSII.2010.2056414

10.1016/j.sigpro.2021.108402

10.1109/TSP.2023.3289699

10.1109/ICASSP49357.2023.10094931

10.1162/neco.1997.9.7.1493

10.1109/TSP.2017.2695457

10.1109/TIT.2004.836929

10.1109/TSP.2007.907884

10.1214/09-AOS752

10.1198/jasa.2009.0101

Hardt, 2014, The noisy power method: A meta algorithm with applications, Proc. Adv. Neural Inf. Process. Syst., 27, 1

Mackey, 2008, Deflation methods for sparse PCA, Proc. Adv. Neural Inf. Process. Syst., 21, 1

10.1016/j.chemolab.2020.104212

Yuan, 2013, Truncated power method for sparse eigenvalue problems, J. Mach. Learn. Res., 14, 899

Deshpande, 2016, Sparse PCA via covariance thresholding, J. Mach. Learn. Res., 17, 4913

10.1214/15-AOS1369

10.1214/15-AOS1310

10.1214/13-AOS1151

10.1109/97.823526

10.1109/78.553469

10.1016/j.sigpro.2015.04.025

10.1016/j.dsp.2017.01.006

10.1145/2939672.2939763

10.1016/j.neucom.2018.11.030

10.1109/ICASSP39728.2021.9413554

Thông tin
Thông tin xuất bản

IEEE Transactions on Signal Processing

Tập 72

521-534

Thông tin tác giả