A Contemporary and Comprehensive Survey on Streaming Tensor Decomposition
Tóm tắt
Tensor decomposition has been demonstrated to be successful in a wide range of applications, from neuroscience and wireless communications to social networks. In an online setting, factorizing tensors derived from multidimensional data streams is however nontrivial due to several inherent problems of real-time stream processing. In recent years, many research efforts have been dedicated to developing online techniques for decomposing such tensors, resulting in significant advances in streaming tensor decomposition or tensor tracking. This topic is emerging and enriches the literature on tensor decomposition, particularly from the data stream analystics perspective. Thus, it is imperative to carry out an overview of tensor tracking to help researchers and practitioners understand its developments and achievements, summarise the current trends and advances, and identify challenging problems. In this article, we provide a contemporary and comprehensive survey on different types of tensor tracking techniques. We particularly categorize the state-of-the-art methods into three main groups: streaming CP decompositions, streaming Tucker decompositions, and streaming decompositions under other tensor formats (i.e., tensor-train, t-SVD, and BTD). In each group, we further divide the existing algorithms into sub-categories based on their main optimization framework and model architectures. Finally, we present several applications, research challenges, open problems, and potential directions of tensor tracking in the future.
Từ khóa
#BTD #CP #data stream #low-rank tensor approximation #online optimization #tensor decomposition #tensor-train #t-SVD #tuckerTài liệu tham khảo
10.1007/978-3-030-19274-7_5
10.1016/j.neucom.2018.11.030
10.1137/1.9781611976236.65
rambhatla, 2020, Provable online CP/PARAFAC decomposition of a structured tensor via dictionary learning, Proc Int Conf Neural Inf Process, 1
chinh, 2016, Adaptive PARAFAC decomposition for third-order tensor completion, Proc IEEE 6th Int Conf Commun Electron, 297
10.1109/ICDM.2018.00200
10.1109/ICDM.2018.00025
10.1109/TSP.2013.2278516
10.1137/1.9781611975321.10
10.1109/TPAMI.2015.2392756
10.1016/j.neucom.2017.04.058
10.1109/IPDPS.2015.27
li, 2021, SGD-Tucker: A novel stochastic optimization strategy for parallel sparse tucker decomposition, IEEE Trans Parallel Distrib Syst, 32, 1828
10.23919/EUSIPCO.2017.8081290
foster, 2020, Designing and Building Parallel Programs Concepts and Tools for Parallel Software Engineering
dung, 2017, Second-order optimization based adaptive PARAFAC decomposition of three-way tensors, Digit Signal Process, 63, 100, 10.1016/j.dsp.2017.01.006
choi, 2014, DFacTo: Distributed factorization of tensors, Proc Int Conf Neural Inf Process, 1
mahoney, 2011, Randomized algorithms for matrices and data, Found Trends Mach Learn, 3, 123
10.1109/ICACI.2018.8377495
10.1016/j.image.2018.03.017
10.1137/21M1391444
10.1109/TSP.2009.2016885
10.1016/j.knosys.2015.07.013
10.1109/TSP.2015.2417491
10.1109/ICASSP.2016.7472876
10.1137/06066518X
10.1109/MCS.2021.3122268
10.1137/070690730
al daas, 2022, Parallel algorithms for tensor train arithmetic, SIAM J Sci Comput, 44, 25, 10.1137/20M1387158
10.1137/S0895479898346995
10.1145/2939672.2939763
10.1088/1741-2552/ab5247
cohen, 2016, Convolutional rectifier networks as generalized tensor decompositions, Proc Int Conf Mach Learn, 955
10.1137/070690729
10.1007/s10589-020-00167-1
10.1109/JPROC.2015.2455028
10.1007/BF02289464
harshman, 1970, Foundations of the PARAFAC procedure: Models and conditions for an explanatory multimodal factor analysis, UCLA Work Papers Phonetics, 16, 1
10.1016/j.laa.2010.09.020
10.1137/090752286
10.1007/s11263-007-0075-7
10.1007/s13042-011-0017-0
10.1109/JPROC.2021.3074329
10.1002/widm.1133
10.1109/TKDE.2021.3135715
10.1109/JPROC.2015.2474704
10.1016/j.knosys.2016.01.027
10.1561/2200000067
10.1145/2915921
10.1002/widm.1
10.1145/2037676.2037686
10.1109/MSP.2022.3163870
10.1145/3097983.3098087
10.1016/j.jneumeth.2015.03.018
10.3390/s21124021
10.1109/MSP.2013.2297439
10.3390/s19235076
10.1049/iet-spr.2020.0373
10.1109/ACCESS.2019.2949814
10.1109/TKDE.2019.2915231
10.1109/97.410547
10.1002/cem.931
10.1109/TITS.2015.2513411
10.1016/j.chemolab.2022.104550
10.1002/cem.776
10.1016/j.compchemeng.2020.107099
10.1109/MSP.2020.3003544
10.1109/ACCESS.2021.3058103
10.1002/cem.1236
10.1016/j.sigpro.2005.12.016
10.1002/gamm.201310004
10.1007/s40314-022-02107-7
10.1561/2200000059
10.1109/JSTSP.2010.2048070
10.1109/MSP.2014.2329429
10.1137/110841229
10.1109/MSP.2014.2298533
10.1145/2512329
10.1109/JPROC.2018.2848209
akidau, 2018, Streaming Systems The What Where When and How of Large-Scale Data Processing
10.1002/cem.1223
thanh, 2021, Robust subspace tracking algorithms in signal processing: A brief survey, Review Elec Commun, 11, 16
hanggi, 2007, Colored noise in dynamical systems, Adv Chem Phys, 89, 239
10.1109/TWC.2020.2966622
10.1109/TCSI.2017.2667705
bifet, 2010, Adaptive Stream Mining Pattern Learning and Mining from Evolving Data Streams
10.1109/TSP.2015.2504349
10.1007/s10462-020-09916-4
10.1145/1921632.1921636
10.1002/widm.1405
10.1109/TIP.2018.2865684
10.1186/s40537-019-0210-7
min, 2008, Inferring segmented dense motion layers using 5D tensor voting, IEEE Trans Pattern Anal Mach Intell, 30, 1589, 10.1109/TPAMI.2007.70802
fang, 2021, Bayesian streaming sparse Tucker decomposition, Proc Conf Uncertainty Artif Intell, 558
10.1109/TCSVT.2022.3190818
10.1109/JSTSP.2021.3058846
10.1109/ICCV.2017.35
10.1109/JSTSP.2021.3061937
10.3390/sym14010113
10.1109/ICCVW.2015.125
10.1109/TSP.2010.2058802
10.1145/3447548.3467290
10.1109/ACCESS.2022.3186364
10.1109/TKDE.2008.112
10.1016/S0169-7439(97)00032-4
10.1109/MSP.2010.936030
10.1137/07070111X
traore, 2019, Singleshot: A scalable Tucker tensor decomposition, Proc Int Conf Neural Inf Process, 1
10.1109/TBME.2016.2553960
10.1016/j.neucom.2019.08.053
10.3389/fnins.2022.861402
pan, 2020, Streaming nonlinear Bayesian tensor decomposition, Proc Conf Uncertainty Artif Intell, 490
10.1109/MC.2008.431
10.1137/19M1257718
10.1037/0033-295X.111.4.931
10.1016/j.tics.2011.03.006
10.1109/ICDM.2018.00180
madhav, 2018, Inductive framework for multi-aspect streaming tensor completion with side information, Proc Int Conf Inf Knowl Manage, 307
10.1109/TBDATA.2018.2824303
10.1145/1541880.1541882
kasai, 2016, Low-rank tensor completion: A Riemannian manifold preconditioning approach, Proc Int Conf Mach Learn, 1012
10.1109/TSP.2017.8076023
10.1109/HPEC.2016.7761607
trung, 2018, A non-linear tensor tracking algorithm for analysis of incomplete multi-channel EEG data, Proc 12th Int Symp Med Inf Commun Technol, 1
10.1109/TCSS.2018.2813320
10.1109/EMBC.2015.7319226
10.1109/TSIPN.2017.2668146
10.1109/10.40805
10.1109/ICDMW.2014.176
10.1109/TITS.2019.2941649
10.1007/s11263-010-0399-6
10.1109/ICCWorkshops53468.2022.9814591
yu, 2015, Accelerated online low rank tensor learning for multivariate spatiotemporal streams, Proc Int Conf Mach Learn, 238
10.1109/TVCG.2017.2744419
10.1109/TETC.2014.2330516
10.1007/s10618-018-0560-3
10.1007/s10115-014-0733-3
10.1109/TNSM.2016.2598788
10.1109/ICCV.2007.4408950
10.1145/1409620.1409621
10.1109/ACCESS.2019.2955134
10.1145/1150402.1150445
10.1007/s10543-013-0455-z
10.1145/3532189
10.23919/APSIPAASC55919.2022.9980029
10.24963/ijcai.2019/442
10.1109/TCSII.2018.2862900
10.1145/3097983.3098007
10.1109/TNNLS.2018.2860964
10.1109/ICDE51399.2021.00098
10.1109/TSUSC.2018.2881439
10.2478/jdis-2020-0010
10.1016/j.patcog.2011.01.004
10.1007/978-3-030-74386-4
10.1109/TSP.2017.2690524
fang, 2021, Streaming Bayesian deep tensor factorization, Proc Int Conf Mach Learn, 3133
thanh, 2022, Streaming tensor-train decomposition with missing data
10.1137/20M1319097
10.23919/EUSIPCO55093.2022.9909702
boyen, 1998, Tractable inference for complex stochastic processes, Proc 14th Conf Uncertainty Artif Intell, 33
broderick, 2013, Streaming variational Bayes, Proc Int Conf Neural Inf Process, 1
thanh, 2020, Adaptive algorithms for tracking tensor-train decomposition of streaming tensors, Proc 28th Eur Signal Process Conf, 995
10.2307/2347162
10.1109/ALLERTON.2010.5706976
harshman, 1972, PARAFAC2: Mathematical and technical notes, UCLA Work Papers Phonetics, 22
10.1109/ICASSP.2016.7472114
10.1109/78.575696
10.1109/TSP.2022.3164837
thanh, 2022, Tracking online low-rank approximations of incomplete high-order streaming tensors, Patterns
10.1109/DSAA49011.2020.00029
10.1109/ICASSP39728.2021.9413554
10.1109/IEEECONF53345.2021.9723160
lyu, 2022, Online nonnegative CP-dictionary learning for Markovian data, J Mach Learn Res, 23, 1
10.1109/TBDATA.2018.2867485
10.1109/TSP.2022.3201640
10.1109/TII.2020.2967768
10.1109/ICDE51399.2021.00076
10.1109/TPAMI.2016.2539944
10.1016/j.eswa.2017.08.039
10.1145/3459637.3482048
smeulders, 2014, Visual tracking: An experimental survey, IEEE Trans Pattern Anal Mach Intell, 36, 1442, 10.1109/TPAMI.2013.230
dongjin, 2021, Robust factorization of real-world tensor streams with patterns, missing values, and outliers, Proc IEEE Int Conf Data Eng, 840
10.1016/j.neucom.2011.05.006