Near-channel classifier: symbiotic communication and classification in high-dimensional space
Tóm tắt
Brain-inspired high-dimensional (HD) computing represents and manipulates data using very long, random vectors with dimensionality in the thousands. This representation provides great robustness for various classification tasks where classifiers operate at low signal-to-noise ratio (SNR) conditions. Similarly, hyperdimensional modulation (HDM) leverages the robustness of complex-valued HD representations to reliably transmit information over a wireless channel, achieving a similar SNR gain compared to state-of-the-art codes. Here, we first propose methods to improve HDM in two ways: (1) reducing the complexity of encoding and decoding operations by generating, manipulating, and transmitting bipolar or integer vectors instead of complex vectors; (2) increasing the SNR gain by 0.2 dB using a new soft-feedback decoder; it can also increase the additive superposition capacity of HD vectors up to 1.7
$$\times$$
in noise-free cases. Secondly, we propose to combine encoding/decoding aspects of communication with classification into a single framework by relying on multifaceted HD representations. This leads to a near-channel classification (NCC) approach that avoids transformations between different representations and the overhead of multiple layers of encoding/decoding, hence reducing latency and complexity of a wireless smart distributed system while providing robustness against noise and interference from other nodes. We provide a use-case for wearable hand gesture recognition with 5 classes from 64 EMG sensors, where the encoded vectors are transmitted to a remote node for either performing NCC, or reconstruction of the encoded data. In NCC mode, the original classification accuracy of 94% is maintained, even in the channel at SNR of 0 dB, by transmitting 10,000-bit vectors. We remove the redundancy by reducing the vector dimensionality to 2048-bit that still exhibits a graceful degradation: less than 6% accuracy loss is occurred in the channel at − 5 dB, and with the interference from 6 nodes that simultaneously transmit their encoded vectors. In the reconstruction mode, it improves the mean-squared error by up to 20 dB, compared to standard decoding, when transmitting 2048-dimensional vectors.
Tài liệu tham khảo
Bogue R (2014) Towards the trillion sensors market. Sensor Rev 34(2):137–142
Liu S, Cai W, Liu S, Zhang F, Fulham M, Feng D, Pujol S, Kikinis R (2015) Multimodal neuroimaging computing: the workflows, methods, and platforms. Brain Informat 2(3):181–195
Rawnaque FS, Rahman KM, Anwar SF, Vaidyanathan R, Chau T, Sarker F, Mamun KAA (2020) Technological advancements and opportunities in Neuromarketing: a systematic review. Brain Informat 7(1):10
Chettri L, Bera R (2020) A comprehensive survey on internet of things (IoT) toward 5G wireless systems. IEEE Internet Things J 7(1):16–32
Rabaey JM (2020) Human-centric computing. IEEE Trans Very Large Scale Integr (VLSI) Syst 28(1):3–11
Samie F, Bauer L, Henkel J (2019) From cloud down to things: an overview of machine learning in internet of things. IEEE Internet Things J 6(3):4921–4934
Yang K, Shi Y, Yu W, Ding Z (2020) Energy-efficient processing and robust wireless cooperative transmission for edge inference. IEEE Internet Things J 7(10):9456–9470
Deng S, Zhao H, Yin J, Dustdar S, Zomaya AY (2019) Edge intelligence: the confluence of edge computing and artificial intelligence. arXiv 7(8):7457–7469
Fafoutis X, Marchegiani L, Elsts A, Pope J, Piechocki R, Craddock I (2018) Extending the battery lifetime of wearable sensors with embedded machine learning. In: 2018 IEEE 4th World Forum on Internet of Things (WF-IoT), pp. 269–274
Kanerva P (2009) Hyperdimensional computing: an introduction to computing in distributed representation with high-dimensional random vectors. Cogn Comput 1(2):139–159
Kanerva P (2019) Computing with high-dimensional vectors. IEEE Design Test 36(3):7–14
Plate TA (1995) Holographic reduced representations. IEEE Trans Neural Netw 6(3):623–641
Gayler RW (1998) Multiplicative binding, representation operators and analogy (Workshop Poster). http://cogprints.org/502/
Frady EP, Kleyko D, Sommer FT (2018) A theory of sequence indexing and working memory in recurrent neural networks. Neural Comput 30(6):1449–1513
Kanerva P (2000) Large patterns make great symbols: an example of learning from example. In: Wermter S, Sun R (eds) Hybrid Neural Syst. Springer, Berlin, Heidelberg, pp 194–203
Kanerva P (2010) What we mean when we say “What’s the dollar of Mexico?”: Prototypes and mapping in concept space. AAAI Fall Symposium-Technical Report FS-10-08:2-6
Kanerva P, Kristoferson J, Holst A (2000) Random indexing of text samples for latent semantic analysis. In: Proceedings of the Annual Meeting of the Cognitive Science Society 22(22)
Joshi A, Halseth JT, Kanerva P (2016) Language geometry using random indexing. In: International Symposium on Quantum Interaction, pp. 265–274
Recchia G, Sahlgren M, Kanerva P, Jones MN (2015) Encoding sequential information in semantic space models: comparing holographic reduced representation and random permutation. Comput Intell Neurosci 2015:986574–986574
Rahimi A, Kanerva P, Rabaey JM (2016) A robust and energy-efficient classifier using brain-inspired hyperdimensional computing. In: Proceedings of the 2016 International Symposium on Low Power Electronics and Design - ISLPED ’16, pp. 64–69. ACM Press, New York, New York, USA
Räsänen O (2015) Generating hyperdimensional distributed representations from continuous-valued multivariate sensory input. In: Proceedings of the 37th Annual Meeting of the Cognitive Science Society, pp. 1943–1948
Moin A, Zhou A, Rahimi A, Benatti S, Menon A, Tamakloe S, Ting J, Yamamoto N, Khan Y, Burghardt F, Benini L, Arias AC, Rabaey JM (2018) An EMG gesture recognition system with flexible high-density sensors and brain-inspired high-dimensional classifier. Proc IEEE Int Symp Circuits Syst 2018–May:1–5
Rahimi A, Kanerva P, Benini L, Rabaey JM (2019) Efficient biosignal processing using hyperdimensional computing: network templates for combined learning and classification of ExG signals. Proc IEEE 107(1):123–143
Chang EJ, Rahimi A, Benini L, Wu AYA (2019) Hyperdimensional computing-based multimodality emotion recognition with physiological signals. In: 2019 IEEE International Conference on Artificial Intelligence Circuits and Systems (AICAS), pp. 137–141
Burrello A, Cavigelli L, Schindler K, Benini L, Rahimi A (2019) Laelaps: an energy-efficient seizure detection algorithm from long-term human ieeg recordings without false alarms. In: 2019 Design, Automation and Test in Europe Conference and Exhibition (DATE), pp. 752–757. IEEE
Mitrokhin A, Sutor P, Fermüller C, Aloimonos Y (2019) Learning sensorimotor control with neuromorphic sensors: toward hyperdimensional active perception. Sci Robotics 4(30):6736
Hersche M, Sangalli S, Benini L, Rahimi A (2020) Evolvable hyperdimensional computing: unsupervised regeneration of associative memory to recover faulty components. In: 2020 2nd IEEE International Conference on Artificial Intelligence Circuits and Systems (AICAS), pp. 281–285
Li H, Wu TF, Rahimi A, Li K-S, Rusch M, Lin C-H, Hsu J-L, Sabry MM, Eryilmaz SB, Sohn J, Chiu W-C, Chen M-C, Wu T-T, Shieh J-M, Yeh W-K, Rabaey JM, Mitra S, Wong H-SP (2016) Hyperdimensional computing with 3D VRRAM in-memory kernels: Device-architecture co-design for energy-efficient, error-resilient language recognition. In: 2016 IEEE International Electron Devices Meeting (IEDM), pp. 1–16
Wu TF, Li H, Huang P-C, Rahimi A, Rabaey JM, Wong H-SP, Shulaker MM, Mitra S (2018) Brain-inspired computing exploiting carbon nanotube FETs and resistive RAM: Hyperdimensional computing case study. In: 2018 IEEE International Solid-State Circuits Conference-(ISSCC), pp. 492–494
Karunaratne G, Le Gallo M, Cherubini G, Benini L, Rahimi A, Sebastian A (2020) In-memory hyperdimensional computing. Nat Electron 3(June):1–11
Jakimovski P, Becker F, Sigg S, Schmidtke HR, Beigl M (2011) Collective communication for dense sensing environments. In: 2011 Seventh International Conference on Intelligent Environments, pp. 157–164
Kleyko D, Lyamin N, Osipov E, Riliskis L (2012) Dependable mac layer architecture based on holographic data representation using hyper-dimensional binary spatter codes. In: Bellalta B, Vinel A, Jonsson M, Barcelo J, Maslennikov R, Chatzimisios P, Malone D (eds) Multiple access communications. Springer, Berlin, Heidelberg, pp 134–145
Kim H-S (2018) HDM: Hyper-dimensional modulation for robust low-power communications. In: 2018 IEEE International Conference on Communications (ICC), pp. 1–6
Hsu CW, Kim HS (2019) Collision-tolerant narrowband communication using non-orthogonal modulation and multiple access. In: 2019 IEEE Global Communications Conference (GLOBECOM), pp. 1–6
Verma D, Bent G, Taylor I (2017) Towards a distributed federated brain architecture using cognitive IoT devices. In: The Ninth International Conference on Advanced Cognitive Technologies and Applications (COGNITIVE)
Tomsett R, Bent G, Simpkin C, Taylor I, Harbourne D, Preece A, Ganti R (2019) Demonstration of dynamic distributed orchestration of node-RED IoT workflows using a vector symbolic architecture. In: 2019 IEEE International Conference on Smart Computing (SMARTCOMP), pp. 464–467
Hsu C-W, Kim H-S (2020) Non-orthogonal modulation for short packets in massive machine type communications. In: GLOBECOM 2020–2020 IEEE Global Communications Conference, pp. 1–6
Schmuck M, Benini L, Rahimi A (2019) Hardware optimizations of dense binary hyperdimensional computing: rematerialization of hypervectors, binarized bundling, and combinational associative memory. ACM J Emerg Technol Comput Syst 15(4):1–25
Eliasmith C (2013) How to Build a Brain. Oxford University Press, Oxford
Gayler RW (2004) Vector symbolic architectures answer Jackendoff’s challenges for cognitive neuroscience. arXiv preprint arXiv:cs/0412059
Cheung B, Terekhov A, Chen Y, Agrawal P, Olshausen B (2019) Superposition of many models into one. Adv Neural Inform Process Syst 32:10868–10877
Sebastian A, Le Gallo M, Khaddam-Aljameh R, Eleftheriou E (2020) Memory devices and applications for in-memory computing. Nat Nanotechnol 15(7):529–544
Bioglio V, Condo C, Land I (2020) Design of polar codes in 5G New Radio. IEEE Communications Surveys and Tutorials (c) 1–1
Balatsoukas-Stimming A, Parizi MB, Burg A (2015) LLR-based successive cancellation list decoding of polar codes. IEEE Trans Signal Process 63(19):5165–5179
Ganesan K, Grover P, Rabaey J (2011) The power cost of over-designing codes. In: 2011 IEEE Workshop on Signal Processing Systems (SiPS), pp. 128–133
Marchioni A, Mangia M, Pareschil F, Rovatti R, Setti G (2018) Rakeness-based compressed sensing of surface electromyography for improved hand movement recognition in the compressed domain. In: 2018 IEEE Biomedical Circuits and Systems Conference (BioCAS), pp. 2018–2021
Cao Y, Zhang H, Choi YB, Wang H, Xiao S (2020) Hybrid deep learning model assisted data compression and classification for efficient data delivery in mobile health applications. IEEE Access 8:94757–94766
Xiang L, Zeng X, Wu S, Liu Y, Yuan B (2021) Computation of cnn’s sensitivity to input perturbation. Neural Process Lett 53(1):535–560
Moin A, Zhou A, Rahimi A, Menon A, Benatti S, Alexandrov G, Tamakloe S, Ting J, Yamamoto N, Khan Y et al (2021) A wearable biosensing system with in-sensor adaptive machine learning for hand gesture recognition. Nat Electron 4(1):54–63
Frady EP, Kent SJ, Olshausen BA, Sommer FT (2020) Resonator networks, 1: an efficient solution for factoring high-dimensional, distributed representations of data structures. Neural Comput 32(12):2311–2331
Kent SJ, Frady EP, Sommer FT, Olshausen BA (2020) Resonator networks, 2: factorization performance and capacity compared to optimization-based methods. Neural Comput 32(12):2332–2388