Computing Reviews
Today's Issue Hot Topics Search Browse Recommended My Account Log In
Review Help
Search
A theoretical perspective on hyperdimensional computing
Thomas A., Dasgupta S., Rosing T. Journal of Artificial Intelligence Research72 (1):215-249,2022.Type:Article
Date Reviewed: Mar 27 2023

Thomas, Dasgupta, and Rosing offer a tricky essay on hyperdimensional (HD) computing, a representational-transformational-retrieval computing paradigm that is brain-inspired, starting from signal levels as electrodes. All the content is highly technical; at the same time, it encourages a nonexpert audience that HD representations of data, such as the extraction of information from images, should work similarly to the recognition accuracy, speed of learning, robustness, and so on, of human brains. Readers are presented with the very simple neural architectures of HD computing, built by sampling each component independent and identically distributed (IID) from some distribution and using the central limit theorem (CLT) on simple inputs x∈Rd that are collected and then “mapped under a random linear transformation to a point ϕ(x)∈ Rd (dn)” in a high-dimensional space, the so-called H-space [1].

“Simple operations like element-wise additions and dot products,” basic objects combined on an orthonormal basis parameter, lookup tables, connectivity matrices, and so on represent seed vectors and are in the form of discrete symbolic primitives as letters or words. Because of retraining purposes, encoding schemes can be hugely modified step by step through rotation and permutation methods, up to the point of constituting a family of vector symbolic architecture (VSA) methods [2]. The simple, non-hierarchic, context-free perception-action cycle (PAC) found in Claude Shannon’s information science, in the 1950s, is enriched with long short-term memory (LSTM) codewords drawn from sub-Gaussian distributions [3]. Therefore, the authors show a number of examples of encodings and decodings to highlight how the representation of sets in HD computing “is a tradeoff between the incoherence of the codebook and robustness to noise.” Binding operators such as “⊗∶ H × H → H creates an embedding for a (feature, value) pair” that satisfies various incoherence properties with high probability. Discrete data like sets, series, and sequences are fused across different modalities, considering also that a measure of the decoding mechanism preserves the angular distance up to an additive distortion for subsequent analysis [4].

This way may be pretty intuitive, that is, what a reasonable superimposition of structured elements should look like, and the real-valued embeddings “arrive in a streaming, or online, fashion, although ... apply to fixed and finite data as well.” This means that HD construction can be applicable to shift-invariant kernels on a Euclidean space, while the correlation between sequences depends on the magnitude of each hypervector, and then only partially improves the scalability of input images.

Reviewer:  Romina Fucà Review #: CR147566 (2306-0077)
1) Ge, L.; Parhi, K. K. Classification using hyperdimensional computing: a review. IEEE Circuits and Systems Magazine 20, 2(2020), 30–47.
2) Gupta, S.; Khaleghi, B.; Salamat, S. Store-n-Learn: classification and clustering with hyperdimensional computing across flash hierarchy. ACM Transactions on Embedded Computing Systems 21, 3(2022), 1–25.
3) Graben, P. b.; Huber, M.; Meyer, W. Vector symbolic architectures for context-free grammars. Cognitive Computation 14, (2022), 733–748.
4) Nachtergaele, B.; Scholz, V. B.; Werner, R. F. Local approximation of observables and commutator bounds. Operator theory: advances and applications (vol. 227) 227, (2013), 143–149.
Bookmark and Share
  Featured Reviewer  
 
General (F.0 )
 
 
General (I.2.0 )
 
 
Other Architecture Styles (C.1.3 )
 
 
Parallel Architectures (C.1.4 )
 
 
Data Storage Representations (E.2 )
 
 
Data Structures (E.1 )
 
  more  
Would you recommend this review?
yes
no
Other reviews under "General": Date
Introduction to computer theory (revised ed.)
Cohen D., John Wiley & Sons, Inc., New York, NY, 1991. Type: Book (9780471510109)
Feb 1 1993
Introduction to computer theory
Cohen D., John Wiley & Sons, Inc., New York, NY, 1986. Type: Book (9789780471802716)
Feb 1 1987
Theory of computation: formal languages, automata, and complexity
Brookshear J., Benjamin-Cummings Publ. Co., Inc., Redwood City, CA, 1989. Type: Book (9789780805301434)
Jan 1 1990
more...

E-Mail This Printer-Friendly
Send Your Comments
Contact Us
Reproduction in whole or in part without permission is prohibited.   Copyright 1999-2024 ThinkLoud®
Terms of Use
| Privacy Policy