Tensor Decomposition for Machine Learning
Xinyu Chen, Dingyi Zhuang, Jinhua Zhao (2024)
This article summarizes the development of tensor decomposition models and algorithms in the literature, offering comprehensive reviews and tutorials on topics ranging from matrix and tensor computations to tensor decomposition techniques across a wide range of scientific areas and applications. Since the decomposition of tensors is often formulated as an optimization problem, this article also provides a preliminary introduction to some classical methods for solving convex and nonconvex optimization problems. This work aims to offer valuable insights to both the machine learning and data science communities by drawing strong connections with the key concepts related to tensor decomposition. To ensure reproducibility and sustainability, we provide resources such as datasets and Python implementations, primarily utilizing Python’s numpy library.
Table of Contents
- Introduction
- Tensor decomposition in the past 10-100 years
- Tensor decomposition in the past decade
- What Are Tensors?
- Tensors in algebra & machine learning
- Tensors in data science
- Foundation of Tensor Computations
- Norms
- Matrix trace
- Kronecker product
- Khatri-Rao product
- Modal product
- Outer product
- Derivatives
- Foundation of Optimization
Materials & References
- Yuejie Chi, Yue M. Lu, and Yuxin Chen (2019). Nonconvex optimization meets low-rank matrix factorization: An overview. IEEE Transactions on Signal Processing, 67(20): 5239-5269.
- Marc Peter Deisenroth, A. Aldo Faisal, and Cheng Soon Ong (2020). Mathematics for Machine Learning, Cambridge University Press. [Book website]
YouTube
- Ankur Moitra: “Tensor Decompositions and their Applications (Part 1/2)”
- Ankur Moitra: “Tensor Decompositions and their Applications (Part 2/2)”