Multilinear principal component analysis (MPCA) is a
multilinear extension of
principal component analysis
Principal component analysis (PCA) is a linear dimensionality reduction technique with applications in exploratory data analysis, visualization and data preprocessing.
The data is linearly transformed onto a new coordinate system such that th ...
(PCA) that is used to analyze M-way arrays, also informally referred to as "data tensors". M-way arrays may be modeled by linear tensor models, such as CANDECOMP/Parafac, or by multilinear tensor models, such as multilinear principal component analysis (MPCA) or multilinear independent component analysis (MICA).
Tensor rank decomposition were introduced by
Frank Lauren Hitchcock in 1927; explanded upon with the
Tucker decomposition; and by the "3-mode PCA" by Kroonenberg
[P. M. Kroonenberg and J. de Leeuw]
Principal component analysis of three-mode data by means of alternating least squares algorithms
Psychometrika, 45 (1980), pp. 69–97 Kroonenbeg's algorithm is an itterative algorithm that employs gradient descent. In 2000, De Lathauwer et al. restated Tucker and Kroonenberg's work in clear terms in their SIAM paper entitled "
Multilinear Singular Value Decomposition",
and provided an itterative algorithm that employed the power method in their paper "On the Best Rank-1 and Rank-(R
1, R
2, ..., R
N ) Approximation of Higher-order Tensors".
Vasilescu and Terzopoulos in their paper "
Multilinear Image Representation: TensorFaces"
[ introduced the M-mode SVD algorithm which is a simple and elegant algorithm suitable for parallel computation. This algorithm is often misidentified in the literature as the HOSVD or the Tucker which are sequential itterative algorithms that employ gradient descent. Vasilescu and Terzopoulos framed the data analysis, recognition and synthesis problems as multilinear tensor problems. Data is viewed as the compositional consequence of several causal factors, and which are well suited for multi-modal tensor factor analysis. The power of the tensor framework was showcased by analyzing human motion joint angles, facial images or textures in the following papers: Human Motion Signatures][M.A.O. Vasilescu (2002]
"Human Motion Signatures: Analysis, Synthesis, Recognition," Proceedings of International Conference on Pattern Recognition (ICPR 2002), Vol. 3, Quebec City, Canada, Aug, 2002, 456–460.
/ref>
(CVPR 2001, ICPR 2002), face recognition – TensorFaces,[M. A. O. Vasilescu, D. Terzopoulos (2005]
"Multilinear Independent Component Analysis"
"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’05), San Diego, CA, June 2005, vol.1, 547–553." terminology as a way to better differentiate between linear and multilinear tensor decomposition, as well as, to better differentiate between the work[M.A.O. Vasilescu, D. Terzopoulos (2002]
"Multilinear Analysis of Image Ensembles: TensorFaces," Proc. 7th European Conference on Computer Vision (ECCV'02), Copenhagen, Denmark, May, 2002, in Computer Vision – ECCV 2002, Lecture Notes in Computer Science, Vol. 2350, A. Heyden et al. (Eds.), Springer-Verlag, Berlin, 2002, 447–460.
/ref>[M.A.O. Vasilescu, D. Terzopoulos (2003]
"Multilinear Subspace Analysis for Image Ensembles,'' M. A. O. Vasilescu, D. Terzopoulos, Proc. Computer Vision and Pattern Recognition Conf. (CVPR '03), Vol.2, Madison, WI, June, 2003, 93–99.
/ref>[M.A.O. Vasilescu, D. Terzopoulos (2004]
"TensorTextures: Multilinear Image-Based Rendering", M. A. O. Vasilescu and D. Terzopoulos, Proc. ACM SIGGRAPH 2004 Conference Los Angeles, CA, August, 2004, in Computer Graphics Proceedings, Annual Conference Series, 2004, 336–342.
/ref> or whose observations are treated as a collection of column/row observations, an "observation as a matrix", and concatenated into a data tensor. The main disadvantage of the latter approach is that MPCA computes a set of orthonormal matrices associated with row and column space that are unrelated to the causal factors of data formation.
The algorithm
The MPCA solution follows the alternating least square (ALS) approach. It is iterative in nature. As in PCA, MPCA works on centered data. Centering is a little more complicated for tensors, and it is problem dependent.
Feature selection
MPCA features: Supervised MPCA is employed in causal factor analysis that facilitates object recognition[M. A. O. Vasilescu, D. Terzopoulos (2003]
"Multilinear Subspace Analysis of Image Ensembles"
"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’03), Madison, WI, June, 2003" while a semi-supervised MPCA feature selection is employed in visualization tasks.
Extensions
Various extension of MPCA:
*Robust MPCA (RMPCA)
*Multi-Tensor Factorization, that also finds the number of components automatically (MTF)
References
{{Reflist
External links
* ''Matlab code''
MPCA
* ''Matlab code''
UMPCA (including data)
* ''R code:'
MTF
Dimension reduction