Within

statistics Statistics (from German: '' Statistik'', "description of a state, a country") is the discipline that concerns the collection, organization, analysis, interpretation, and presentation of data. In applying statistics to a scientific, indust ...

, Multilinear principal component analysis (MPCA) is a multilinear extension of

principal component analysis Principal component analysis (PCA) is a popular technique for analyzing large datasets containing a high number of dimensions/features per observation, increasing the interpretability of data while preserving the maximum amount of information, and ...

(PCA). MPCA is employed in the analysis of M-way arrays, i.e. a cube or hyper-cube of numbers, also informally referred to as a "data tensor". M-way arrays may be modeled by * linear tensor models such as CANDECOMP/Parafac, or * multilinear tensor models, such as multilinear principal component analysis (MPCA), or multilinear independent component analysis (MICA), etc. The origin of MPCA can be traced back to the

Tucker decomposition In mathematics, Tucker decomposition decomposes a tensor into a set of matrices and one small core tensor. It is named after Ledyard R. Tucker although it goes back to Hitchcock in 1927. Initially described as a three-mode extension of factor an ...

and Peter Kroonenberg's "3-mode PCA" work.P. M. Kroonenberg and J. de Leeuw
Principal component analysis of three-mode data by means of alternating least squares algorithms
Psychometrika, 45 (1980), pp. 69–97. In 2000, De Lathauwer et al. restated Tucker and Kroonenberg's work in clear and concise numerical computational terms in their SIAM paper entitled " Multilinear Singular Value Decomposition", (HOSVD) and in their paper "On the Best Rank-1 and Rank-(R₁, R₂, ..., R_N ) Approximation of Higher-order Tensors". Circa 2001, Vasilescu and Terzopoulos reframed the data analysis, recognition and synthesis problems as multilinear tensor problems. Tensor factor analysis is the compositional consequence of several causal factors of data formation, and are well suited for multi-modal data tensor analysis. The power of the tensor framework was showcased by analyzing human motion joint angles, facial images or textures in terms of their causal factors of data formation in the following works: Human Motion SignaturesM.A.O. Vasilescu (2002
"Human Motion Signatures: Analysis, Synthesis, Recognition," Proceedings of International Conference on Pattern Recognition (ICPR 2002), Vol. 3, Quebec City, Canada, Aug, 2002, 456–460.
/ref> (CVPR 2001, ICPR 2002), face recognition – TensorFaces, (ECCV 2002, CVPR 2003, etc.) and computer graphics – TensorTextures (Siggraph 2004). Historically, MPCA has been referred to as "M-mode PCA", a terminology which was coined by Peter Kroonenberg in 1980. In 2005, Vasilescu and Terzopoulos introduced the Multilinear PCAM. A. O. Vasilescu, D. Terzopoulos (2005
"Multilinear Independent Component Analysis"
"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’05), San Diego, CA, June 2005, vol.1, 547–553." terminology as a way to better differentiate between linear and multilinear tensor decomposition, as well as, to better differentiate between the work that computed 2nd order statistics associated with each data tensor mode(axis), and subsequent work on Multilinear Independent Component Analysis that computed higher order statistics associated with each tensor mode/axis. Multilinear PCA may be applied to compute the causal factors of data formation, or as signal processing tool on data tensors whose individual observation have either been vectorized,M.A.O. Vasilescu, D. Terzopoulos (2002
"Multilinear Analysis of Image Ensembles: TensorFaces," Proc. 7th European Conference on Computer Vision (ECCV'02), Copenhagen, Denmark, May, 2002, in Computer Vision – ECCV 2002, Lecture Notes in Computer Science, Vol. 2350, A. Heyden et al. (Eds.), Springer-Verlag, Berlin, 2002, 447–460.
/ref>M.A.O. Vasilescu, D. Terzopoulos (2003
"Multilinear Subspace Analysis for Image Ensembles,'' M. A. O. Vasilescu, D. Terzopoulos, Proc. Computer Vision and Pattern Recognition Conf. (CVPR '03), Vol.2, Madison, WI, June, 2003, 93–99.
/ref>M.A.O. Vasilescu, D. Terzopoulos (2004
"TensorTextures: Multilinear Image-Based Rendering", M. A. O. Vasilescu and D. Terzopoulos, Proc. ACM SIGGRAPH 2004 Conference Los Angeles, CA, August, 2004, in Computer Graphics Proceedings, Annual Conference Series, 2004, 336–342.
/ref> or whose observations are treated as a collection of column/row observations, "data matrix" and concatenated into a data tensor. The main disadvantage of this approach is that rather than computing computing all possible combinations MPCA computes a set of orthonormal matrices associated with each mode of the data tensor which are analogous to the orthonormal row and column space of a matrix computed by the matrix SVD. This transformation aims to capture as high a variance as possible, accounting for as much of the variability in the data associated with each data tensor mode(axis).

The algorithm

The MPCA solution follows the alternating least square (ALS) approach.It is iterative in nature. As in PCA, MPCA works on centered data. Centering is a little more complicated for tensors, and it is problem dependent.

Feature selection

MPCA features: Supervised MPCA is employed in causal factor analysius that facilitates object recognitionM. A. O. Vasilescu, D. Terzopoulos (2003
"Multilinear Subspace Analysis of Image Ensembles"
"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’03), Madison, WI, June, 2003" while a semi-supervised MPCA feature selection is employed in visualization tasks.

Extensions

Various extension of MPCA: *Robust MPCA (RMPCA) *Multi-Tensor Factorization, that also finds the number of components automatically (MTF)

References

{{Reflist

External links

* ''Matlab code''
MPCA
* ''Matlab code''
UMPCA (including data)
* ''R code:'
MTF
Dimension reduction