Tensor Reshaping

In multilinear algebra, a reshaping of tensors is any bijection between the set of indices of an order-$d$ tensor and the set of indices of an order-$\ell$ tensor, where $\ell < d$. The use of indices presupposes tensors in coordinate representation with respect to a basis. The coordinate representation of a tensor can be regarded as a multi-dimensional array, and a bijection from one set of indices to another therefore amounts to a rearrangement of the array elements into an array of a different shape. Such a rearrangement constitutes a particular kind of linear map between the vector space of order-$d$ tensors and the vector space of order-$\ell$ tensors.

Definition

Given a positive integer $d$, the notation /math> refers to the set $\$ of the first positive integers.

For each integer $k$ where $1 \le k \le d$ for a positive integer $d$, let ''V''_''k'' denote an ''n''_''k''-dimensional vector space over a field $F$. Then there are vector space isomorphisms (linear maps)

$\begin V_1 \otimes \cdots \otimes V_d & \simeq F^ \otimes \cdots \otimes F^ \\ & \simeq F^ \otimes \cdots \otimes F^ \\ & \simeq F^ \otimes F^ \otimes \cdots \otimes F^ \\ & \simeq F^ \otimes F^ \otimes F^ \otimes \cdots \otimes F^ \\ & \,\,\,\vdots \\ & \simeq F^, \end$

where $\pi \in \mathfrak_d$ is any permutation and $\mathfrak_d$ is the symmetric group on $d$ elements. Via these (and other) vector space isomorphisms, a tensor can be interpreted in several ways as an order-$\ell$ tensor where $\ell \le d$.

Coordinate representation

The first vector space isomorphism on the list above, $V_1 \otimes \cdots \otimes V_d \simeq F^ \otimes \cdots \otimes F^$, gives the coordinate representation of an abstract tensor. Assume that each of the $d$ vector spaces $V_k$ has a basis $\$. The expression of a tensor with respect to this basis has the form $\mathcal = \sum_^\ldots\sum_^ a_ v_^1 \otimes v_^2 \otimes \cdots \otimes v_^,$ where the coefficients $a_$ are elements of $F$. The coordinate representation of $\mathcal$ is $\sum_^\ldots\sum_^ a_ \mathbf_^1 \otimes \mathbf_^2 \otimes \cdots \otimes \mathbf_^d,$where $\mathbf_^k$ is the $j^\text$ standard basis vector of $F^$. This can be regarded as a ''d''-dimensional array whose elements are the coefficients $a_$.

Vectorization

By means of a bijective map
\mu : _1\times \cdots \times _d\to _1\cdots n_d, a vector space isomorphism between $F^ \otimes \cdots \otimes F^$ and $F^$ is constructed via the mapping $\mathbf_^1 \otimes \cdots \otimes \mathbf_^d \mapsto \mathbf_,$ where for every natural number $j$ such that $1 \le j \le n_1 \cdots n_d$, the vector $\mathbf_j$ denotes the ''j''th standard basis vector of $F^$. In such a reshaping, the tensor is simply interpreted as a vector in $F^$. This is known as vectorization, and is analogous to vectorization of matrices. A standard choice of bijection $\mu$ is such that

:$\operatorname(\mathcal) := \begin a_ & a_ & \cdots & a_ & a_ & \cdots & a_ \end^T,$

which is consistent with the way in which the colon operator in Matlab and GNU Octave reshapes a higher-order tensor into a vector. In general, the vectorization of $\mathcal$ is the vector $$a_ ^ .

General flattenings

For any permutation $\pi \in \mathfrak_d$ there is a canonical isomorphism between the two tensor products of vector spaces $V_1 \otimes V_2 \otimes \cdots \otimes V_d$ and $V_ \otimes V_ \otimes \cdots \otimes V_$. Parentheses are usually omitted from such products due to the natural isomorphism between $V_i\otimes(V_j\otimes V_k)$ and $(V_i\otimes V_j)\otimes V_k$, but may, of course, be reintroduced to emphasize a particular grouping of factors. In the grouping,
::$(V_ \otimes \cdots \otimes V_)\otimes(V_ \otimes \cdots \otimes V_)\otimes\cdots\otimes(V_ \otimes \cdots \otimes V_),$
there are $\ell$ groups with $r_j-r_$ factors in the $j^\text$ group (where $r_0=0$ and $r_\ell=d$).

Letting $S_j=(\pi(r_+1),\pi(r_+2),\ldots,\pi(r_j))$ for each $j$ satisfying $1\le j\le\ell$, an $(S_1,S_2,\ldots,S_\ell)$-flattening of a tensor $\mathcal$, denoted $\mathcal_$, is obtained by applying the two processes above within each of the $\ell$ groups of factors. That is, the coordinate representation of the $j^\text$ group of factors is obtained using the isomorphism $(V_ \otimes V_ \otimes \cdots \otimes V_)\simeq(F^\otimes F^\otimes\cdots\otimes F^)$, which requires specifying bases for all of the vector spaces $V_k$. The result is then vectorized using a bijection \mu_j: _times _times\cdots\times _to _/math> to obtain an element of $F^$, where $N_ := \prod_^ n_$, the product of the dimensions of the vector spaces in the $j^\text$ group of factors. The result of applying these isomorphisms within each group of factors is an element of $F^ \otimes \cdots \otimes F^$, which is a tensor of order $\ell$.

The vectorization of $\mathcal$ is an $(S_1)$-reshaping, $\mathcal_$ wherein $S_1 = (1,2,\ldots,d)$.

Matrixize

Let $\mathcal \in F^ \otimes F^ \otimes \cdots \otimes F^$ be the coordinate representation of an abstract tensor with respect to a basis.
A standard mode-''m'' flattening of $\mathcal$ is an $$
_1, S_2/math>-reshaping in which $S_1 = (m)$ and $S_2 = (1,2,\ldots,m-1,m+1,\ldots,M)$. Usually, a standard flattening is denoted by

: $\mathcal_ := \mathcal_$

These reshapings are sometimes called matrixizing, matricizations or unfoldings in the literature. A standard choice for the bijections $\mu_1,\ \mu_2$ is the one that is consistent with the reshape function in Matlab and GNU Octave, namely

: $\mathcal_ := \begin a_ & a_ & \cdots & a_ \\ a_ & a_ & \cdots & a_ \\ \vdots & \vdots & & \vdots \\ a_ & a_ & \cdots & a_ \end{bmatrix}$

Tensors
Multilinear algebra