Khatri–Rao Product

In mathematics, the Khatri–Rao product or block Kronecker product of two partitioned matrices $\mathbf$ and $\mathbf$ is defined as
:$\mathbf \ast \mathbf = \left(\mathbf_ \otimes \mathbf_\right)_$

in which the ''ij''-th block is the sized Kronecker product of the corresponding blocks of A and B, assuming the number of row and column partitions of both matrices is equal. The size of the product is then .

For example, if A and B both are partitioned matrices e.g.:
: \mathbf =
\left \begin
\mathbf_ & \mathbf_ \\
\hline
\mathbf_ & \mathbf_
\end
\right=
\left \begin
1 & 2 & 3 \\
4 & 5 & 6 \\
\hline
7 & 8 & 9
\end
\right,\quad
\mathbf =
\left \begin
\mathbf_ & \mathbf_ \\
\hline
\mathbf_ & \mathbf_
\end
\right=
\left \begin
1 & 4 & 7 \\
\hline
2 & 5 & 8 \\
3 & 6 & 9
\end
\right,

we obtain:
:
\mathbf \ast \mathbf =
\left \begin
\mathbf_ \otimes \mathbf_ & \mathbf_ \otimes \mathbf_ \\
\hline
\mathbf_ \otimes \mathbf_ & \mathbf_ \otimes \mathbf_
\end
\right=
\left \begin
1 & 2 & 12 & 21 \\
4 & 5 & 24 & 42 \\
\hline
14 & 16 & 45 & 72 \\
21 & 24 & 54 & 81
\end
\right

This is a submatrix of the Tracy–Singh product

of the two matrices (each partition in this example is a partition in a corner of the Tracy–Singh product).

Column-wise Kronecker product

The column-wise Kronecker product of two matrices is a special case of the Khatri-Rao product as defined above, and may also be called ''the'' Khatri–Rao product. This product assumes the partitions of the matrices are their columns. In this case , , and for each ''j'': . The resulting product is a matrix of which each column is the Kronecker product of the corresponding columns of ''A'' and ''B''. Using the matrices from the previous examples with the columns partitioned:
:
\mathbf =
\left \begin
\mathbf_1 & \mathbf_2 & \mathbf_3
\end
\right=
\left \begin
1 & 2 & 3 \\
4 & 5 & 6 \\
7 & 8 & 9
\end
\right,\quad
\mathbf =
\left \begin
\mathbf_1 & \mathbf_2 & \mathbf_3
\end
\right=
\left \begin
1 & 4 & 7 \\
2 & 5 & 8 \\
3 & 6 & 9
\end
\right,

so that:
:
\mathbf \ast \mathbf
=
\left \begin
\mathbf_1 \otimes \mathbf_1 & \mathbf_2 \otimes \mathbf_2 & \mathbf_3 \otimes \mathbf_3
\end
\right=
\left \begin
1 & 8 & 21 \\
2 & 10 & 24 \\
3 & 12 & 27 \\
4 & 20 & 42 \\
8 & 25 & 48 \\
12 & 30 & 54 \\
7 & 32 & 63 \\
14 & 40 & 72 \\
21 & 48 & 81
\end
\right

This column-wise version of the Khatri–Rao product is useful in linear algebra approaches to data analytical processing and in optimizing the solution of inverse problems dealing with a diagonal matrix.

In 1996 the column-wise Khatri–Rao product was proposed to estimate the angles of arrival (AOAs) and delays of multipath signals and four coordinates of signals sources at a digital antenna array.

Face-splitting product

An alternative concept of the matrix product, which uses row-wise splitting of matrices with a given quantity of rows, was proposed by V. SlyusarAnna Esteve, Eva Boj & Josep Fortiana (2009): "Interaction Terms in Distance-Based Regression," ''Communications in Statistics – Theory and Methods'', 38:19, p. 350

/ref> in 1996.

This matrix operation was named the "face-splitting product" of matrices or the "transposed Khatri–Rao product". This type of operation is based on row-by-row Kronecker product, Kronecker products of two matrices. Using the matrices from the previous examples with the rows partitioned:
:$\mathbf = \begin \mathbf_1 \\\hline \mathbf_2 \\\hline \mathbf_3\\ \end = \begin 1 & 2 & 3 \\\hline 4 & 5 & 6 \\\hline 7 & 8 & 9 \end ,\quad \mathbf = \begin \mathbf_1\\\hline \mathbf_2\\\hline \mathbf_3\\ \end = \begin 1 & 4 & 7 \\\hline 2 & 5 & 8 \\\hline 3 & 6 & 9 \end ,$

the result can be obtained:
:$\mathbf \bull \mathbf = \begin \mathbf_1 \otimes \mathbf_1\\\hline \mathbf_2 \otimes \mathbf_2\\\hline \mathbf_3 \otimes \mathbf_3\\ \end = \begin 1 & 4 & 7 & 2 & 8 & 14 & 3 & 12 & 21 \\\hline 8 & 20 & 32 & 10 & 25 & 40 & 12 & 30 & 48 \\\hline 21 & 42 & 63 & 24 & 48 & 72 & 27 & 54 & 81 \end.$

Main properties

In the following properties, the operator $\bull$ denotes the row-wise Kronecker product (face-splitting product) and the operator $\ast$ denotes the column-wise Kronecker product

Examples

Source:

: \begin
&\left(
\begin
1 & 0 \\
0 & 1 \\
1 & 0
\end
\bullet
\begin
1 & 0 \\
1 & 0 \\
0 & 1
\end
\right)
\left(
\begin
1 & 1 \\
1 & -1
\end
\otimes
\begin
1 & 1 \\
1 & -1
\end
\right)
\left(
\begin
\sigma_1 & 0 \\
0 & \sigma_2 \\
\end
\otimes
\begin
\rho_1 & 0 \\
0 & \rho_2 \\
\end
\right)
\left(
\begin
x_1 \\
x_2
\end
\ast
\begin
y_1 \\
y_2
\end
\right)
\\ pt = &\left(
\begin
1 & 0 \\
0 & 1 \\
1 & 0
\end
\bullet
\begin
1 & 0 \\
1 & 0 \\
0 & 1
\end
\right)
\left(
\begin
1 & 1 \\
1 & -1
\end
\begin
\sigma_1 & 0 \\
0 & \sigma_2 \\
\end
\begin
x_1 \\
x_2
\end
\,\otimes\,
\begin
1 & 1 \\
1 & -1
\end
\begin
\rho_1 & 0 \\
0 & \rho_2 \\
\end
\begin
y_1 \\
y_2
\end
\right)
\\ pt = &
\begin
1 & 0 \\
0 & 1 \\
1 & 0
\end
\begin
1 & 1 \\
1 & -1
\end
\begin
\sigma_1 & 0 \\
0 & \sigma_2 \\
\end
\begin
x_1 \\
x_2
\end
\,\circ\,
\begin
1 & 0 \\
1 & 0 \\
0 & 1
\end
\begin
1 & 1 \\
1 & -1
\end
\begin
\rho_1 & 0 \\
0 & \rho_2 \\
\end
\begin
y_1 \\
y_2
\end
.
\end

Theorem

Source:

If $M = T^ \bullet \dots \bullet T^$, where $T^, \dots, T^$ are independent components a random matrix $T$ with independent identically distributed rows $T_1, \dots, T_m\in \mathbb R^d$, such that
: E\left T_1x)^2\right= \left\, x\right\, _2^2 and E\left T_1 x)^p\right\frac \le \sqrt\, x\, _2,

then for any vector $x$
: $\left, \left\, Mx\right\, _2 - \left\, x\right\, _2 \ < \varepsilon \left\, x\right\, _2$

with probability $1 - \delta$ if the quantity of rows
:$m = (4a)^ \varepsilon^ \log 1/\delta + (2ae)\varepsilon^(\log 1/\delta)^c.$

In particular, if the entries of $T$ are $\pm 1$ can get
: $m = O\left(\varepsilon^\log1/\delta + \varepsilon^\left(\frac\log1/\delta\right)^c\right)$

which matches the Johnson–Lindenstrauss lemma of $m = O\left(\varepsilon^\log1/\delta\right)$ when $\varepsilon$ is small.

Block face-splitting product

According to the definition of V. Slyusar the block face-splitting product of two partitioned matrices with a given quantity of rows in blocks

: \mathbf =
\left \begin
\mathbf_ & \mathbf_ \\
\hline
\mathbf_ & \mathbf_
\end
\right
,\quad
\mathbf =
\left \begin
\mathbf_ & \mathbf_ \\
\hline
\mathbf_ & \mathbf_
\end
\right,

can be written as :

:$\mathbf$bull \mathbf =
\left \begin
\mathbf_ \bull \mathbf_ & \mathbf_ \bull \mathbf_ \\
\hline
\mathbf_ \bull \mathbf_ & \mathbf_ \bull \mathbf_
\end
\right.

The transposed block face-splitting product (or Block column-wise version of the Khatri–Rao product) of two partitioned matrices with a given quantity of columns in blocks has a view:

:$\mathbf$ast \mathbf =
\left \begin
\mathbf_ \ast \mathbf_ & \mathbf_ \ast \mathbf_ \\
\hline
\mathbf_ \ast \mathbf_ & \mathbf_ \ast \mathbf_
\end
\right

Main properties

#Transpose:
#:$\left(\mathbf$ast \mathbf \right)^\textsf = \textbf^\textsf bull\mathbf^\textsfVadym Slyusar
New Matrix Operations for DSP
(Lecture). April 1999. – DOI: 10.13140/RG.2.2.31620.76164/1

Applications

The Face-splitting product and the Block Face-splitting product used in the tensor-matrix theory of digital antenna arrays. These operations are also used in:
* Artificial Intelligence and Machine learning systems to minimization of convolution and tensor sketch operations,
* A popular Natural Language Processing models, and hypergraph models of similarity,
* Generalized linear array model in statistics
* Two- and multidimensional P-spline approximation of data,
* Studies of genotype x environment interactions.Johannes W. R. Martini, Jose Crossa, Fernando H. Toledo, Jaime Cuevas. On Hadamard and Kronecker products in covariance structures for genotype x environment interaction.//Plant Genome. 2020;13:e20033. Page 5

/ref>

See also

* Kronecker product
* Hadamard product (matrices), Hadamard product

Notes

References

*
*
*
*

{{DEFAULTSORT:Khatri-Rao Product
Matrix theory