Kernel-independent Component Analysis

In statistics, kernel-independent component analysis (kernel ICA) is an efficient algorithm for independent component analysis which estimates source components by optimizing a ''generalized variance'' contrast function, which is based on representations in a reproducing kernel Hilbert space. Those contrast functions use the notion of mutual information as a measure of statistical independence.

Main idea

Kernel ICA is based on the idea that correlations between two random variables can be represented in a reproducing kernel Hilbert space (RKHS), denoted by $\mathcal$, associated with a feature map $L_x: \mathcal \mapsto \mathbb$ defined for a fixed $x \in \mathbb$. The $\mathcal$-correlation between two random variables $X$ and $Y$ is defined as

: $\rho_(X,Y) = \max_ \operatorname( \langle L_X,f \rangle, \langle L_Y,g \rangle)$

where the functions $f,g: \mathbb \to \mathbb$ range over $\mathcal$ and

: $\operatorname( \langle L_X,f \rangle, \langle L_Y,g \rangle) := \frac$

for fixed $f,g \in \mathcal$. Note that the reproducing property implies that $f(x) = \langle L_x, f \rangle$ for fixed $x \in \mathbb$ and $f \in \mathcal$. It follows then that the $\mathcal$-correlation between two independent random variables is zero.

This notion of $\mathcal$-correlations is used for defining ''contrast'' functions that are optimized in the Kernel ICA algorithm. Specifically, if $\mathbf := (x_) \in \mathbb^$ is a prewhitened data matrix, that is, the sample mean of each column is zero and the sample covariance of the rows is the $m \times m$ dimensional identity matrix, Kernel ICA estimates a $m \times m$ dimensional orthogonal matrix $\mathbf$ so as to minimize finite-sample $\mathcal$-correlations between the columns of $\mathbf := \mathbf \mathbf^$.

References

{{Statistics-stub

Statistical algorithms