Data Whitening¶

A whitening transformation is a decorrelation transformation that transforms a set of random variables into a set of new random variables with identity covariance (uncorrelated with unit variances).

In particular, suppose a random vector has covariance $\mathbf{C}$ , then a whitening transform $\mathbf{W}$ is one that satisfy:

$\mathbf{W}^T \mathbf{C} \mathbf{W} = \mathbf{I}$

Note that $\mathbf{W}$ is generally not unique. In particular, if $\mathbf{W}$ is a whitening transform, so is any of its rotation $\mathbf{W} \mathbf{R}$ with $\mathbf{R}^T \mathbf{R} = \mathbf{I}$ .

Whitening¶

The package uses Whitening defined below to represent a whitening transform:

immutable Whitening{T<:FloatingPoint}
    mean::Vector{T}     # mean vector (can be empty to indicate zero mean), of length d
    W::Matrix{T}        # the transform coefficient matrix, of size (d, d)
end

An instance of Whitening can be constructed by Whitening(mean, W).

There are several functions to access the properties of a whitening transform f:

indim(f)¶: Get the input dimension, i.e d.

outdim(f)¶: Get the out dimension, i.e d.

mean(f)¶

Get the mean vector.

Note: if f.mean is empty, this function returns a zero vector of length d.

transform(f, x)¶: Apply the whitening transform to a vector or a matrix with samples in columns, as $\mathbf{W}^T (\mathbf{x} - \boldsymbol{\mu})$ .

Data Analysis¶

Given a dataset, one can use the fit method to estimate a whitening transform.

fit(Whitening, X; ...)¶

Estimate a whitening transform from the data given in X. Here, X should be a matrix, whose columns give the samples.

This function returns an instance of Whitening.

Keyword Arguments:

name description default

regcoef

The regularization coefficient. The covariance will be regularized as follows when regcoef is positive:

C + (eigmax(C) * regcoef) * eye(d)

zero(T)

mean

The mean vector, which can be either of:

0: the input data has already been centralized
nothing: this function will compute the mean
a pre-computed mean vector

nothing

Note: This function internally relies on cov_whiten to derive the transformation W. The function cov_whiten itself is also a useful function.

cov_whitening(C)¶

Derive the whitening transform coefficient matrix W given the covariance matrix C. Here, C can be either a square matrix, or an instance of Cholesky.

Internally, this function solves the whitening transform using Cholesky factorization. The rationale is as follows: let $\mathbf{C} = \mathbf{U}^T \mathbf{U}$ and $\mathbf{W} = \mathbf{U}^{-1}$ , then $\mathbf{W}^T \mathbf{C} \mathbf{W} = \mathbf{I}$ .

Note: The return matrix W is an upper triangular matrix.

cov_whitening(C, regcoef): Derive a whitening transform based on a regularized covariance, as C + (eigmax(C) * regcoef) * eye(d).

In addition, the package also provides cov_whiten!, in which the input matrix C will be overwritten during computation. This can be more efficient when C is no longer used.

invsqrtm(C)¶: Compute inv(sqrtm(C)) through symmetric eigenvalue decomposition.