Principal Component Analysis (PCA) (with MATLAB + Simulator)

PCA for Signal Processing - Example

We have 2 signals sampled at 3 time points:

X = \begin{bmatrix} 2 & 0 \\ 0 & 1 \\ 1 & 2 \end{bmatrix}

Each row is a time sample, each column is a signal.

Compute column means:

\bar{x}_1 = \frac{2+0+1}{3} = 1, \quad \bar{x}_2 = \frac{0+1+2}{3} = 1

Subtract the mean from each column:

X_\text{centered} = \begin{bmatrix} 2-1 & 0-1 \\ 0-1 & 1-1 \\ 1-1 & 2-1 \end{bmatrix} = \begin{bmatrix} 1 & -1 \\ -1 & 0 \\ 0 & 1 \end{bmatrix}

C = \frac{1}{N-1} X_\text{centered}^T X_\text{centered}

Here, \(N = 3\):

C = \frac{1}{2} \begin{bmatrix} 1 & -1 & 0 \\ -1 & 0 & 1 \end{bmatrix} \begin{bmatrix} 1 & -1 \\ -1 & 0 \\ 0 & 1 \end{bmatrix} = \frac{1}{2} \begin{bmatrix} 2 & -1 \\ -1 & 2 \end{bmatrix} = \begin{bmatrix} 1 & -0.5 \\ -0.5 & 1 \end{bmatrix}

Solve \( C v = \lambda v \). The eigenvalues are:

\lambda_1 = 1.5, \quad \lambda_2 = 0.5

Corresponding normalized eigenvectors (principal components):

v_1 = \frac{1}{\sqrt{2}} \begin{bmatrix} 1 \\ -1 \end{bmatrix}, \quad v_2 = \frac{1}{\sqrt{2}} \begin{bmatrix} 1 \\ 1 \end{bmatrix}

Y = X_\text{centered} V = \begin{bmatrix} 1 & -1 \\ -1 & 0 \\ 0 & 1 \end{bmatrix} \begin{bmatrix} \frac{1}{\sqrt{2}} & \frac{1}{\sqrt{2}} \\ -\frac{1}{\sqrt{2}} & \frac{1}{\sqrt{2}} \end{bmatrix} = \begin{bmatrix} \sqrt{2} & 0 \\ -\frac{1}{\sqrt{2}} & -\frac{1}{\sqrt{2}} \\ \frac{1}{\sqrt{2}} & \frac{1}{\sqrt{2}} \end{bmatrix}

The first principal component (\(v_1\)) captures the most variance (\(\lambda_1 = 1.5\)).
The second component (\(v_2\)) captures less variance (\(\lambda_2 = 0.5\)).
By keeping only \(v_1\), we can reduce dimensionality and remove noise.

This is the complete PCA process for a small signal dataset with math calculations.

Applications: