Dimensionality Reduction

机器学习

---

SVD

$$A_{[m \times n]} = U_{[m \times r]} \Sigma_{[r \times r]} V_{[n \times r]}^T$$
where $r$ is rank of $A$, $U$ and $V$ are column orthonormal ($U^T U = I$, $V^T V = I$), and $\Sigma$ a is diagonal matrix. $U$, $\Sigma$, and $V$ are unique.

Dimensionality Reduction with SVD

$B = USV^T$ is a solution to $\min_B \|A - B\|_F$, where $s_{ii} = \Sigma_{ii} (i = 1, 2, \dots, k)$ else $s_{ii} = 0$. A rule-of-a-thumb: take $80\% \sim 90\%$ eigenvalues.