SDM and its Applications to Face Alignment

论文笔记

---

Derivation of SDM

Given an image $d \in \mathfrak R^{m \times 1}$ of $m$ pixels, $d(x) \in \mathfrak R^{p \times 1}$ indexes p landmarks in the image. $h$ is a non-linear feature extraction function (e.g., SIFT), and $h(d(x)) \in \mathfrak R^{128p \times 1}$ in case of SIFT.

During training, we will assume that the correct $p$ landmarks (in our case $66$) are known and refered to as $x_*$. We ran the face detector on the training images to provide an initial configuration of the landmarks $x_0$, which corresponds to an average shape.

In this setting, face alignment can be framed as minimizing the following function over $\Delta x$

$$f(x_0 + \Delta x) = \| h(d(x_0 + \Delta x)) - \phi_* \|_2^2$$
where $\phi_* = h(d(x_*))$.

For derivation purposes, we will assume that $h$ is twice differentiable. We apply a second order Taylor expansion

$$f(x_0 + \Delta x) \approx f(x_0) + J_f(x_0)^T \Delta x + \frac 1 2 \Delta x^T H(x_0) \Delta x$$
where $J_f(x_0) \in \mathfrak R^{p \times 1}$, $H(x_0) \in \mathfrak R^{p \times p}$.

SDM will learn a sequence of generic descent directions $\{ R_k \}$ and bias terms $\{ b_k \}$

$$x_k = x_{k-1} + R_{k-1} \phi_{k-1} + b_{k-1}$$
such that the succession of $x_k$ converges to $x_*$.

Learning for SDM

Minimize

$$\arg \min_{R_k, b_k} \sum_{d^i} \sum_{x^i_k} \| \Delta x^{ki}_* - R_k \phi_k^i - b_k \|^2$$
where $\Delta x^{ki} = x_*^i - x_k^i$.