CNNVR: Linear Classification

机器学习

---

Score function

Define a linear classification score function:

$$f(x_i, W, b) = Wx_i + b$$
where $W$ is weight, $b$ is bias, and $x_i$ is input image.

Loss function

Hinge loss (SVM) is defined as:

$$L_i = \sum_{j \not= y_i} \max{(0, f(x_i, W)_j - f(x_i, W)_{y_i} + \Delta)}$$

Cross-Entropy loss (Softmax) is defined as:

$$L_i = -\log{\left( \frac{e^{f_{y_i}}}{\sum_j e^{f_j}} \right)}$$

L2 regularization

Define complexity as

$$R(W) = \sum_k \sum_l W_{k,l}^2$$
and total loss is
$$L = \frac{1}{N} \sum_i L_i + \lambda R(W)$$
where $\lambda$ is called regularization strength.