感知机

机器学习

---

Perceptron model

Let $f(x)$ denotes perceptron:

$$f(x) = sign(w \centerdot x + b),$$
where $w \in \mathbf{R}^n$ is weight, $b \in \mathbf{R}$ is bias, and
$$sign(x) = \begin{cases} +1, & x \ge 0 \\ -1, & x \lt 0 \end{cases}.$$

Perceptron learning strategy

The loss function of perceptron $sign(w \centerdot x + b)$ is defined as

$$L(w, b) = -\sum_{x_i \in M} y_i(w \centerdot x_i + b),$$
where $M$ is the set of incorrectly classified points.

Perceptron learning algorithm

Gradient of loss $L(w, b)$ is given by

$$\nabla_w L(w,b) = -\sum_{x_i \in M} y_ix_i,$$
and
$$\nabla_bL(w,b) = -\sum_{x_i \in M} y_i.$$

The selection of initial value of $w$ and $b$ does effect the solution. There are many solutions. If the dataset is not linearly separable, perceptrons will fail to converge.