MLE, MAP, and Beysian inference

机器学习

---

Maximum likelihood estimation

Likelihood function: $L(\theta) = \prod_i p(X_i, \theta)$, estimate the $\theta$ maximizing $L$.

Example: Maximum likelihood estimation of Bern

$$B(x \mid u) u^x (1 - u)^{1-x}$$

$$L(X \mid u) = \prod_i u^{X_i} (1 - u)^{1 - X_i}$$

and $\hat u = \frac S N$ is the solution.

Maximum a posteriori estimation

$$p(\theta \mid x) = \frac {p(x \mid \theta) \times p(\theta)} {p(x)}$$

Then MAP is

$$\hat \theta_{\rm MAP} = \arg \max_\theta p(x \mid \theta) g(\theta)$$
where $g(\theta)$ is the distribution of $\theta$.

Bayesian Inference