@nrailgun 2016-03-08T13:59:17.000000Z 字数 585 阅读 2126

MLE, MAP, and Beysian inference

机器学习

Maximum likelihood estimation

Likelihood function: $L(\theta) = \prod_i p(X_i, \theta)$ , estimate the $\theta$ maximizing $L$ .

Example: Maximum likelihood estimation of Bern

B (x ∣ u) u x (1 - u) 1 - x

$B(x \mid u) u^x (1 - u)^{1-x}$

L (X ∣ u) = \prod i u X i (1 - u) 1 - X i

$L(X \mid u) = \prod_i u^{X_i} (1 - u)^{1 - X_i}$

and $\hat u = \frac S N$ is the solution.

p (θ ∣ x) = p ( x ∣ θ ) \times p ( θ ) p ( x )

$p(\theta \mid x) = \frac {p(x \mid \theta) \times p(\theta)} {p(x)}$

Then MAP is

θ^M A P = arg max θ p (x ∣ θ) g (θ)

$\hat \theta_{\rm MAP} = \arg \max_\theta p(x \mid \theta) g(\theta)$
where

g(θ) $g(\theta)$ is the distribution of

θ $\theta$ .