EM

机器学习

EM Algorithm

Input: Observable variable $Y$, latent variable $Z$, joint-distribution $P(Y, Z \mid \theta)$, conditional distribution $P(Z \mid Y, \theta)$;
Output: Model parameter $\theta$.

1. Pick initial value $\theta^{(0)}$;
2. E: Calculate
   $$Q(\theta, \theta^{(0)}) = E_Z \left[ \log{P(Y,Z \mid \theta)} \mid Y, \theta^{(i)} \right] = \sum_Z \log{P(Y, Z \mid \theta) P(Z \mid Y, \theta^{(i)})}$$
3. M: Find $\theta$ maximize $Q$ and let
   $$\theta^{(i+1)} = \mathrm{argmax}_\theta Q(\theta, \theta^{(i)})$$
4. Repeat step E and step M until converge.