@nrailgun 2016-08-25T07:26:15.000000Z 字数 574 阅读 1945

Auto-Encoder

机器学习

An auto encoder is an unsupervised learning algorithm that applies backpropagation, setting the target values to be equal to the inputs, i.e., $y = x$ .

To impose a "sparsity" constraint on hidden units, we add a KL divergence term to the loss function.

\sum j s (ρ log ρ ρ ^ j + (1 - ρ) log 1 - ρ 1 - ρ ^ j)

$\sum_j^{s} \left( \rho \log \frac \rho {\hat \rho_j} + (1 - \rho) \log \frac {1 - \rho} {1 - \hat \rho_j} \right)$
where

ρ^j=1m∑mi[aj(x)] $\hat \rho_j = \frac 1 m \sum_i^m \left[ a_j(x) \right]$ ,

s $s$ is the number of hidden units.

Our overall loss function is

J s p a r s e (W, b) = J (W, b) + β \sum j s K L (ρ ∥ ρ^j)

$J_{\mathrm{sparse}}(W, b) = J(W, b) + \beta \sum_j^s \mathrm{KL}(\rho \| \hat \rho_j)$

Auto-Encoder

内容目录

选择主题