二项分布 均值 方差 推导

机器学习 PRML

---

二项分布：

$$\mathrm{Bin}(m \mid N, \mu) = \binom{N}{m}\mu^m(1-\mu)^{N-m}$$

有两种方法求解

方法一

第一种方法使用了均值的加和规则：$\mathrm{E}[x+y] = \mathrm{E}[x] + \mathrm{E}[y]$，$\mathrm{var}[x+y] = \mathrm{var}[x] + \mathrm{var}[y]$

$$\begin{align*} \mathrm{E}[x+y] & = \int\int (x+y)p(x)p(y)\mathrm{d}x\mathrm{d}y \\ & = \int\int xp(x)p(y)\mathrm{d}x\mathrm{d}y + \int\int yp(x)p(y)\mathrm{d}x\mathrm{d}y \\ & = \int xp(x)\mathrm{d}x + \int yp(y)\mathrm{d}y \\ & = \mathrm{E}[x] + \mathrm{E}[y] \\ \mathrm{E}[xy] & = \int\int xyp(x)p(y)\mathrm{d}x\mathrm{d}y \\ & = \int xp(x)\mathrm{d}x \cdot \int yp(y)\mathrm{d}y \\ & = \mathrm{E}[x] \cdot \mathrm{E}[y] \\ \mathrm{var}[x+y] & = \mathrm{E}[(x+y)^2] - \mathrm{E}^2[x+y] \\ & = \mathrm{E}[x^2] + \mathrm{E}[y^2] + 2\mathrm{E}[xy] - \mathrm{E}^2[x] - \mathrm{E}^2[y] - 2\mathrm{E}[x]\mathrm{E}[y] \\ & = \mathrm{E}[x^2] + \mathrm{E}[y^2] - \mathrm{E}^2[x] - \mathrm{E}^2[y] \\ & = \mathrm{var}[x] + \mathrm{var}[y] \\ \mathrm{var}[xy] & = \mathrm{E}[x^2y^2] - \mathrm{E}^2[xy] \\ & = \mathrm{E}[x^2]\mathrm{E}[y^2] - \mathrm{E}^2[x]\mathrm{E^2}[y] \\ & = (\mathrm{var}[x]+\mathrm{E}^2[x])(\mathrm{var}[y] + \mathrm{E}^2[y]) - \mathrm{E}^2[x]\mathrm{E^2}[y] \\ & = \mathrm{var}[x]\mathrm{var}[y] + \mathrm{var}[x]\mathrm{E}^2[y] + \mathrm{var}[y]\mathrm{E}^2[x] \end{align*}$$
二项式分布可由伯努力分布加和构成
$$x \sim \mathrm{bern}(x\mid \mu) = \mu^x(1-\mu)^{1-x}$$
二项式分布$m\sim \mathrm{Bin}(m\mid N,\mu)$，事件以概率$\mu$在$N$次实验中发生$m$次的概率，则$m$与伯努力实验的关系为
$$m = x_1 + x_2 + \cdots + x_N = \sum_{i=0}^N x_i$$
故
$$\begin{align*} \mathrm{E}[m] & = \mathrm{E}[x_1+x_2+\cdots+x_N] \\ & = \mathrm{E}[x_1] + \mathrm{E}[x_2] + \cdots + \mathrm{E}[x_N] \\ & = N\mu \\ \mathrm{var}[m] & = N\mu(1-\mu) \end{align*}$$

方法二

$$\begin{align*} \mathrm{E}[m] & = \sum_{m=0}^N m\binom{N}{m}\mu^m(1-\mu)^{N-m} \\ & = \sum_{m=1}^N m\cdot \frac{N!}{(N-m)!m!} \mu^m(1-\mu)^{N-m} \\ & = \sum_{m=1}^N\frac{N!}{(N-m)!(m-1)!} \mu^m(1-\mu)^{N-m} \\ & = N\mu \cdot \sum_{m=1}^N\frac{(N-1)!}{(N-m)!(m-1)!} \mu^{m-1}(1-\mu)^{N-m} \\ & = N\mu \cdot (\mu + 1 -\mu)^{N-1} \\ & = N\mu \\ \mathrm{var}[m] & = \mathrm{E}[m^2] - \mathrm{E}^2[m] \\ & = N\mu\cdot \left( \sum_{m=1}^N m \cdot \frac{(N-1)!}{(N-m)!(m-1)!} \mu^{m-1}(1-\mu)^{N-m} - \sum_{m=1}^N m\cdot \frac{N!}{(N-m)!m!} \mu^m(1-\mu)^{N-m} \right) \\ \end{align*}$$