ISYE 6412 HW #2

Name: Haoming Jiang

##Problem #1
###(a)
$R_{\delta_c}(\theta) = E(L(\theta,\delta_c(X))) = r (2c\sigma) - E(I(\theta\in [\bar{X}_n-c\sigma,\bar{X}_n+c\sigma ])) \\ =r (2c\sigma) - P(\bar{X}_n -\theta \in [-c\sigma,c\sigma ]) \\= r (2c\sigma) - P(\frac{\bar{X}_n-\theta}{\sqrt{n}\sigma} \in [-\sqrt{n}c,+\sqrt{n}c ])$, 
with the fact that $\frac{\bar{X}_n-\theta}{\sqrt{n} \sigma} \sim N(0,1)$,$R_{\delta_c}(\theta) = (2c\sigma) - P(Z \in [-\sqrt{n}c,+\sqrt{n}c ]) = 2cr\sigma-2\Phi(c\sqrt{n}) + 1$
###(b)
$\frac{d}{dc}R_{\delta_c}(\theta) = 2r\sigma - 2f(c\sqrt{n})*\sqrt{n}$, $f$ is the density function of $Z$. So $\frac{d}{dc}R_{\delta_c}(\theta) = 2r\sigma-\frac{2\sqrt{n}}{\sqrt{2\pi}}e^{-nc^2/2}$
###(c)
Since $e^{-nc^2/2} \leq 1$, the derivative is positive if $r\sigma > \sqrt{n}/\sqrt{2\pi}$. 
###(d)
When $r\sigma \leq \sqrt{n}/\sqrt{2\pi}$. The risk is minimized when the derivative is 0. $\frac{d}{dc}R_{\delta_c}(\theta) = 2r\sigma - \frac{2\sqrt{n}}{\sqrt{2\pi}}e^{-nc^2/2} = 0$ leads to $-nc^2/2 = log(\sqrt{\frac{2\pi}{n}}r\sigma)$. So $c_{opt} = \sqrt{\frac{2}{n}log(\sqrt{\frac{2\pi}{n}}r\sigma)}$
###(e)
In the case $c_{opt}=z_{\alpha/2}/\sqrt{n}$. So $log(\sqrt{\frac{2\pi}{n}}r^*\sigma) = z_{\alpha/2}^2/2$. Which means $r^* = \frac{\sqrt{n}}{\sqrt{2\pi}\sigma}e^{z_{\alpha/2}^2/2}$
##Problem #2
###(a)
$S:$ ${0,1}$
$\Omega:$ $P(head) = 1/(2+\theta),P(tail) = 1 - 1/(2+\theta),\ (\theta \in \{0,1\})$  
$D:{d_0,d_1}$ 
$L_s(d)= I(d\ is\ wrong)$ 
###(b)
$R_\delta(\theta) = E_\theta(I(d\ is\ wrong)) = P_\theta(\delta \ reaches\ wrong\ decision)$
$R_{\delta_1}(\theta) = \theta$
$R_{\delta_2}(\theta) = 1-\theta$
$R_{\delta_3}(\theta) = 1/(2+\theta)$
$R_{\delta_4}(\theta) = 1-1/(2+\theta)$
###(c)
(I) $\delta_1,\ \delta_2,\ \delta_4$ are admissible
(II) $r_{\delta_1}(\pi) = 0.1$
$r_{\delta_2}(\pi) = 0.9$
$r_{\delta_3}(\pi) = 29/60$
$r_{\delta_4}(\pi) = 31/60$
$\delta_1$ is Bayes.
(III)$r_{\delta_1}(\pi) = 0.6$
$r_{\delta_2}(\pi) = 0.4$
$r_{\delta_3}(\pi) = 0.4$
$r_{\delta_4}(\pi) = 0.6$
$\delta_2$ and $\delta_3$ are Bayes.
###(d)
Assume for some prior distribution, it is also Bayes. Assume that prior distribution is $P_\pi(1/3)=\alpha$. Then,
$r_{\delta_1}(\pi) = \alpha$
$r_{\delta_2}(\pi) = 1-\alpha$
$r_{\delta_3}(\pi) = (1-\alpha)/2 + \alpha/3$
$r_{\delta_4}(\pi) = (1-\alpha)/2 + 2\alpha/3$
And $r_{\delta_1}(\pi)$ is the smallest, if and only if $\alpha \leq \frac{3}{7}$
##Problem #3
$R_\delta(f_i) = \int L(f_i(x),\delta(x))f_i(x)dx$
$r_\delta(\pi) = R_\delta(f_0) \pi(f_0) + R_\delta(f_1) (1-\pi(f_0))\\
= \int (L(f_0(x),\delta(x))f_0(x)\pi(f_0)+L(f_1(x),\delta(x))f_1(x)\pi(f_1))dx$.
For convenience, $(L(f_0(x),\delta(x))f_0(x)\pi(f_0)+L(f_1(x),\delta(x))f_1(x)\pi(f_1))$ is denoted as $h(x)$
For a certain $x$, if $\delta(x) = f_0$, $h(x) = w_1f_1(x)\pi(f_1)$, if $\delta(x) = f_1$, $h(x) = w_0f_0(x)\pi(f_0)$
In order to minimize $r_\delta(\pi)$. 
$\delta(x) = f_0$, when $w_1f_1(x)\pi(f_1) > w_0f_0(x)\pi(f_0) \Leftrightarrow \frac{f_1(x)}{f_0(x)} > \frac{w_0\pi(f_0)}{w_1\pi(f_1)}$
$\delta(x) = f_1$, when $w_1f_1(x)\pi(f_1) < w_0f_0(x)\pi(f_0) \Leftrightarrow \frac{f_1(x)}{f_0(x)} < \frac{w_0\pi(f_0)}{w_1\pi(f_1)}$
$\delta(x) = f_1 or f_2$, when $w_1f_1(x)\pi(f_1) = w_0f_0(x)\pi(f_0) \Leftrightarrow \frac{f_1(x)}{f_0(x)} =  \frac{w_0\pi(f_0)}{w_1\pi(f_1)}$
##Problem #4
###(a)
This follows at once from our discussion in class that a procedure $\delta$ is Bayes relative to $\pi$ if and only if, for every $x$; it assigns a decision $\delta(x)$ which minimizes (over $D$) $h_\pi^*(x,d) = \int_\Omega L(\theta,d)p_\theta(x)\pi(\theta)d\theta$
or equivalently, to minimize $h_\pi(x,d)  = \int_\Omega L(\theta,d) \frac{p_\theta(x)\pi(\theta)}{m(x)}d\theta= \int_\Omega L(\theta,d) \pi(\theta|x)d\theta$
###(b)
When $r=2$, we have  $h_\pi(x,d) = E(\theta^2|x) - 2E(\theta|x)d+d^2$, which is minimized at $d=E(\theta|x)$
###(c)
When $r=1$, we have  
$h_\pi(x,d) \\
= \int_\Omega |\theta-d| \pi(\theta|x)d\theta \\
= \int_{\theta>d} (\theta-d) \pi(\theta|x)d\theta+\int_{\theta<d} (-\theta+d) \pi(\theta|x)d\theta
$
$\frac{\partial h_\pi(x,d)}{ \partial d} \\
= \int_{\theta>d} - \pi(\theta|x)d\theta+\int_{\theta<d}  \pi(\theta|x)d\theta \\
= P(\theta<d|x)-P(\theta>d|x)$, which is monotonically increasing and reaches 0 when $P(\theta<d|x)-P(\theta>d|x) = 0$. In orther words, $h_\pi(x,d)$ is minimized when $d$ is the median of the posterior distribution.