Question

Show that

Ec[logƒ(Y)] = —— log(26²) —

7²+(m_µ)²

202

(c) Show that Kullback-Leibler divergence of f (y) with respect to g(y) is given by

D(glf) EG[logg (Y)]-EG[log f(Y)]

0² t² + (m-μ)² - 1}

Answer 1

The Normal distribution of generating data g(y) and specified model f(y) is-

g(y)~N($m,t^2$)

f(y)~N$(\mu,o^2)$

The probability density function is-

$Y=\dfrac{1}{\sqrt{2\pi}}e^{\frac{-x^2}{2}}$

$logY=log\dfrac{1}{\sqrt{2\pi}}+loge^{\frac{-x^2}{2}}=log\dfrac{1}{\sqrt{2\pi}}+\dfrac{-x^2}{2}~~~~~-(1)$

So $E(logY)=\int_{0}^{r}xlogYdx$

     $=\int_{0}^{r} xlog\dfrac{1}{\sqrt{2\pi}}+\dfrac{-x^3}{2}dx\\[9pt] =\dfrac{x^2}{2}log\dfrac{1}{\sqrt{2\pi}}|_0^r-\dfrac{x^4}{8}|_0^r\\[9pt]=\dfrac{r^2}{2}log\dfrac{1}{\sqrt{2\pi}}-\dfrac{r^4}{8}\\[9pt]=-log(2rt^2)-1$

Hence $E(logY)= -log(2rt^2)-1$