Answer to Question #191316 in Statistics and Probability for tejaswini

Question

Let (Z₁, Y₁), . . . , (Z_n, Y_n) be generated as follows:

Z_i ∼ Bernoulli(p)

Y_i ∼ { N(0, 1) if Z_i = 0 , N(5, 1) if Z_i = 1

(a) Assume we do not observe the Z_i ’s. Write the pdf f(y) of Y as a mixture of two normal distribution pdf. (Use the notation φ(·) which is the standard normal pdf.)

(b) Write down the likelihood function for p (without Zi ’s).

(c) Write down the complete likelihood function for p (assuming the Zi ’s are observed).

(d) Find the maximum likelihood estimation of p using the likelihood from (c). 

Answer 1

we see that N(0,1) and N(5,1) with probability 1-p and p respectively.

Thus density is $R(y)=(1-p)\dfrac{1}{\sqrt{2\pi}}e^{-\frac{y^2}{2}}+p\dfrac{1}{\sqrt{2\pi}}e^{-\frac{(y-5)^2}{2}}$

(b) The likelihood function from the above density, the likelihood function without the z's is:

$L_n(P)= ((1-p)\dfrac{1}{\sqrt{2\pi}}e^{-\frac{y_i^2}{2}}+p\dfrac{1}{\sqrt{2\pi}}e^{-\frac{(y_i-5)^2}{2}}$

(c) Note that if the $z_i$ is known, then the likelihood for p depend exclusively on $Z_i$ as y/z has a normal distribution independent of p.

Thus the complete likelihood function is-

$L_n(P)= ((1-p)\dfrac{1}{\sqrt{2\pi}}e^{-\frac{y_i^2}{2}}+p\dfrac{1}{\sqrt{2\pi}}e^{-\frac{(y_i-5)^2}{2}}$

.

(d) As we know, The likelihood for p depends exclusively on $Z_i's \text{ as } Y|Z$ has a normal distribution of p.

Thus the likelihood of p now is

 $L_n(p)=f_n(Y,Z|p)=f_n(Z|p)\times f_n(Y|Z),$ where $f_n$ is the joint density of the sample. Here $f_n(Y|Z)$ is independent of p and thus

     $L_n(p) \propto f_n(Z|p)$

But now, note that $Z_1,Z_2,...,Z_n$

$\Rightarrow f_n(Z|p)=\Pi_{i=1}^n P^{Z_i}(1-p)^{1-Z_i}$

'

         $=p^{\sum Z_i}(1-p)^{n-\sum Z_i}$

Thus the likelihood function is-

$L_n(p) \propto p^{\sum Z_i}(1-p)^{n-\sum Z_i}$

Taking log of above equation and we get-

$LogL_n=logP\sum z_i+(1-\sum z_i)log(1-p)$

Thus $\dfrac{dL_n}{dp}=0$

$=\dfrac{\sum z_i}{p}-\dfrac{1}{1-p}\sum (n-\sum z_i)=0$

$=\dfrac{1}{P}\sum z_i$

$=\dfrac{1}{1-P}(n-\sum z_i)=n$

$\Rightarrow \hat{p}=\dfrac{1}{n}\sum z_i=\bar{z}$

This is the required MLE.