Question

If X is a binomial random variable, for what value of Theta the probability is maximum?

Answer 1

$$X\sim bin(k;n,\theta)$$

$$f(k)=P(X=k)=\binom{n}{k}\theta^k(1-\theta)^{n-k}$$

We can look at the ratio of successive outcomes

$$r={P(X=k+1) \over P(X=k)}$$

When $r>1$ then $P(X=k+1)>P(X=k)$

When $r<1$ then $P(X=k+1)<P(X=k)$

Maximum of the distribution occurs when $r$ switches from being greater than $1$ to less than $1.$

$$r={P(X=k+1) \over P(X=k)}={\binom{n}{k+1}\theta^{k+1}(1-\theta)^{n-(k+1)} \over \binom{n}{k}\theta^k(1-\theta)^{n-k}}=$$

$$={n-k \over k+1}\cdot{\theta \over1-\theta}$$

 What value of k results in $r\leq1?$

$${n-k \over k+1}\cdot{\theta \over1-\theta}\leq1$$

$$\theta(n-k)\leq(1-\theta)(k+1)$$

$$n\theta-k\theta\leq k+1-k\theta-\theta$$

$$k\geq n\theta-(1-\theta)$$

 Max probability is the smallest interger value of $k\geq n\theta-(1-\theta)$

Likelihood function

$$L(\theta|n, x)=\binom{n}{x}\theta^x(1-\theta)^{n-x}=$$ $$={n! \over x!(n-x)!}\theta^x(1-\theta)^{n-x}$$

Log-likelihood function 

$$\ln{L(\theta|n, x)}=\ln{\bigg({n! \over x!(n-x)!}\bigg)}+x\ln{\theta}+(n-x)\ln({1-\theta})$$

$${d({L(\theta|n, x)}) \over dp}=0+x\cdot{1 \over \theta}+(n-x)\cdot(-{1 \over 1-\theta})=0$$

$$x(1-\theta)-(n-x)\theta=0$$

$$x-x\theta-n\theta+x\theta=0$$

$$x-n\theta=0$$

$$\theta={x \over n}$$

The maximum likelihood estimate for $\theta$ is just the average.