Answer to Question #67183 in Engineering for Jeff

Question

Consider the following problem related to entropy maximisation in information theory. For ease of solution you can solve the problem ignoring the non-negativity constraints on p and q and then verify that they are satisfied.
min p log P + q log q
subject to p+q =1
p,q.>=0
i. Solve the primal problem directly
ii. Obtain the dual problem and solve it. Verify it gives the same answer as the primal problem.
iii. By doing an internet search or looking up textbooks, explain why this problem would be
called an “entropy maximisation” problem.