Question

3. a) Three messages A, B, C have probabilities 0.4, 0.3 and 0.3 respectively, of their

occurrences. If total 100 massages are sent out, calculate the total information (Itotal)

carried by them. Also calculate the entropy of the signal.

Answer 1

Probability of message A, $p_1=0.4$

Probability of message A, $p_2=0.3$

Probability of message A, $p_3=0.3$

Total messsages L=100

Total information

$I_{total}=p_1Llog_2\dfrac{1}{p_1}+p_2Llog_2\dfrac{1}{p_2}+p_3Llog_2\dfrac{1}{p_3}$

$=(0.4)(100)log_2\dfrac{1}{0.4}+(0.3)(100)log_2\dfrac{1}{0.3}+(0.3)(100)log_2\dfrac{1}{0.3}$

$=40\times1.32+30\times 1.735+30\times 1.735$

$=52.8+52.06+52.06=156.92$ bits

Entropy $=\dfrac{I_{total}}{L}=\dfrac{156.92}{100}=1.5692 bits$