1. The problem is to understand and use the entropy formula in information theory. 2. The entropy $H$ of a discrete random variable with possible outcomes $x_1, x_2, ..., x_n$ and corresponding probabilities $p_1, p_2, ..., p_n$ is given by the formula: $$H = -\sum_{i=1}^n p_i \log_2 p_i$$ 3. Important rules: - Each probability $p_i$ must satisfy $0 \leq p_i \leq 1$ and $\sum_{i=1}^n p_i = 1$. - The logarithm is base 2, which measures entropy in bits. - If $p_i = 0$, the term $p_i \log_2 p_i$ is defined as 0 because $\lim_{p \to 0} p \log p = 0$. 4. To calculate entropy: - Identify all possible outcomes and their probabilities. - Compute each term $-p_i \log_2 p_i$. - Sum all these terms to get the total entropy. 5. Example: For a fair coin toss, $p_1 = p_2 = 0.5$. $$H = - (0.5 \log_2 0.5 + 0.5 \log_2 0.5) = - (0.5 \times (-1) + 0.5 \times (-1)) = 1 \text{ bit}$$ This means the uncertainty or information content of a fair coin toss is 1 bit.