📘 information theory

1. The problem is to understand the formula for the function $H(X,Y)$. 2. Typically, $H(X,Y)$ could represent a function of two variables $X$ and $Y$. Without additional context, a

1. The problem is to understand if Shannon entropy can be calculated using counts without probabilities. 2. Shannon entropy is defined as $$H = -\sum_{i} p_i \log_2 p_i$$ where $p_

1. **Stating the problem:** We are given a list of source encodings (binary strings) and corresponding values of $r$. We need to encode each source message using the given $r$ valu

1. The problem is to complete the channel encoded message by adding a parity bit to the source encoding. 2. The parity bit is added to ensure the total number of 1s in the channel

1. The problem is to understand and use the entropy formula in information theory. 2. The entropy $H$ of a discrete random variable with possible outcomes $x_1, x_2, ..., x_n$ and

1. The problem involves decoding the word CLOCK, which corresponds to the binary code 10101001101000, and using similar logic to find the code for the word OLC. 2. Since the code i