Subjects probability

Markov Stationary

Step-by-step solutions with LaTeX - clean, fast, and student-friendly.

Search Solutions

Markov Stationary


1. **State the problem:** Find a stationary distribution $\pi$ for the Markov chain with transition matrix $$ P = \begin{pmatrix} \frac{1}{3} & \frac{2}{3} & 0 \\ \frac{1}{6} & \frac{1}{3} & \frac{1}{2} \\ 0 & \frac{1}{3} & \frac{2}{3} \end{pmatrix} $$ This means to find a vector $\pi = (\pi_1, \pi_2, \pi_3)$ such that: $$\pi P = \pi$$ and $$\pi_1 + \pi_2 + \pi_3 = 1.$$ 2. **Write the stationarity equations explicitly:** $$\pi_1 = \pi_1 \cdot \frac{1}{3} + \pi_2 \cdot \frac{1}{6} + \pi_3 \cdot 0,$$ $$\pi_2 = \pi_1 \cdot \frac{2}{3} + \pi_2 \cdot \frac{1}{3} + \pi_3 \cdot \frac{1}{3},$$ $$\pi_3 = \pi_1 \cdot 0 + \pi_2 \cdot \frac{1}{2} + \pi_3 \cdot \frac{2}{3}.$$ 3. **Rewrite the equations:** From the first: $$\pi_1 = \frac{1}{3} \pi_1 + \frac{1}{6} \pi_2 \implies \pi_1 - \frac{1}{3} \pi_1 = \frac{1}{6} \pi_2 \implies \frac{2}{3} \pi_1 = \frac{1}{6} \pi_2 \implies \pi_2 = 4 \pi_1.$$ 4. Second equation: $$\pi_2 = \frac{2}{3} \pi_1 + \frac{1}{3} \pi_2 + \frac{1}{3} \pi_3 \\ \implies \pi_2 - \frac{1}{3} \pi_2 = \frac{2}{3} \pi_1 + \frac{1}{3} \pi_3 \\ \implies \frac{2}{3} \pi_2 = \frac{2}{3} \pi_1 + \frac{1}{3} \pi_3 \\ \implies 2 \pi_2 = 2 \pi_1 + \pi_3 \\ \implies \pi_3 = 2 \pi_2 - 2 \pi_1.$$ 5. Substitute $\pi_2 = 4 \pi_1$ into the last expression: $$\pi_3 = 2(4 \pi_1) - 2 \pi_1 = 8 \pi_1 - 2 \pi_1 = 6 \pi_1.$$ 6. Use normalization condition: $$\pi_1 + \pi_2 + \pi_3 = 1 \\ \pi_1 + 4 \pi_1 + 6 \pi_1 = 1 \\ 11 \pi_1 = 1 \\ \pi_1 = \frac{1}{11}.$$ 7. Find $\pi_2$ and $\pi_3$: $$\pi_2 = 4 \times \frac{1}{11} = \frac{4}{11},$$ $$\pi_3 = 6 \times \frac{1}{11} = \frac{6}{11}.$$ **Final answer:** $$\pi = \left(\frac{1}{11}, \frac{4}{11}, \frac{6}{11}\right).$$