1. **Problem Statement:** We want to prove the least squares estimates for a model with one categorical variable having $J$ levels. 2. **Objective Function:** The sum of squared residuals is $$S(\beta_0, \ldots, \beta_{J-1}) = \sum_{i=1}^n \left(y_i - \beta_0 - \sum_{j=1}^{J-1} \beta_j D_{ij}\right)^2,$$ where $D_{ij} = 1$ if observation $i$ is in category $j$, and 0 otherwise. The base category is $j=J$ where all $D_{ij}=0$. 3. **Key Insight:** Because the model has only one categorical variable with $J$ levels, the observations can be partitioned into $J$ disjoint groups corresponding to each category. This allows us to split the objective function into sums over each category: $$S = \sum_{j=1}^J \sum_{i: x_i=j} \left(y_i - \beta_0 - \beta_j\right)^2,$$ where for the base category $j=J$, $\beta_J=0$ by definition. 4. **Minimizing for the Base Category $j=J$:** We minimize $$S_J(\beta_0) = \sum_{i: x_i=J} (y_i - \beta_0)^2.$$ Taking derivative and setting to zero: $$\frac{\partial S_J}{\partial \beta_0} = -2 \sum_{i: x_i=J} (y_i - \beta_0) = 0,$$ which implies $$n_J \beta_0 = \sum_{i: x_i=J} y_i \implies \beta_0 = \bar{y}_J,$$ the mean of $y_i$ in the base category. 5. **Minimizing for Other Categories $j=1, \ldots, J-1$:** For each category $j$, minimize $$S_j(\beta_j) = \sum_{i: x_i=j} (y_i - \beta_0 - \beta_j)^2.$$ Taking derivative and setting to zero: $$\frac{\partial S_j}{\partial \beta_j} = -2 \sum_{i: x_i=j} (y_i - \beta_0 - \beta_j) = 0,$$ which gives $$n_j (\beta_0 + \beta_j) = \sum_{i: x_i=j} y_i \implies \beta_j = \bar{y}_j - \beta_0 = \bar{y}_j - \bar{y}_J,$$ where $\bar{y}_j$ is the mean of $y_i$ in category $j$. 6. **Why Splitting the Objective Function is Valid:** Because the categories are mutually exclusive and collectively exhaustive, the residual sum of squares decomposes into independent sums over each category. This means minimizing the total sum of squares is equivalent to minimizing each category's sum separately, allowing us to find closed-form solutions for each $\beta_j$. **Final answer:** $$\boxed{\beta_0 = \bar{y}_J, \quad \beta_j = \bar{y}_j - \bar{y}_J \text{ for } j=1, \ldots, J-1}.$$