Quadratic Form Expansion
1. Let's start by stating the problem: We want to verify why $ (y - Xw)^T (y - Xw) = \|y\|^2 - 2y^T X w + w^T X^T X w $.
2. Recall that the squared norm of a vector $a$ is $a^T a$. Here $a = y - Xw$, so $\|y - Xw\|^2 = (y - Xw)^T(y - Xw)$.
3. Expand the product using distributive property for transposes:
$$ (y - Xw)^T (y - Xw) = y^T y - y^T X w - (X w)^T y + (X w)^T (X w) $$
4. Since $ (X w)^T = w^T X^T $ and $ y^T X w $ is a scalar (a number), it equals its transpose, so $ y^T X w = (X w)^T y $.
5. Substitute to get:
$$ y^T y - y^T X w - y^T X w + w^T X^T X w = y^T y - 2 y^T X w + w^T X^T X w $$
6. Recognize that $ y^T y = \|y\|^2 $, which is the squared norm of $y$.
7. Therefore, the original expression equals
$$ \|y\|^2 - 2 y^T X w + w^T X^T X w $$
This confirms the identity given in the problem statement.