Subjects Linear Algebra

Quadratic Form Expansion

Step-by-step solutions with LaTeX - clean, fast, and student-friendly.

Search Solutions

Quadratic Form Expansion


1. Let's start by stating the problem: We want to verify why $ (y - Xw)^T (y - Xw) = \|y\|^2 - 2y^T X w + w^T X^T X w $. 2. Recall that the squared norm of a vector $a$ is $a^T a$. Here $a = y - Xw$, so $\|y - Xw\|^2 = (y - Xw)^T(y - Xw)$. 3. Expand the product using distributive property for transposes: $$ (y - Xw)^T (y - Xw) = y^T y - y^T X w - (X w)^T y + (X w)^T (X w) $$ 4. Since $ (X w)^T = w^T X^T $ and $ y^T X w $ is a scalar (a number), it equals its transpose, so $ y^T X w = (X w)^T y $. 5. Substitute to get: $$ y^T y - y^T X w - y^T X w + w^T X^T X w = y^T y - 2 y^T X w + w^T X^T X w $$ 6. Recognize that $ y^T y = \|y\|^2 $, which is the squared norm of $y$. 7. Therefore, the original expression equals $$ \|y\|^2 - 2 y^T X w + w^T X^T X w $$ This confirms the identity given in the problem statement.