Newton Iteration Afcb69
1. **Stating the problem:**
We want to find the result of the second iteration using Newton's method for the function
$$f(x) = (x_1^2 + x_2 - 11)^2 + (x_1 + x_2^2 - 7)^2$$
starting from the initial point $x^{(0)} = [6,6]$.
2. **Newton's method formula for multivariable functions:**
$$x^{(k+1)} = x^{(k)} - H_f(x^{(k)})^{-1} \nabla f(x^{(k)})$$
where $\nabla f(x)$ is the gradient vector and $H_f(x)$ is the Hessian matrix of $f$ at $x$.
3. **Calculate the gradient $\nabla f(x)$:**
Let
$$g_1 = x_1^2 + x_2 - 11, \quad g_2 = x_1 + x_2^2 - 7$$
Then
$$f = g_1^2 + g_2^2$$
Gradient components:
$$\frac{\partial f}{\partial x_1} = 2g_1 \cdot 2x_1 + 2g_2 \cdot 1 = 4x_1 g_1 + 2 g_2$$
$$\frac{\partial f}{\partial x_2} = 2g_1 \cdot 1 + 2g_2 \cdot 2x_2 = 2 g_1 + 4 x_2 g_2$$
4. **Calculate the Hessian matrix $H_f(x)$:**
Second derivatives:
$$\frac{\partial^2 f}{\partial x_1^2} = 4 g_1 + 8 x_1^2 + 0 = 8 x_1^2 + 4 g_1$$
$$\frac{\partial^2 f}{\partial x_2^2} = 0 + 4 g_2 + 8 x_2^2 = 4 g_2 + 8 x_2^2$$
$$\frac{\partial^2 f}{\partial x_1 \partial x_2} = 4 x_1 + 4 x_2$$
Symmetric Hessian:
$$H_f = \begin{bmatrix} 8 x_1^2 + 4 g_1 & 4 x_1 + 4 x_2 \\ 4 x_1 + 4 x_2 & 4 g_2 + 8 x_2^2 \end{bmatrix}$$
5. **Evaluate at initial point $x^{(0)} = [6,6]$:**
Calculate $g_1$ and $g_2$:
$$g_1 = 6^2 + 6 - 11 = 36 + 6 - 11 = 31$$
$$g_2 = 6 + 6^2 - 7 = 6 + 36 - 7 = 35$$
Gradient:
$$\nabla f = \begin{bmatrix}4 \cdot 6 \cdot 31 + 2 \cdot 35 \\ 2 \cdot 31 + 4 \cdot 6 \cdot 35 \end{bmatrix} = \begin{bmatrix}744 + 70 \\ 62 + 840 \end{bmatrix} = \begin{bmatrix}814 \\ 902 \end{bmatrix}$$
Hessian:
$$H_f = \begin{bmatrix}8 \cdot 36 + 4 \cdot 31 & 4 \cdot 6 + 4 \cdot 6 \\ 4 \cdot 6 + 4 \cdot 6 & 4 \cdot 35 + 8 \cdot 36 \end{bmatrix} = \begin{bmatrix}288 + 124 & 24 + 24 \\ 24 + 24 & 140 + 288 \end{bmatrix} = \begin{bmatrix}412 & 48 \\ 48 & 428 \end{bmatrix}$$
6. **Compute the inverse of Hessian $H_f^{-1}$:**
Determinant:
$$\det(H_f) = 412 \times 428 - 48 \times 48 = 176336 - 2304 = 174032$$
Inverse:
$$H_f^{-1} = \frac{1}{174032} \begin{bmatrix}428 & -48 \\ -48 & 412 \end{bmatrix}$$
7. **Calculate the Newton step:**
$$\Delta x = H_f^{-1} \nabla f = \frac{1}{174032} \begin{bmatrix}428 & -48 \\ -48 & 412 \end{bmatrix} \begin{bmatrix}814 \\ 902 \end{bmatrix}$$
Multiply:
$$= \frac{1}{174032} \begin{bmatrix}428 \times 814 - 48 \times 902 \\ -48 \times 814 + 412 \times 902 \end{bmatrix} = \frac{1}{174032} \begin{bmatrix}348392 - 43296 \\ -39072 + 371224 \end{bmatrix} = \frac{1}{174032} \begin{bmatrix}305096 \\ 332152 \end{bmatrix}$$
Divide:
$$\Delta x \approx \begin{bmatrix}1.752 \\ 1.909 \end{bmatrix}$$
8. **Update to get $x^{(1)}$:**
$$x^{(1)} = x^{(0)} - \Delta x = \begin{bmatrix}6 \\ 6 \end{bmatrix} - \begin{bmatrix}1.752 \\ 1.909 \end{bmatrix} = \begin{bmatrix}4.248 \\ 4.091 \end{bmatrix}$$
9. **Second iteration: Evaluate gradient and Hessian at $x^{(1)}$:**
Calculate $g_1$ and $g_2$:
$$g_1 = (4.248)^2 + 4.091 - 11 = 18.05 + 4.091 - 11 = 11.141$$
$$g_2 = 4.248 + (4.091)^2 - 7 = 4.248 + 16.73 - 7 = 13.978$$
Gradient:
$$\nabla f = \begin{bmatrix}4 \cdot 4.248 \cdot 11.141 + 2 \cdot 13.978 \\ 2 \cdot 11.141 + 4 \cdot 4.091 \cdot 13.978 \end{bmatrix} = \begin{bmatrix}189.3 + 27.956 \\ 22.282 + 228.7 \end{bmatrix} = \begin{bmatrix}217.256 \\ 250.982 \end{bmatrix}$$
Hessian:
$$H_f = \begin{bmatrix}8 \cdot (4.248)^2 + 4 \cdot 11.141 & 4 \cdot 4.248 + 4 \cdot 4.091 \\ 4 \cdot 4.248 + 4 \cdot 4.091 & 4 \cdot 13.978 + 8 \cdot (4.091)^2 \end{bmatrix}$$
Calculate terms:
$$8 \times 18.05 + 44.564 = 144.4 + 44.564 = 188.964$$
$$4 \times 4.248 + 4 \times 4.091 = 16.992 + 16.364 = 33.356$$
$$4 \times 13.978 + 8 \times 16.73 = 55.912 + 133.84 = 189.752$$
So
$$H_f = \begin{bmatrix}188.964 & 33.356 \\ 33.356 & 189.752 \end{bmatrix}$$
10. **Compute inverse of $H_f$ at $x^{(1)}$:**
Determinant:
$$188.964 \times 189.752 - 33.356^2 = 35868.5 - 1113.9 = 34754.6$$
Inverse:
$$H_f^{-1} = \frac{1}{34754.6} \begin{bmatrix}189.752 & -33.356 \\ -33.356 & 188.964 \end{bmatrix}$$
11. **Calculate Newton step at $x^{(1)}$:**
$$\Delta x = H_f^{-1} \nabla f = \frac{1}{34754.6} \begin{bmatrix}189.752 & -33.356 \\ -33.356 & 188.964 \end{bmatrix} \begin{bmatrix}217.256 \\ 250.982 \end{bmatrix}$$
Multiply:
$$= \frac{1}{34754.6} \begin{bmatrix}189.752 \times 217.256 - 33.356 \times 250.982 \\ -33.356 \times 217.256 + 188.964 \times 250.982 \end{bmatrix}$$
Calculate:
$$= \frac{1}{34754.6} \begin{bmatrix}41222.5 - 8370.3 \\ -7243.3 + 47418.3 \end{bmatrix} = \frac{1}{34754.6} \begin{bmatrix}32852.2 \\ 40175.0 \end{bmatrix}$$
Divide:
$$\Delta x \approx \begin{bmatrix}0.945 \\ 1.156 \end{bmatrix}$$
12. **Update to get $x^{(2)}$:**
$$x^{(2)} = x^{(1)} - \Delta x = \begin{bmatrix}4.248 \\ 4.091 \end{bmatrix} - \begin{bmatrix}0.945 \\ 1.156 \end{bmatrix} = \begin{bmatrix}3.303 \\ 2.935 \end{bmatrix}$$
**Final answer:**
The result of the second iteration using Newton's method is approximately
$$x^{(2)} = [3.303, 2.935]$$