Consider the matrix $\mathbf{B}\in\mathbb{R}^{2\times 2}$ and vector $\vec{b}\in\mathbb{R}^2$ given by $$ \mathbf{B} = \left(\begin{array}{c c}8 & \alpha \\ \alpha & 2\end{array}\right),\quad\vec{b} = \left(\begin{array}{c}6 \\ \beta \end{array}\right), $$ and hence define $$ f(x) = x^\intercal \mathbf{B} x + b\cdot x,\quad x\in\mathbb{R}^2, $$ I'm trying to evaluate the Hessian of this matrix without explicitly calculating the second order partial derivatives of $f$. To this end, we note that the Jacobian matrix of $f$, $f'$, is given by $$ f'(x) = 2x^\intercal \mathbf{B} + \vec{b}^\intercal,\quad x\in\mathbb{R}^2. $$ I was simply wondering whether or not, from here, to determine the Hessian matrix of $f$, $f''$, we can simply differentiate symbolically again to obtain $$ f''(x) = 2\mathbf{B},\quad x\in\mathbb{R}^2. $$ I've double-checked by explicitly calculating the second-order partial derivatives of $f$, and it seems the Hessian of $f$ is indeed $2\mathbf{B}$, although I was also wondering whether or not the above approach is valid in general? Thanks in advance.
$\begingroup$ $\endgroup$
2 - $\begingroup$ As noted in the answer by @Golden_Ratio, this is correct since $ \mathbf B $ is symmetric. Conversely, if $ \mathbf B $ were not symmetric, then even $ f ' ( x ) $ would have to be $ x ^ \intercal \mathbf B + x ^ \intercal \mathbf B ^ \intercal + \vec b ^ \intercal $. This may make it more obvious why the second derivative is $ \mathbf B + \mathbf B ^ \intercal $ in general. $\endgroup$Toby Bartels– Toby Bartels2023-04-04 16:26:53 +00:00Commented Apr 4, 2023 at 16:26
- 1$\begingroup$ Please note that it is far more common to write the first derivative as a vector, i.e., $f'(x) = 2B x + b.$ That is why below people write $\partial^2 f/ \partial x \partial x^\intercal := \partial/ \partial x (\partial f / \partial x^\intercal)$ and $\partial f / \partial x^\intercal := (\partial f / \partial x)^\intercal.$ (Here I've used $:=$ to mean "is defined as.") $\endgroup$William M.– William M.2023-04-04 16:40:32 +00:00Commented Apr 4, 2023 at 16:40
Add a comment |
1 Answer
$\begingroup$ $\endgroup$
In general, for constant matrix $B$, you have
$${\partial^2 \over \partial x\partial x'}x'Bx =B+B'.$$
Of course, if $B$ is symmetric as in your example, then this simplifies to $2B$.