5
$\begingroup$

I stumbled upon a property in solutions of some exercises which stated that if a hessian of a possibly non-convex function f(x) is bounded in spectral norm then its eigenvalues lie in the interval.

$$ ||\nabla^2f(x)||_2 \leq L $$ $$ eigenvalues \in [-L, L]$$

I fail to understand or more I am unable to find where this property comes from, I looked through many materials about spectral norm, spectral radius and I think at this point I am completely confused. I know that spectral norm is the maximal singular value of a matrix. In this case does it mean that hessian is symmetric so eigenvalues == singular values? How do we go further with that to get the interval? I get the upper bound of the interval, it's obvious but why the lower bound. Thank you in advance for pointing me to right sources or directly answering.

$\endgroup$
1
  • $\begingroup$ If $\lambda$ is an eigenvalue of some $A$ then $\|A\| \ge |\lambda|$. $\endgroup$ Commented Jul 18, 2020 at 19:46

2 Answers 2

Reset to default
4
$\begingroup$

If $A v = \lambda v$, then $\|Av\| =|\lambda| \|v\|$ and so if $\|A\|$ is the corresponding induced norm we have $\| A\| \ge |\lambda|$ for any eigenvalue (equivalently, $|\lambda| \in [-\|A\|,\|A\|]$).

Note that the spectral norm is induced by the Euclidean norm.

Since the Hessian is real symmetric, the eigenvalues are real and so for any eigenvalue $\lambda$ we have from above that $\lambda \in [-\|A\|,\|A\|]$.

$\endgroup$
5
  • $\begingroup$ I am unable to understand the implication ||$A$||$\ge$ |$\lambda$| from ||$Av$||$=\vert \lambda \vert$||$v$||. $\endgroup$ Commented Jul 19, 2020 at 2:26
  • $\begingroup$ @NitinUniyal: An induced norm is defined by $\|A\| = \sup_{\|x\| \le 1} \|Ax\|$. In particular, if $v$ is a unit vector, $\|A\| \ge |\lambda|$. $\endgroup$ Commented Jul 19, 2020 at 3:15
  • $\begingroup$ Thanks copper.hat...but in that case the implication seems to be valid if the eigenvector belongs to the unit ball. $\endgroup$ Commented Jul 19, 2020 at 3:26
  • $\begingroup$ @NitinUniyal: You can scale the eigenvector to be any length you want. $\endgroup$ Commented Jul 19, 2020 at 3:27
  • 1
    $\begingroup$ oh right! Thanks I understood now. +1 $\endgroup$ Commented Jul 19, 2020 at 3:28
1
$\begingroup$

The spectral norm of a matrix is, by definition, the largest absolute value of its eigenvalues. If the largest absolute value of the eigenvalues is less than or equal to $L$, then all the eigenvalues have absolute value less than or equal to $L$, so if they are real, they all lie in the interval $[-L,L]$.

$\endgroup$

You must log in to answer this question.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.