0
$\begingroup$

I am a bit confused about the concept of convexity analysis when doing model fitting. Say I have developed some model of two parameters $f(x;\theta_1,\theta_2)$, that I will plan to fit to some data I will measure. I plan to do least squares regression to fit the model to the data.

The loss function will be $$L(\theta_1,\theta_2) = \frac{1}{N}\sum_i(f(x_i;\theta_1,\theta_2) - y_i)^2 $$ where the $y_i$ are the measured values. The convexity of this loss function tells us how easy the fitting will be.

Can I say anything about the convexity of the loss function, without knowing what the data is? Intuitively, a 'simple' model should make it more likely for the problem to be convex, and a 'complicated' model less likely, but I can't calculate the convexity (either symbolically or numerically) without data.

Improve this question
$\endgroup$
2
  • $\begingroup$ Have you looked at the Hessian matrix of $L$? If this is positive semidefinite, then $L$ is convex. It will probably depend much more on how well-behaved your model $f$ is. $\endgroup$ Commented May 7 at 8:07
  • 1
    $\begingroup$ It depends on $f.$ In general, there's little that can be said for the simple reason that even when $f$ is twice differentiable in the $\theta_i,$ for any given $n$ your question amounts to solving a complicated set of inequalities determined by the Hessian of $f.$ $\endgroup$ Commented May 7 at 13:59

0

Reset to default

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.