-1
$\begingroup$

All, To deduce the regression coefficient $$c=(A^T A)^{-1} A^T B $$, we assume that the minimum of the square root error $\sqrt{\sum({y_i - y_{hat})}^2}$, reduces to finding the minimum of the argument under the root, ${\sum({y_i - y_{hat})}^2}$, given the monotonicity of the square root (see for example this medium page, equation 8).

Would anybody be able to show that by example? Thanks

$\endgroup$
8
  • 3
    $\begingroup$ What exactly do you mean by "show that by example"? $\endgroup$ Commented Apr 10 at 15:56
  • $\begingroup$ I just mean that I am not interested in having a complicated proof using linear algebra. $\endgroup$ Commented Apr 10 at 16:00
  • 3
    $\begingroup$ Wouldn't "$(3 < 4) \iff (\sqrt{3} < \sqrt{4})$" be a complete Answer to this Question? $\endgroup$ Commented Apr 10 at 16:12
  • 1
    $\begingroup$ If $f(x)$ is any real-valued function (on any domain), and $g(x)$ is any strictly increasing function from $\Bbb R$ to $\Bbb R$, then the minimum of $f(x)$ occurs with the same value of $x$ as the minimum of $g(f(x))$. This is easy to prove from the definition of strictly increasing. $\endgroup$ Commented Apr 10 at 16:19
  • 2
    $\begingroup$ @Kernel This less formal phrasing might help: as long as $f(x)$ has non-negative values (as long as we can take a square root), making $f(x)$ as small as possible is the same thing as making $\sqrt{f(x)}$ as small as possible. So, finding the $x$ that minimizes $f(x)$ is the same as finding the $x$ that minimizes $\sqrt{f(x)}$. $\endgroup$ Commented Apr 10 at 17:29

1 Answer 1

Reset to default
1
$\begingroup$

Here's an analogous example that might help you understand what's going on. Suppose we want to find the point on the line $y = 2 - x$ that is closest to the origin in $\Bbb R^2$. We can parameterize this line as $$ \mathbf x(t) = (2 - t, t). $$ The value that we are trying to minimize is $$ d(t) = \|\mathbf x(t) - 0\| = \sqrt{(2 - t)^2 + t^2}. $$ The $t$ that minimizes this function corresponds to the point on the line that is closest to $(0,0)$. That said, the $t$ that minimizes $d(t)$ will also minimize $[d(t)]^2$. So, we can make the calculus simpler by minimizing the function $$ f(t) = [d(t)]^2 = (2 - t)^2 + t^2 = 2t^2 - 4t + 4. $$ From here, straightforward calculus gives us the answer: $$ f'(t) = 4t - 4 = 0 \implies t = 1 $$ So, the closest point to $(0,0)$ will be $\mathbf x(1,1) = (2 - 1, 1) = (1,1)$.

$\endgroup$
4
  • $\begingroup$ so both $f(t)$ and $d(t)$ get their minimum at $t=1$ $\endgroup$ Commented Apr 11 at 17:05
  • $\begingroup$ I donot think that $(2-t)^{2}+t^{2}$ is a monotonic function. $\endgroup$ Commented Apr 11 at 17:12
  • 1
    $\begingroup$ @Kernel That is correct, neither $f$ nor $d$ are monotonic. But the function $g(t) = \sqrt{t}$ is monotonic. Because of this, the functions $f(x)$ and $g(f(x))$ will have exactly the same minima $\endgroup$ Commented Apr 11 at 18:59
  • 1
    $\begingroup$ @Kernel Or similarly, $h(t) = t^2$ is monotonically increasing over $[0,\infty)$, which means that $d(x)$ and $h(d(t))$ have the same minima $\endgroup$ Commented Apr 11 at 19:45

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.