0
$\begingroup$

The derivative of distance function with respect to time give velocity function in single variable calculus.

But how does gradient of a multivariable function point steepest ascent?

I have been reading about the concept and trying to make an analogy with single variable calculus examples to understand the logic behind that but I still don't understand.

The derivative of distance function with respect to time give velocity but the derivative of a scalar field with respect to position(x,y,z,) gives steepest ascent but how?

$\endgroup$
3
  • 1
    $\begingroup$ The best way to think about the derivative is:$ f (x + \Delta x) \approx f (x) + f'(x) \Delta x $. When $ f:\mathbb R^n \to \mathbb R $, $ f'(x) $ is the gradient and $ f'(x) \Delta x$ is an inner product. What should be the direction of $\Delta x $ to make that inner product as large as possible (to increase $ f $ the most)? $\endgroup$ Commented Jul 29, 2015 at 8:52
  • $\begingroup$ Does this answer your question? Is Gradient really the direction of steepest ascent? $\endgroup$ Commented Apr 27, 2022 at 5:30
  • $\begingroup$ math.stackexchange.com/questions/223252/… $\endgroup$ Commented Apr 27, 2022 at 5:31

1 Answer 1

Reset to default
2
$\begingroup$

The rate of increase of the function $f(\vec{x})$ in the direction of $\vec{u}$, a unit vector, is $$D :=\lim_{t \to 0} \frac{f(\vec{x} + t\vec{u}) - f(\vec{x})}{t}$$ Now let $F(t) = f(\vec{a} + t\vec{u})$, then $$D = \frac{dF}{dt}(0) = \frac{\partial f}{\partial x_1}(\vec{a}) \, \frac{dx_1}{dt} + ... + \frac{\partial f}{\partial x_n}(\vec{a}) \, \frac{dx_n}{dt} = \frac{\partial f}{\partial x_1}(\vec{a}) \, u_1 + ... + \frac{\partial f}{\partial x_n}(\vec{a}) \, u_n$$ by the chain rule for partial derivatives. But this is just $$D = \nabla{f}(\vec{a}) \cdot \vec{u}$$ which is clearly maximised by $\vec{u} = \nabla{f}(\vec{a})$ (or in the direction of $\nabla f(\vec{a})$ as it is a unit vector) so the greatest increase is in the direction of $\nabla{f}$.

$\endgroup$
5
  • $\begingroup$ So $\nabla{f}$ does already point the greatest increase. right? So I am trying to understand the reason behind that $\endgroup$ Commented Jul 30, 2015 at 17:31
  • $\begingroup$ @user50322 I'm not sure what you mean by 'does already point' or 'the reason behind it' exactly? The above doesn't define $\nabla f$ as pointing in that direction, we've just extended the traditional derivative to the directional derivative for multi-variable functions to find rate of increase in any given direction and then attempted to maximise that by choosing the right direction $\vec{u}$, using standard properties of the dot product. This shows that this is the direction of greatest increase. $\endgroup$ Commented Jul 30, 2015 at 21:01
  • 1
    $\begingroup$ @user50322 If you're looking for a more intuition based understanding then note $\nabla f = (\frac{\partial f}{\partial x}, \frac{\partial f}{\partial y}, \frac{\partial f}{\partial z})$ and these components are the rate of increase along the x,y and z directions respectively. In general, if we want to maximise increase we should move in ratio to the rate of increase along the axes. $\endgroup$ Commented Jul 30, 2015 at 21:04
  • $\begingroup$ Why does it have to be along the axes? Why does maximum increase have to be along the x,y and z direction? I am confused here. Thanks for help $\endgroup$ Commented Sep 3, 2015 at 8:14
  • $\begingroup$ @user50322 Maximum increase doesn't have to be along the axes, but it can always be decomposed in to components along the axes. (like any vector in our space). Then you're looking for a unit vector $\vec{u}$ that when decomposed in to components "combines" the increase along the axes in the optimal way. I.e. you want $u_1\frac{\partial f}{\partial x} + u_2 \frac{\partial f}{\partial y} + u_3 \frac{\partial f}{\partial z}$ to be maximised. But this is just $\vec{u} \cdot \nabla f$ and then the result follows from properties of dot products as $\vec{u}$ is a unit vector. $\endgroup$ Commented Sep 7, 2015 at 21:53

You must log in to answer this question.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.