Source
Adapted from my own handwritten and hand-typed notes from Year 1 to Year 3 of my EEE degree
The gradient puts all first partial derivatives into one vector:
For variables, .
Key properties
Two facts make the gradient useful:
- points in the direction where increases fastest, and is that fastest rate (see directional derivative)
- is perpendicular to the level curve through , and is perpendicular to the level surface through
This second fact is why is used as the normal vector of a tangent plane. The direction (fastest decrease) is the idea behind the gradient descent algorithm.
Why it is perpendicular to a level curve
Differentiate implicitly with respect to :
The vector points along the tangent of the level curve, so is perpendicular to it. For a surface, take any curve on the surface: gives by the chain rule.
Worked example (two variables)
Let and .
Since , the level curve through is , a parabola. The vector is perpendicular to its tangent at (the tangent slope there is , direction , and ).
Worked example (three variables)
: