Source
Adapted from my own handwritten and hand-typed notes from Year 1 to Year 3 of my EEE degree
When there are many data points, or the data contain measurement error, forcing a curve through every point is a poor idea. Least squares instead finds a low degree polynomial that fits the points best overall.
Setup
Given points , look for
that minimises the sum of squared errors
Normal equations
Set for . With the shorthand
this gives linear normal equations:
Note , the number of points. The system has a unique solution and it really is the minimum.
For a straight line ():
Method
Steps:
- Choose the degree
- Tabulate and and sum them to get and
- Write the normal equations and solve for
Worked example
Fit a line to , , , .
Solution:
- , , , ,
- and
- ,
- best line:
A shift of variables (for example ) can make the sums smaller and easier by hand. For exact fits through a few points, see Lagrange Interpolation.