Source

Adapted from my own handwritten and hand-typed notes from Year 1 to Year 3 of my EEE degree

When there are many data points, or the data contain measurement error, forcing a curve through every point is a poor idea. Least squares instead finds a low degree polynomial that fits the points best overall.

Setup

Given points , look for

that minimises the sum of squared errors

Normal equations

Set for . With the shorthand

this gives linear normal equations:

Note , the number of points. The system has a unique solution and it really is the minimum.

For a straight line ():

Method

Steps:

  1. Choose the degree
  2. Tabulate and and sum them to get and
  3. Write the normal equations and solve for

Worked example

Fit a line to , , , .

Solution:

  • , , , ,
  • and
  • ,
  • best line:

A shift of variables (for example ) can make the sums smaller and easier by hand. For exact fits through a few points, see Lagrange Interpolation.