Introduction
The quadratic classifier is the Gaussian Bayes classifier in the general case, where each class has its own covariance matrix . Each class then measures distance in its own way, and the boundary between classes is curved.
Main idea
With different $\boldsymbol{\Sigma}_i$, only the constant $\frac{d}{2}\ln 2\pi$ can be dropped. The score is $g_i(\mathbf{x})=\mathbf{x}^T\mathbf{W}_i\mathbf{x}+\mathbf{w}_i^T\mathbf{x}+w_{i0}$, with a quadratic term $\mathbf{W}_i=-\frac{1}{2}\boldsymbol{\Sigma}_i^{-1}$ that differs between classes. So the boundaries $g_i(\mathbf{x})=g_j(\mathbf{x})$ are hyperquadrics: curves in 2D, curved surfaces in higher dimensions.
Overview
The score has the same three parts as in 1D:
Sizes: is , is , and is a number.
Step 1: discriminant function
For the multivariate Gaussian , from Gaussian Discriminant Function:
This is the general multivariate normal case, where the covariance matrices are different for each class. The only term that can be dropped is , and the resulting discriminant functions are quadratic in .
Step 2: expand into three parts
Expand the distance term. Each cross term is a single number, and a number equals its own transpose, so , using that is symmetric:
Multiply by and group the terms by their power of :
The middle term uses , again because is symmetric.

The figure writes the bias as and the transpose as , and these are the same and .
Step 3: why the boundary curves
The decision boundary between class and class is :
When , the matrix is not zero, so the quadratic term does not cancel and the boundary is curved.
The intuition:
- each class measures distance with its own
- a tight class is only the best fit close to its own mean
- a spread-out class is the better fit everywhere else, so the boundary bends around the tight class
- a decision region does not need to be one connected piece (bottom left panel of the first figure below)
Step 4: boundary shapes
The decision surfaces are hyperquadrics. Depending on the means and covariances, they can be:
- hyperplanes or pairs of hyperplanes
- hyperspheres or hyperellipsoids
- hyperparaboloids
- hyperhyperboloids of various types
Examples of the quadratic classifier with 2 dimensions and 2 classes:

Reading the figure:
- each panel shows the two class-conditional PDFs (red and grey) as hills over the plane
- on the floor, the small circles and ellipses are contours of the two classes, and the larger curve is the decision boundary
- top left: the boundary is an open curve between the classes
- top right: the boundary is a closed ellipse around the tight grey class
- bottom left: the boundary has two separate branches, so one class's region is in two pieces
- bottom right: one class sits inside the other, and the boundary is a circle around it
Complex decision boundaries for a 3-dimensional 2-class case, and a 2-dimensional case with many classes:

Reading the figure:
- left: in 3D, the grey and red ellipsoids are contours of the two classes, and the curved saddle-shaped sheet between them is the decision surface
- right: four classes in 2D, shown as hills, with their contours and the regions to drawn on the floor
In the many-class figure, each region is where is the largest, and the borders between neighbouring regions are curves, not straight lines.
Step 5: cost
Each class needs its own mean and its own covariance matrix:
- the mean has numbers
- the covariance has different numbers, since it is symmetric
- so the number to estimate grows like , which needs a very large number of samples per class when is large
With few samples per class, the estimated can be poor, and a simpler classifier may work better.
Why do we need this
It is the full Gaussian Bayes classifier, correct whenever the classes really have different shapes. The linear and minimum distance classifiers are shortcuts that are only correct when the covariances are alike.
Next step
Linear Classifier shows what happens when all classes share one covariance, so the quadratic term cancels.
Visual 1: two thresholds in 1D

A tight class and a wide class with equal priors:
- top: the tight only wins near its own mean, between and , and the wide wins on both sides
- bottom: the scores are a narrow and a wide parabola, because the term differs between the classes
- two different parabolas cross twice, so comes in two pieces, the 1D version of the boundary bending around the tight class
Visual 2: cost of a covariance per class

Numbers to estimate for classes:
- each class needs numbers for its mean and for its own , so the quadratic count grows like
- the linear classifier shares one , and the minimum distance classifier needs only the means
- at that is , and numbers, all to be estimated from the same samples, as in Step 5
Visual 3: few samples bend the boundary

Two classes that truly share one , so the best boundary is the grey dashed line:
- left: with only 6 samples per class, each own comes out with the wrong shape (thin, wrongly tilted), so the quadratic boundary bends and even gives a stray region far from its samples
- right: pooling all 12 offsets into one gives a straight boundary close to the true one
- this draw is the middle one of 300 random draws of 6 samples; in about of them the quadratic boundary makes more errors on new data than the linear one
- this is what Step 5 means by a poor : the extra freedom fits the noise in a few points