Introduction
An artificial neuron is the basic unit of every neural network. It adds up its inputs with a weight on each, compares the sum with a threshold, and passes the result through an activation function. This note writes the neuron in two equivalent ways: with a separate threshold, and with the threshold turned into one more weight.
Main idea
$y_k=f(u_k-\theta_k)$ with $u_k=\sum_{j=1}^{p}w_{kj}x_j$. Treat the threshold as one more weight, $w_{k0}=-\theta_k$ on a constant input $x_0=1$, and the neuron becomes $y_k=f\left(\sum_{j=0}^{p}w_{kj}x_j\right)=f(\mathbf{w}_k^T\mathbf{x})$: a weighted sum, then an activation.
Overview
One neuron is two operations in a row:
A whole network is many of these, with the outputs of one layer used as the inputs of the next.
Step 1: network and neuron

Reading the figure:
- left: a network of artificial neurons, called an artificial neural network (ANN) or just a neural network (NN), where each circle is a neuron and each line carries one weight ( in the first layer, in the second), and the output neurons use , which may differ from the hidden activation
- right: one neuron opened up, with inputs, synaptic weights, a summing junction, a threshold and an activation function
Step 2: neuron model
In mathematical terms, neuron with inputs is:
The parts:
- are the input signals
- are the synaptic weights of neuron , one per input, which say how much each input counts
- is the linear combiner output, the weighted sum, computed at the summing junction
- is the threshold: is positive only when the weighted sum is above it
- is the activation function, and is the neuron's output
- here numbers the inputs, and from Multilayer Perceptron Structure on, numbers the inputs and the hidden neurons
Step 3: threshold as a weight
The threshold is an external parameter that also has to be learned, just like a weight. So treat it as one. Add a constant input and give it a weight:
Then the sum starts from :
Here is only a name for the weighted sum, which Backpropagation calls . It is not the weight used in later notes.
Why this is the same neuron: the extra term is , so
Some diagrams draw the same thing as a fixed input with the weight , which gives the same product .
In vector form, put the constant input first:
What this buys:
- the neuron is now just "weighted sum, then activation", with no separate threshold
- every later formula can be written as , with the threshold hidden inside
- the weight is often called the bias, and it still counts as one trainable parameter of the neuron
Step 4: one neuron is a linear classifier
With a step activation (output 1 when , else 0, one of the choices in Activation Functions), the neuron answers one question: which side of the boundary does fall on? That boundary is a straight line in 2-D, a plane in 3-D and a hyperplane in general, so one neuron is a linear classifier, the same kind of boundary as the Gaussian linear classifier.
Why do we need this
Every network in this topic is made of these units. Seeing the neuron as "weighted sum, then activation" makes the larger networks easy to write down, and the bias trick keeps all the formulas in the simple form .
Next step
Activation Functions looks at the choices for .
Visual 1: threshold shifts, weights turn

How to read it:
- the boundary of neuron is where , a straight line when there are two inputs
- left: with fixed, changing moves the line parallel to itself, and points to the side where
- right: with fixed, turning turns the line, which stays at distance from the origin
- so the weights set the direction of the boundary and the threshold sets where it sits, and both must be learned (Step 3)
Visual 2: threshold as a weight on a constant input

Step 3 drawn for a neuron with one input:
- left: with and the neuron switches on at
- right: adding puts every input on the grey line of a 2-D space, with weights and , and since runs across and runs up, the arrow goes 2 right and 3 down
- the boundary is now a line through the origin, at a right angle to , and it cuts the grey line at the same
- every input gets the same output as before, so the threshold is now an ordinary weight and