The forward pass
From one neuron to a network
Everything so far: inputs, weights, weighted sum, activation, has described exactly one neuron. A neural network is just many of those, arranged in layers, each layer feeding the next:
- Input layer: not really neurons at all, just the raw numbers you feed in (pixel values, measurements, whatever your features are). No computation happens here.
- Hidden layer(s): the layers in between. Each neuron here takes every output from the previous layer, runs its own weighted sum + activation, and passes the result forward. "Hidden" just means the outside world never looks at these values directly.
- Output layer: the last layer. Its result is whatever the network actually reports: a class, a probability, a number.
This is exactly what fixes the XOR wall from the perceptron lesson. A single perceptron draws one line. But give a hidden layer two neurons, and now you have two lines, one from each neuron. The output neuron then combines those two lines, which is enough to carve out the bent, non-linear boundary that XOR needs. Stack more hidden layers, with more neurons each, and the network can bend its decision boundary into almost anything.
Which layer is this?
For each description, pick which part of the network it's describing.
Receives the raw data: pixel values, measurements, whatever your features are, with no computation of its own.
Sits between input and output. Its neurons aren't directly seen by the outside world, they compute intermediate representations that only the rest of the network uses.
The very last layer, whose result is what the network actually reports as its prediction.
Adding one of these, with a couple of neurons of its own, is exactly what lets a network solve XOR, something a single perceptron couldn't do.
Has exactly as many units as you have input features, no more, no less, since nothing here is actually computing anything.
For a yes/no classifier, this usually has a single neuron with a sigmoid activation.
A network can stack several of these one after another, this is literally what makes a network "deep."
The forward pass is one big chain of matrix multiplications
Put the layers back into math and the whole network collapses into a short, repeatable formula. For any layer: output = activation(W · x + b), take the previous layer's output x, multiply it by that layer's weight matrix W (every neuron's weights, all at once), add the vector b, then apply the activation function to every result.
A forward pass is just that formula, repeated once per layer, left to right:
- hidden = activation(W₁ · input + b₁)
- output = activation(W₂ · hidden + b₂)
- ...repeated again for however many hidden layers the network actually has
That's the entire trick from the very first lesson of this module, paying off: "we bent the biology to fit the hardware" because a forward pass, no matter how many layers or how many millions of neurons, is nothing but a chain of matrix multiplications. That's precisely the one operation GPUs are optimized to do at enormous scale. Running this chain once, to turn an input into a prediction, is what people mean when they say a model performs "inference."
One thing a forward pass does not do: learn. It only computes an answer using whatever weights the network currently has. Figuring out how to improve those weights is a completely separate step, , which is exactly where this module goes next.
Check your understanding
What is a "forward pass"?
What to remember
- ✓A network is layers of neurons: an input layer (raw data), one or more hidden layers (internal computation), and an output layer (the final prediction).
- ✓Hidden layers are what fix the XOR wall: multiple neurons each learn their own line, and combining those lines lets the network bend its decision boundary into shapes a single perceptron never could.
- ✓For each layer, output = activation(W · x + b), a weight matrix multiply, plus bias, plus activation.
- ✓A forward pass is that formula repeated once per layer, left to right, until you reach the output, and it's exactly the chain of matrix multiplications GPUs are built to run fast. This single run is called "inference."
- ✓A forward pass only produces a prediction, it never learns. Improving the weights is a separate step, covered next: backpropagation.