FOUNDATIONS
What Is a Neural Network?
A neural network is a function made of layers of simple units (neurons), each one a weighted sum of its inputs passed through a small nonlinearity. Nothing about what the network should do is programmed in; every weight is learned by adjusting it against training data until the network's output gets closer to the target.
One neuron, then a lot of them
A single artificial neuron takes several numbers in, multiplies each by a weight, adds them up with a bias term, and passes that sum through a nonlinear function (an activation function, commonly ReLU today: output the value if positive, otherwise zero). On its own that neuron can only draw a straight-line decision boundary between two kinds of input.
A neural network stacks many neurons into layers, and stacks layers on top of each other. The output of one layer becomes the input to the next. Because each layer's activation function is nonlinear, stacking layers lets the network approximate far more complex functions than any single neuron could, curved decision boundaries, image patterns, sequences of language, whatever the training data represents.
Where the weights come from: training, not code
At the start, every weight in the network is a small random number, so its output is close to noise. Training shows the network an input paired with the correct output, measures how wrong its actual output was with a loss function, and then adjusts every weight slightly in the direction that would have reduced that error.
That adjustment step is backpropagation: the error at the output is worked backward through the network, layer by layer, to work out exactly how much each individual weight contributed to the mistake, then each weight moves a small step (set by the learning rate) opposite to its share of the blame. Repeated over millions or billions of examples, this is the entire mechanism, there is no separate step where anyone writes rules for what the network should do.
The same core idea, different shapes on top
Every model on this site, language, image, audio, video, is this same weighted-layers-plus-backpropagation idea underneath. What differs is the architecture built on top of it and what data it trains on: a transformer arranges its layers around attention (see what attention is) and trains on token sequences; a diffusion model arranges its layers to predict and remove noise from an image; an audio model does the same over sound. The training loop, guess, measure the error, adjust the weights, is the constant across all of them.
FAQ
- Is a neural network modeled on the human brain?
- Loosely, and mostly at the naming level. The idea of a neuron summing weighted inputs was inspired by a simplified 1940s model of a biological neuron. Modern networks are trained by backpropagation, a mechanism with no known biological equivalent, and their scale and structure are driven by what works in practice, not by anatomical accuracy.
- What makes a network 'deep'?
- Depth just means more than one hidden layer stacked between the input and the output. "Deep learning" is the name for training networks with many such layers; there's no fixed layer count where a network stops being shallow and becomes deep.
Sources
Related
Last updated 2026-09-20