Artificial intelligence can now write essays, diagnose diseases, fold proteins and beat grandmasters, and underneath it all lies an idea from the 1940s refined by mathematics anyone with calculus can grasp. A neural network is a vast assembly of simple units, loosely inspired by brain neurons, connected by adjustable numerical weights. Learning means tuning those weights so the network’s outputs match desired answers, using an algorithm called backpropagation that is essentially the chain rule of calculus applied at staggering scale. There is no magic and no real understanding, just optimisation: minimising error across billions of examples until the network’s behaviour becomes astonishingly capable. Understanding this mathematics demystifies AI, revealing both why it works so well and why it fails in such alien ways.

The neuron, simplified to arithmetic

An artificial neuron is almost embarrassingly simple: it takes a set of numbers as inputs, multiplies each by a weight, adds them up, adds a bias, and passes the result through a nonlinear activation function. Stack thousands of these into layers, connect every neuron to every neuron in the next layer, and you have a network that computes a fantastically flexible mathematical function with millions or billions of adjustable parameters. The nonlinearity is crucial: without it, the whole stack would collapse into a single linear transformation, incapable of learning complex patterns. Training presents the network with examples, compares its output to the correct answer using a loss function, and then adjusts every weight slightly to reduce the error. A network that begins producing gibberish gradually, over millions of such adjustments, learns to recognise faces, translate languages or predict protein shapes. The architecture is dumb; the scale and the data make it smart.

Backpropagation: learning by blame assignment

The breakthrough that made deep learning possible was backpropagation, popularised in the 1980s by Rumelhart, Hinton and Williams. The problem it solves is credit assignment: when the network errs, which of its billions of weights deserve the blame? Backpropagation answers with calculus: it computes the gradient of the loss with respect to every weight, the direction in which each weight should change to reduce error, using the chain rule to propagate blame backward from output to input, layer by layer. Then gradient descent takes a small step in that direction, and the process repeats over millions of examples. The mathematics is elegant, but the scale is brutal: training a frontier model like GPT-4 class systems involves thousands of specialised chips running for months, consuming megawatt-hours of electricity and costing tens of millions of dollars. The result is a set of weights, a hundred gigabytes of numbers, that somehow encodes grammar, facts, reasoning patterns and much more, though nobody fully understands how the knowledge is organised inside.

Why deep learning works, and where it breaks

Theory still lags practice: mathematicians cannot fully explain why networks with billions of parameters, capable of memorising their training data outright, instead learn generalisable patterns. Concepts like the lottery ticket hypothesis, flat minima and double descent offer partial explanations, but deep learning remains partly alchemy. Its failures are revealing: networks confidently hallucinate false facts because they optimise for plausible text, not truth; they are fooled by adversarial examples, images altered imperceptibly to humans but misclassified with confidence, because their features differ from ours; and they inherit biases from training data, reproducing societal prejudices at scale. India’s AI researchers grapple with an additional gap: frontier models trained overwhelmingly on English and Western data perform worse on Indian languages and contexts, motivating efforts like Bhashini and indigenous models. The mathematics is universal, but the data is not, and intelligence built from biased data inherits the bias. Understanding the maths is thus also understanding the limits: these are pattern machines of unprecedented power, not minds.

• A modern frontier AI model contains hundreds of billions of adjustable weights.
• Backpropagation, the core learning algorithm, was popularised in the 1980s.
• Training a frontier model can cost tens of millions of dollars in computing.
• Neural networks optimise for plausible outputs, which is why they hallucinate confidently.
• India’s Bhashini mission aims to build AI for the country’s many languages.

FAQs

Do neural networks work like the human brain? Only loosely. The inspiration is real, but artificial neurons are mathematical simplifications, and backpropagation has no known counterpart in biological brains.

What is the difference between AI, machine learning and deep learning? AI is the broad goal; machine learning is AI that learns from data; deep learning is machine learning with multi-layered neural networks, the approach behind recent breakthroughs.

Will AI keep improving with scale? So far, larger models trained on more data perform better, a pattern called scaling laws, but data, energy and chip constraints may eventually bite.

Calculus, applied billions of times over, has produced machines that write, see and reason. The mathematics is not mysterious; what remains mysterious is how much can emerge from so simple a rule.

Compiled by the Khabar 24h Editorial Desk from publicly available sources.