Neural networks
A transformer is the neural network design behind modern language models; it uses attention to weigh every part of the input against every other part at once.
Google researchers introduced the transformer in a 2017 paper titled 'Attention Is All You Need.' It replaced older designs that read text one word at a time. A transformer looks at a whole sequence together. Each word can gather information from every other word. That makes it better with context and much faster to train.
The key ingredient is attention. For each word, the model computes how relevant every other word is and blends their information accordingly. Stack many attention layers and the model builds up a rich understanding of who did what to whom, across long passages.
The 'T' in GPT stands for transformer. The same design, with small changes, powers chatbots, translation, code assistants, image generators, and even protein research. Its ability to keep improving as you add data and computing power is why it dominates AI as of 2026.
Translating 'The cat sat on the mat because it was warm,' a transformer lets 'it' look back at both 'mat' and 'cat' to work out which one was warm.