Transformer
Definition
A neural network design that uses attention to weigh how every part of the input relates to every other part. Most modern language models are built on it.
In plain English
Older language models read text one word at a time, which made long sentences hard. A transformer looks at all the tokens at once and learns which ones matter most to each other.
Because that work can run in parallel, transformers train efficiently on huge datasets, which is a big reason large language models became possible.
Example
In “The trophy didn't fit in the suitcase because it was too big,” attention helps the model link “it” to “trophy” rather than “suitcase.”
Draft, not yet reviewed by a person · Updated Nov 2026