How do large language models (LLMs) work?

Every chatbot answer is built one guess at a time. Train a real tiny language model on a paragraph about monsoons and chai, watch a tokenizer chop Hindi into bytes, click a word to see what it attends to, and catch the model saying something false with total confidence.

Read how it works: How do large language models (LLMs) work? · The history of large language models · More boxes on Glassbox

Chapters in this interactive model

1. Predict the next letter: A language model is a machine for guessing what comes next.
2. Text becomes tokens: Before a model reads a word, the word is chopped into numbered pieces.
3. Words as points in space: Each token becomes a long list of numbers. Similar meanings end up close together.
4. Attention: every word looks at the others: The transformer lets each token pull in meaning from the tokens before it.
5. Scale: more numbers, more text, more power: The same next-token idea, grown a million times bigger, then taught to be helpful.
6. Use them well: fluent is not the same as true: What LLMs do well, where they go wrong, and how to think about them.