How LLMs Actually Work
Predicting the next token — that's the whole mechanism. Every strength and every flaw grows out of it.
Every time it generates, this is all it does: guess the next token. All the magic and all the flaws grow out of that.
It isn't thinking about your question; it's computing what the next token most likely is. That sounds like a put-down, but it's actually the only doorway into understanding it.
Signs you'll recognize:
- Ask the same question twice and get different answers — both stated with total confidence.
- Ask it to think before answering, and the results genuinely improve.
- It notices halfway through that an earlier part is wrong, but doesn't fix it — it pushes on and smooths over.
The loop
Feed it a block of text and all it does is repeat four steps: read the whole existing sequence, compute the probability distribution of the next token, draw one sample, append it to the end. Then start over. This step-by-step autoregressive loop is the core mechanism of the Transformer architecture, and today's mainstream models are its direct descendants. transformer
Watch step three. It doesn't pick the highest-probability token — it samples. That difference is the source of all randomness. The 62% token usually wins, but the 18% token still gets its chance, and once chosen, the whole sentence veers elsewhere.
Watch step one too. Every new token requires re-reading the entire sequence. There's no pre-formed answer stored anywhere, no draft. The streaming output you watch is the actual reasoning process, not a reading-out of a finished thought.
People overestimate it, so here's "what you think it does" versus "what it actually does":
| You think it does | It actually does |
|---|---|
| Understand your question | Cut your words into tokens and compute numbers |
| Look things up | Nothing to look up — only statistical patterns in the weights |
| Think of an answer, then reply | No such phase; it decides as it writes |
| Check the answer before sending | No such phase, unless you make it run another round |
The middle two rows cause the most misunderstanding. Every fact it gives you is "computed to look real," not "checked to be real."
Where the ability comes from
Why can something that merely continues text write code, revise copy, and read between the lines of half a sentence? Because "guess the next word" accurately enough requires learning a lot of other things along the way.
- To guess the next line of code, it must learn syntax and common structures. Parentheses must pair, variables must be declared — these patterns appear hundreds of millions of times in the corpus, and wrong guesses get penalized.
- To guess the end of a piece of reasoning, it must learn the shape of reasoning. What follows "because...therefore..." is tightly constrained. It learns the shape, not the logic — but when the shape is right, the conclusion often is too.
- To guess the next line of a conversation, it must learn tone and intent. Customer-service register isn't followed by insults; that's a statistical regularity.
At sufficient scale and corpus size, capability overshoots "continuation" — that's what the famous "few-shot learners" paper set out to show: as parameters and data grow, task abilities don't tick up gradually; they emerge in chunks. gpt3-paper
That's also why "think step by step" prompts work: you force it to generate intermediate steps, those steps enter the sequence and become inputs to every later step. It hasn't gotten smarter; it has laid better continuation material for itself. Making it talk more is making it compute more rounds.
The flaws come from the same place
Flip the same mechanism over and you have its entire list of weaknesses. These aren't bugs; they're direct consequences of the design. A different model can ease them, never remove them.
| Symptom | Mechanism |
|---|---|
| Different answer every time | Step three samples; it doesn't take the max |
| Confident fabrication | It wants "most likely next line," not "true next line" |
| No going back after a mistake | Generated tokens are already in the sequence; it can only round it off |
| Bad at counting and arithmetic | Digits become tokens; place value isn't a strong statistical pattern |
| Drifts the longer it writes | Each step's error feeds into the next step's input |
The third row matters most for product design: once the opening goes off course, it only rounds further away. Instead of hoping it self-corrects mid-write, cut tasks into small pieces and verify each one separately.
Three actions land on your desk. One, split long tasks into shorter segments; shorter segments accumulate less error. Two, pin down the opening — the first sentence sets a tone that's hard to pull back, so put format, style, and constraints before it starts speaking. Three, verify everything that must be right: numbers, API names, citations. Don't trust it just because it sounds certain.