Verifying Answers

Check the source, look for a counterexample, recompute the key numbers. How hard to check follows the consequence, not the confidence.

Checking every sentence for truth is not realistic, and you do not need to. Three moves — check the source, look for a counterexample, recompute the key numbers — each aim at one of the main shapes fabrication takes, and together they filter out most of the errors that would hurt you. What remains gets weighed by consequence, not by how confident it sounds.

After reading, you should be able to answer:

• How far should you check a given answer, and what decides that?
• Why is checking the source the highest-value of the three moves?
• When it says "always" or "never", which move is the one to make?
• How do you make checking work without relying on anyone's vigilance?

How hard you check follows the consequence, not the tone

The easiest thing to get wrong is using its tone as your evidence: the more certain it sounds, the more likely you are to skip the check. But tone carries no information about accuracy — it fabricates in the same voice it uses to tell the truth. So the only sound basis is consequence, read off what the output will become:

• Output that only shapes your next move. A search entry point, a sparring partner for your own reasoning, a narrator for code you have never read. Being wrong costs a redo, so do none of the moves and keep going.
• Output that reaches someone else's judgment. Written into a spec, pasted into meeting notes, posted as a conclusion for the team to act on. At this level the sources have to be checked first, because you are answerable for the content.
• Output that becomes a commitment or enters a system. A quote, a public announcement, production parameters, a field written into a database, a clause about to be signed. This level gets all three moves, plus a person signing off at the final gate.

The dividing line is the middle one: once it moves from "I will use this" to "someone else will act on this," checking stops being optional. The last level carries a second implication — that class of output should not run to completion on its own at all, which is the line drawn in The Human Boundary.

Three places that fail most, and one move for each

The three moves were not chosen at random. Each one covers a spot where failure is dense.

1. Check the source. Wherever a citation, figure, name or paper shows up in the answer, open the original. If you find it, cross-check the numbers it wrote against the numbers in the original; if you cannot find it, the claim was never made. This runs in two steps, because for a quotation "the link opens" and "the page really says that" are different things, and the second is where it paraphrased — It Is Not a Search Engine breaks that step out. This move filters out the most, because inventing a plausible-looking source costs the model nothing — report titles, author names, years, and link formats are all strongly patterned, and generating one is easier than telling the truth. hallucination-survey There is a trap in the other direction too: any source it volunteers that falls outside the material you supplied should be treated as false until proven otherwise.

2. Look for a counterexample. Whenever you read a claim built on "always", "definitely" or "never", construct one scenario in which it does not hold. If it can name the exceptions and mark its own boundaries, it has genuinely considered the conditions. If it insists no exception exists, it probably did not think it through — or it is simply agreeing with you. This move targets its sycophancy: the pull toward producing the conclusion you expect, because what it optimises is looking right, not being right.

3. Recompute the key numbers. Percentages, amounts, dates, conversions — do the arithmetic yourself. This is where fabrication appears most densely and also where it is easiest to catch: the moment a number fails to check out, that sentence is void, however smoothly the rest of it reads. A machine that writes one word at a time by probability is inherently bad at exact arithmetic, and counting and exact sums are the same story.

Together the three close off invented sources, agreeable logic, and wrong arithmetic — the main shapes fabrication takes. Vendor guidance on reducing hallucination lands on the same line: identify the high-risk cases first, then check and constrain them deliberately, with the honest caveat that such measures reduce it substantially and do not remove it. claude-hall

Build the check into the product, not into people's vigilance

As a personal habit, checking is the first thing dropped in a busy week. The steadier version builds it into the product:

• Answers arrive with sources. A source you can open and locate in the original gives checking somewhere to land. Retrieve, Then Answer covers how that layer is built.
• Let code decide whatever code can decide. Anything code can judge — formats, interface names, figures — has no business on a manual checklist; what each layer costs is covered in Three Lines of Defence.
• Do not push the job onto users. A line like "AI may be wrong, please verify" takes the least effort to write and achieves the least: what the product owes becomes a burden the user carries instead.
• Keep "I do not know" a legal answer. Without that exit, when the material runs short it can only invent one for you — where that exit belongs.

This chapter closes the column — where to go next

That completes the learn column: what kind of machine it is, where it stops, how to ask it clearly, where knowledge comes from, what it costs, and this chapter — which parts can never be handed over.

Two directions follow. One is choosing the machinery. What the model can do is settled; the open question is how much scaffolding the job deserves — a single call, retrieval, a loop, several roles. The answer is not "as much as possible" but a walk up from the lowest rung, asking "do I actually need this?" — Does This Even Need an Agent? is that ladder. The other is feeding it context. On any given task, the biggest variable is rarely the prompt; it is how much it knows about your project: structure, naming conventions, domain rules, reference examples. Context Engineering: Good Context Beats a Good Prompt is how those pillars get built.

Neither road is short, but the order is clear: keep the judgment with yourself first, and only then decide how much of the work goes over.