LLMs and Search

Search hands back a place you can open; a model hands back a generated conclusion. The difference is not accuracy — it is that a conclusion has no source.

It is not a search engine, but almost everyone uses it like one at first. The difference is not which is more accurate — it is that the two hand back completely different things. Search gives you a place you can open. The model gives you a conclusion generated out of statistical patterns, and a conclusion has no source.

After reading, you should be able to answer:

• Why does a fact it states so confidently turn out to have no source anywhere?
• Why does it mix up two similar things?
• Once you connect retrieval, which problems are solved and which are still there?

A location and a piece of content are two different deliverables

While a search engine works, the original pages stay where they are. What it does is index, match, and rank, and what it hands you is the position of a passage: which document this text came from, and where inside it. That makes checking cheap — open the link and the sentence, the author, and the publication date are all in front of you. You do not have to trust any judgment made behind the search box.

The model's route is nothing like that. Once training ends the corpus is let go, leaving only a pile of weights. What the weights preserve is not "this sentence came from that paper" but "content of this kind is normally followed by content of that kind." The statistical regularities were pressed into the parameters; the corpus itself was never put on disk. So "it remembers that paper" is imprecise — what it holds are the phrasings, structures, and terminology common to papers of that sort, reassembled at answering time. The result is grammatically clean and reads like prose, yet it corresponds to no particular document. Search fails too, of course — nothing found, or the right page ranked below other pages — but its failures are visible; the model's failures are silent. The dividing line here is whether the answer can be checked, not which tool scores higher.

Having no source brings three things with it

The first is that content bleeds between similar things. Nothing in the weights marks where a given piece of information came from, so material that looks alike contaminates material that looks alike.

Two major versions of the same library where a same-named function takes its arguments in a different order; two vendors whose names differ by a letter or two, each publishing a different battery capacity; two adjacent chapters of one document with different effective dates. Pairs that are "alike but not the same" fail most often.

What makes it worse is how it looks afterwards: nothing in the tone changes, and the answer still comes out calm and even. The hallucination literature maps this failure mode thoroughly. hallucination-survey

The second is that it only knows the world up to the end of training. The weights froze the moment training finished, so versions released, rules amended, or rates adjusted afterwards are simply unknown to it. It does not fall silent when asked, though: it keeps extrapolating along the pre-cutoff pattern, and the wrong answer it produces still reads plausibly. Which day it actually stops at, and where that hurts most, is Where Its World Stops.

The third is the worst of the three: it has no source, and it can make one up. Document numbers, volume and page numbers, clause numbers, publication dates — the regularities in these formats are very strong, and forging them costs it nothing. It may even list several at once, neatly formatted, every one of them convincing. Treat any citation it volunteers as unverified until you check it.

Of the three, a newer model eases the first two. It does not ease the third: inventing a source and stating a conclusion are the same action to it.

What retrieval adds is a source, not an end to generating

Many products search first, then answer: a few real passages go into this request's input and the model writes from them. That is retrieval-augmented generation — the setup the original RAG paper named. rag-paper

What you bought is a source. What you did not buy is an end to generating. Whenever the retrieved passages are incomplete, it still fills the gaps from memory. When the passages conflict with each other, it picks whichever version reads better and smooths it over. So an answer carrying citations is not an answer you can cite safely. Two checks, in this order:

1. Open the link and confirm it exists and points at the source it claims.
2. Find the quoted sentence and confirm it really does appear on that page.

The second check is the one people skip. The link is fine and the page exists, but the quoted wording is its own reworking — that error is far harder to spot than a dead link, because it survives a first glance.

The boundaries of retrieval are yours to draw

In product terms the judgment is concrete: when provenance matters, the retrieval layer has to be yours to hold. "Where to go looking" must not be left to the model's discretion either; you write down which libraries, which documents, which time range are allowed.

Tightening the boundaries does two things at once. It leaves the model no room to fill gaps from memory, so the checkable share of the answer steps up immediately. And when your own data sources change, refreshing the material is a matter of swapping documents, not waiting for the next training run.

Go the other way and let it decide for itself which document to consult, and you are stacking a second layer of uncertainty on top of one you already have: it might dig up a clause from three years ago, or it might never search at all, and in your logs those two cases look identical. If the material is internal to a company, this layer has to be built even earlier — that is the argument in Align the Domain Before RAG.

Facts and transformation run on two separate paths

Lay the division of labour out and the choice stops being hard. When you need to establish what something is — a fact, a number, a source, the current state — go to search or to your own retrieval, because what you need is checkability. When you need to process something already established — rewrite, summarize, translate, reformat, pull a conclusion out of material — use a model, because what you need is transformation.

The order is not reversible: establish the facts by retrieval first, then let the model work on them. Reverse it and you are asking a component with no sources to manufacture facts, then trying to retrofit sources onto them. The two often appear in one feature, but appearing back to back is not the same as being interchangeable.

What fabrication actually looks like and how to defend against it is Hallucination Is Not a Bug. How to build the retrieval layer is Retrieve Then Answer: Four Steps.