Two Retrieval Methods

Keywords nail exact strings, vectors handle paraphrase, and their blind spots are exact opposites. What actually moves quality is the reranking pass after recall.

Keywords nail exact strings, vectors handle paraphrase, and their blind spots are exact opposites. So real systems mix them and then add a reranking pass. "A knowledge base means a vector store" skips the last two steps — and those are the steps that decide whether the answer is right. The paper that named this pattern — retrieve first, then generate — used dense-vector recall for its retrieval stage. rag-paper

After reading, you should be able to answer:

A user pastes a tracking number and finds nothing. Which path failed?
Why run both paths — what do you lose by running one?
What does reranking do that vectors cannot, and why is it the first thing to add?
Why does stuffing in every recalled passage make answers worse?

A code that finds nothing means the literal path is missing

Two failures look identical from the outside and have completely different causes. Sort out which one you have before changing anything.

Keyword search works on literal overlap: whatever words the user types, it finds documents containing them, scored by frequency and rarity. A decades-old approach that search engines have kept using — mature, fast, and easy to explain. Its strength is exact strings: shipment tracking numbers, drawing revision numbers, device serial numbers, staff IDs. No synonyms exist for these; the literal text is the whole of the information. But the moment the user rephrases, it is lost: they write "the box won't start up," the ticket says "device fails to start," zero overlap, and it is missed.

Vector search looks at closeness of meaning, and the strengths flip: rephrased questions, casual phrasing, and related concepts all land, but exact strings do not. Why can a vector not handle exact strings? A tracking number carries little semantics in a training corpus, so as a coordinate it ends up crowded in with other digit strings, indistinguishable from them. What it excels at is "close enough," and a tracking number demands not one character of difference.

Two more soft spots: vectors are insensitive to negation ("supported" and "not supported" sit close together), and they are helpless with new terms — a feature released last week was never seen by the embedding model in training, so it has no idea where to place it on the map. Keywords have no trouble with new words at all; literal matching is enough.

Flip that around: a user pastes an identifier and gets nothing back, and the likeliest culprit is not vectors at all — it is a keyword path that was never wired up.

Read real questions before you configure anything

This section is about the order of decisions, not specific numbers — the thresholds have to come from your own data.

The last action belongs first, because it changes every value downstream: pull your users' actual queries and read straight through them. You will quickly see whether your scenario leans casual or leans on identifiers, and which way to weight the two paths. This beats tuning any parameter, and most teams have never done it.

What you settle once the logs are in front of you comes down to three things — the mechanics follow in the next two sections, and the judgments belong here:

Recall on both paths, and leave room on both. Tuning the size of one path leaves the other path's blind spot in place.

Boost passages both paths found. If a passage was recalled by both, it is more likely to be genuinely relevant, so give it extra credit.

Set a score floor after reranking. If even the top rescored passage is weak, the library holds nothing relevant, and the right move is the "I don't know" branch rather than forcing a few passages on the model to work from. For cutting wrong answers, this does more than any ranking tweak.

Merge by rank, never by raw score

Put the two paths side by side and the conclusion is direct: their blind spots barely overlap, so running one path means voluntarily giving up half your recall.

The hybrid recipe is plain, and it is one line with four steps:

Each path recalls its own batch — keywords on one side, vectors on the other, both with room to spare.
Merge and dedupe, marking any passage that appeared on both sides.
Rank the combined set as one, ready for the next step.
Rerank, keep the top few, hand them to the model.

Step three carries a detail worth knowing: the two paths' scores are not on the same scale, so you cannot add them. The common practice is to fuse by rank rather than by score — each result gets a weight from where it placed in its own path, and then the sets combine. There is no need to work through the algorithm, but this is a value you have to tune yourself.

The engineering cost is small, and most vector stores and search engines ship the capability. Research on RAG failure modes reaches the same conclusion — when a query is semantically close but literally different, relying on a single method is among the most common ways things go wrong. rag-failure-points Practice agrees: hybrid almost always beats a single path, and that shows most sharply in enterprise knowledge bases that field both casual questions and exact codes.

Only reranking actually reads the question

Recall aims at missing nothing, so over-recruit — several dozen passages across both paths. But you cannot feed all of them to the model: it is expensive and it dilutes the signal.

So add a reranker: it reads the question and each passage as a pair, judges whether that passage can answer that question, rescores accordingly, and keeps only the best few.

The key difference from vectors sits right there: vectors give the question and the document separate coordinates and then compare distances, so information is already gone the moment it is compressed into a coordinate. A reranker reads both together, sees the details, and judges considerably more accurately. Slower and pricier is the trade-off, but with only a few dozen passages per call the total stays low. The problem of similarity not being relevance is mostly corrected here.

Many teams skip this pass, and it is usually the highest-return single improvement in the pipeline. The reason is structural: no amount of tuning in recall changes anything except the candidate set; reranking is the only stage that truly reads the question. If you currently run vector search alone, adding a keyword path and then a reranker usually beats continuing to tune similarity thresholds.

When the answer is wrong, split the diagnosis in two

What this layer produces is a few candidate passages, and that decides what the model can see. So once it is configured, you should be able to answer one question: when the answer is wrong, how do you know whether it was never found or just answered badly? That is what walking the four steps forward is for. To see where these passages sit in the whole pipeline, retrieve then answer runs from indexing through context assembly.

One boundary worth keeping in mind: better recall still only guarantees "possibly relevant content." In complex settings the constraints that decide whether a judgment is right are often worded unlike the question and will not surface on their own in a retrieval result — Align the Domain Before RAG explains why that gap needs structure rather than more retrieval tuning.