Lost in the Middle

Attention is not spread evenly across a long context: both ends are read most reliably, and the middle fades as length grows. Position is a design decision.

The context is not a uniform canvas. Set the same sentence at either end rather than burying it mid-way and the odds it gets used differ widely — and the longer the context, the wider that gap. Since the model will not read evenly, arranging position is your job. lost-in-middle

After reading, you should be able to answer:

• Why does a constraint written mid-prompt quietly stop working?
• Why do more retrieved passages often make the answer worse?
• What belongs at the start, and what has to go at the end?
• Which long tasks escape this effect entirely?

Position is a weight

Draw a curve with the position of a piece of information on the horizontal axis and the chance it is used correctly on the vertical one, and it comes out roughly U-shaped: high at both ends, sagging in the middle. This is not one vendor's defect; it is a fairly general rule of how these models behave — newer generations keep improving, but the rule itself has not gone away.

How likely the middle is to be skipped tracks length directly. At a few thousand tokens the dip is too shallow to notice. Push to tens or hundreds of thousands and it runs deep enough to bury a whole key fact. So "the window got bigger" also enlarged the surface area this effect covers.

Where the curve comes from

Before predicting the next word, the model sweeps back over everything ahead of it and decides how much weight each part deserves, and that weighting was learned. The text it learned from follows one very strong pattern: an article tends to open with its subject and setup, close with its conclusion and current focus, and leave development and filler to the middle. The model carries that pattern as a prior — the two ends count for more. Layer on the fact that attention gets diluted as content grows, and the parts sitting mid-context that read as "filler" are the first to be given up.

That is also where structure earns its keep: putting explicit headings on middle content re-tags it as "a new section starts here," which partly offsets the pull toward being buried. It is part of why breaking the prompt into parts works.

Load-bearing constraints go at the ends

The mechanism reduces to a single design line: if missing this constraint voids the answer, it does not belong in the middle.

Sort content by that test:

• Start: global, unchanging across rounds. Role, product voice, long-running rules. These must be read every round, and the opening is the most reliable place.
• End: this round's, the least violable. The specific task requirement, acceptance criteria, output format. Right next to the user input is the second high-attention zone.
• Middle: the expendable. Development, examples, background, piles of reference material. The working assumption is that only part of the middle gets read.

A counterexample sticks best: put "never name a competitor in the answer" halfway into a long prompt, and the competitor's name shows up anyway. It is not disobeying; that sentence simply carries too little weight where it sits.

There is also a patch worth keeping: say critical constraints twice. If a restriction absolutely cannot be broken, state it at the start and stress it again at the close. The repeat costs extra tokens, and against the price of it failing that is a bargain.

Fewer passages usually beat more

To avoid missing anything, people raise the recall count. The result runs against expectation:

• The middle passages barely get used — tokens spent for nothing;
• Irrelevant passages mixed in also interfere with judgment, competing for attention with the correct passage sitting mid-pile;
• The longer the material, the more likely the correct passage lands in the middle, and the likelier it is missed.

A few passages that genuinely fit beat dozens of mixed-quality ones. That is not a cost-saving tip; it is an accuracy tip. It is also exactly why reranking exists — it lets you send in only the most relevant few instead of dumping in everything recall returned.

The same mechanism has a quieter face. Midway through a conversation, the ground rules agreed early sit neither at the start (the system prompt holds that spot) nor at the end, but precisely where attention is weakest. The symptom is that the assistant gradually stops sounding like your product.

Capacity comes from the window, attention does not

A long window is a capability, not a licence. How much fits inside and how much the model can genuinely put to use are two separate curves. Capacity comes from the window; attention comes from position. You control the first and can only work along the grain of the second.

The most thorough move routes around the mechanism rather than pushing against it: when one task would need the model to digest dozens of pages at once, do not cram them all in — run it over several rounds instead, pulling the points out section by section first and then synthesizing from them. Each round's context stays short, so a fading middle never gets the chance.

When the conversation itself gets long, the problem escalates from "which passage was ignored" to "which passage is no longer in the context at all." That is the compaction article. Once this runs inside a multi-step task, noise also accumulates step by step — that is the thread in How Context Gets Dirty.