Lost in the Middle

The start and end are read clearest; the middle fades as the context grows.

The context window is not a uniform canvas. The same sentence, placed at the start or the end versus buried in the middle, gets used with very different probability. And the longer the context, the bigger the gap.

Symptoms you'll recognize:

• You put an important restriction in the middle of a long prompt and it often ignores it.
• You stuff in ten documents and it only uses the first and the last.
• In a long conversation it remembers the opening setup and the most recent message, but the middle feels like a blank.

High at both ends, low in the middle

If you plotted "probability that this information gets correctly used," the shape would be a U-curve: high at both ends, sagging in the middle.

This has been observed repeatedly in the industry and is usually called "lost in the middle." It's not one vendor's bug; it's a fairly general characteristic of today's models. lost-in-middle Newer models keep improving, but the trend persists.

The critical part: it gets worse with length. At a few thousand tokens, the dip is shallow enough that you barely notice it. Push to tens or hundreds of thousands of tokens, and the dip can swallow an entire piece of key information.

Why this happens

No math, just intuition.

When the model predicts the next word, it "looks back" across all prior content and weighs the parts. That weighting is learned, and the training text has a strong pattern: openings carry the theme and setup, endings carry the conclusion and current focus, middles carry the development and filler. Think of a news article — the headline and the closing paragraph get read; the middle paragraphs get skimmed.

The model learns that prior: the two ends deserve more attention. Add more content, attention gets diluted, and the middle's "filler" parts are the first to be sacrificed.

That also explains why structured writing helps: explicit headings re-tag the middle content as "this is a new section," partially resisting the tendency to be buried.

How it shows up in a product

Three most common manifestations; recognize them and you know the fix.

• A restriction in a long prompt silently fails. You write "don't mention price" in the middle of a long prompt and it mentions price anyway. It's not being disobedient; that sentence simply carries too little weight in that position.
• More RAG context makes things worse. To be safe, people set recall to fifteen or twenty chunks. The middle ones barely get used, and the irrelevant bits mixed in also interfere with judgment. Five accurate chunks usually beat twenty mixed-quality ones.
• Settings drift mid-conversation. By round twenty or thirty, the early agreements sit neither at the start (taken by the system prompt) nor at the end — right in the least-visible zone. The symptom: it gradually stops behaving like your product.

Four countermoves

• Put important things at both ends. The most direct move. System prompt first; the task's key instructions and acceptance criteria last, right next to the user input. This is exactly why the "four components" advice places constraints and acceptance at the end.
• Repeat critical constraints. If a restriction absolutely cannot be violated, state it at the start and stress it again at the end. Repetition costs tokens, but it beats failure.
• Put in less, not more. Five precise chunks beat twenty diluted ones. The reranking step exists precisely for this — it lets you feed in only the most relevant chunks.
• Split long tasks. If one task requires the model to digest dozens of pages at once, don't cram them in. Break it into rounds: first extract points section by section, then synthesize from those points. Every round's context is short, so lost-in-the-middle stops being a problem.

Last point: a long window is a capability, not a permission. A model supporting 1M tokens doesn't mean you should fill it. What you put in and what it actually uses are two different curves.