Layered Loading and Handoffs
Keeping the whole manual resident is wasted attention. Load in five layers by timing and scope, and pass state between phases as a structured handoff artifact.
Keeping the whole manual resident does not make an agent safer. Every resident rule competes with the task in front of the model, and a rule set with no versioned maintenance rots on schedule. Load in layers instead: inject by read timing and task scope, and pass state between phases as a structured handoff artifact.
Every resident rule competes for attention
OpenAI built an internal product with Codex in five months: three to seven engineers, around 1500 pull requests, roughly a tenth of the usual timeline. Their first attempt dumped every rule into one monolithic AGENTS.md and failed immediately. It crowded out task context, made everything look equally important, rotted quickly, and made it impossible to verify whether any rule was followed. openai-harness
Those four failures generalize well beyond AGENTS.md. "Everything looks equally important" is the fatal one: when "never probe data YOLO-style" sits beside "how to name functions" at equal weight, the model has no reason to honor the first.
When each layer loads, and when it is released
| Layer | Contents | Loading policy |
|---|---|---|
| Core | Role, P0/P1 summary, current objective, stop conditions | Every run |
| Scoped rules | Current directory, component, and stack rules | By project scope |
| Phase context | Current phase contract, checklist, input artifacts | On phase entry |
| On-demand reference | Focused specs, examples, historical evidence | On trigger |
| Artifact data | Large results, full logs, files, datasets | Partial reads only |
Two disciplines make the table real. Maintain a required-read list for material a phase cannot skip, and release context with no downstream dependents when the phase ends. Without release, layering is pure addition: you load five layers and drop none.
What a handoff artifact carries
State crossing a phase boundary travels as a structured artifact carrying at least: target scope, verified facts, open assumptions, current checkpoint, unconfirmed side effects, approval status, stop conditions, and who receives the result.
Two fields get dropped and cost you. A dedicated field for abandoned paths stops the next phase repeating a dead end; concrete IDs, not generalizations, because "processed those orders" is useless to the next step.
The test is blunt: handing over a paragraph of natural-language summary is dressing up context loss as division of labor. Downstream phases must publish their own contract rather than depend on upstream chat history, or every reworded sentence upstream becomes an outage downstream.
Stable prefix, dynamic suffix
Split the prompt by how stable each part is. Stable prefix: platform rules, organization policy, project rules, agent contract, tool definitions, output contract. Dynamic suffix: current objective, phase context, selected artifacts, working state, latest user input.
The order is not aesthetic. Dynamic content must follow stable instructions, because otherwise the prefix changes every call, the cache misses entirely, and you pay full price for the same rules repeatedly. Keep the ordering of system rules, project rules, agent definitions, and tool schemas stable too, and never pad in irrelevant content just to preserve a cache hit.
Cache economics has a priced example behind it. A support agent running 100,000 conversations a day behaved normally until an engineer added Current time: {{now}} to the system prompt so the model would "know" what time it was. The next day, time-to-first-token across every conversation went from 0.5 seconds to 3 or 5, and the monthly inference bill roughly doubled ai-agent-book. No logic changed and no model was swapped. The cache matches token byte sequences, so everything after the first differing token has to be recomputed — and the system prompt sits at the front, which made that recomputation nearly the whole input.
The same mechanism quietly decides two designs teams usually get wrong. Runtime conditions do not belong before the cache boundary. Content ahead of it is shared across users and sessions, so every binary condition doubles the key count: three of them (macOS or Linux, normal or debug, Chinese or English) already produce eight keys. Dynamic elements go after the boundary, which means prompt order is decided by cache economics first and by semantics second. Fix your few-shot sets per task type. Examples sit near the front, so retrieving the "most relevant" ones per request rewrites the prefix every time; two or three examples that cover edge cases beat ten near-duplicates, which additionally dilute attention on the rules themselves ai-agent-book. Sub-agents add one more rule: prompt, tool definitions, model config, message prefix, and thinking configuration must match the parent byte for byte, or none of that cache carries over.
Progressive disclosure is the same idea from the other side: use the rules file as a map pointing at structured docs read on demand, rather than unrolling them into context.
Three cases: the map, the catalog, the trio
OpenAI compressed AGENTS.md to about 100 lines, used purely as a table of contents pointing into structured docs: give Codex a map, not a thousand-page manual. Alongside it, background jobs scanned for codebase drift, and a doc-gardening agent retired stale documentation. openai-harness
Stripe's Kai platform hit the wall at scale. Once more than 150 skill descriptions sat in the system prompt, quality degraded even on frontier models. The fix was two-stage loading: let the LLM pick the needed skill from a catalog, then let that skill's allowedTools decide which tools load, with foundational skills still injected by default. One counterintuitive finding came with it: at their scale, plain LLM selection beat RAG prefiltering. langchain-stripe
Long-horizon work leans on durable memory in the filesystem. Codex ran 25 hours uninterrupted, 13M tokens and 30,000 lines, on three files: plan.md, with milestones, an acceptance command each, and a stop-and-fix rule; implement.md, the runbook; and documentation.md, an audit log you can return to hours later. openai-long-horizon
The cost and how to check
The cost is infrastructure: a layered loader, required-read lists, contract versioning, and a way to retire stale docs. Small projects and single-file tasks do not justify it; overhead buries you first.
Two checks settle it: how many resident rules are irrelevant to the current task, and can the downstream phase name what the previous phase abandoned? Above a third, start layering. Blank on the second, you handed over nothing.