Layered Loading and Handoffs

Keeping the whole manual resident is wasted attention. Load in five layers by timing and scope, and pass state between phases as a structured handoff artifact.

Keeping the whole manual resident does not make an agent safer. Every resident rule competes with the task in front of the model, and a rule set with no versioned maintenance rots on schedule. Load in layers instead: inject by read timing and task scope, and pass state between phases as a structured handoff artifact.

Every resident rule competes for attention

OpenAI built an internal product with Codex in five months: three to seven engineers, around 1500 pull requests, roughly a tenth of the usual timeline. Their first attempt dumped every rule into one monolithic AGENTS.md and failed immediately. It crowded out task context, made everything look equally important, rotted quickly, and made it impossible to verify whether any rule was followed. openai-harness

Those four failures generalize well beyond AGENTS.md. "Everything looks equally important" is the fatal one: when "never probe data YOLO-style" sits beside "how to name functions" at equal weight, the model has no reason to honor the first.

When each layer loads, and when it is released

Two disciplines make the table real. Maintain a required-read list for material a phase cannot skip, and release context with no downstream dependents when the phase ends. Without release, layering is pure addition: you load five layers and drop none.

Query the index before reading the body

How does the fifth layer's "partial reads on demand" actually work? Not by loading the whole file and letting the model search it, but by querying a lightweight index first and reading only the selected passages: Query → Index Search → Top-K Metadata → Selected Content Read, falling back to source search when the index is missing, stale, or low-confidence.

Three disciplines come with it. The index must be tied to the source file's version — a stale index pointing at deleted content is more dangerous than no index. Only the top-ranked few candidate passages get read; the point of Top-K is controlling the bytes that enter context. And large codebases narrow the file range with ASTs, symbol indexes, and dependency graphs instead of pasting the directory tree into the prompt. This is the rules-file-as-map principle applied to a different object: give the model a cheap lookup structure first, and pay for the body only on hits.

What a handoff artifact carries

State crossing a phase boundary travels as a structured artifact. The eight base fields are covered in An Answer Is Not Accountability; the two that get dropped and cost you are a dedicated field for abandoned paths, which stops the next phase repeating a dead end, and concrete IDs, not generalizations, because "processed those orders" is useless to the next step.

The test is blunt, and An Answer Is Not Accountability states it plainly: handing over one paragraph of natural-language summary dresses up context loss as division of labor. Downstream phases must publish their own contract rather than depend on upstream chat history, or every reworded sentence upstream becomes an outage downstream.

The Taobao venue agent adds a third requirement, and it is the one that actually breaks in practice: hand over a conclusion plus the archived location of the execution record. A subagent that reports only "the product set was created" leaves the main agent knowing the ID but not which items were excluded on price and which on skin-type constraints. When the user relaxes the budget next turn, the first group can be re-queried and the second must stay excluded. With only the conclusion and no conditions or re-readable evidence, the next turn re-guesses judgments already made. Writing the process into the workspace while the conclusion enters the main loop is the fix taobao-venue.

Stable prefix, dynamic suffix

Split the prompt by how stable each part is. Stable prefix: platform rules, organization policy, project rules, agent contract, tool definitions, output contract. Dynamic suffix: current objective, phase context, selected artifacts, working state, latest user input.

The order is not aesthetic. Dynamic content must follow stable instructions, because otherwise the prefix changes every call, the cache misses entirely, and you pay full price for the same rules repeatedly. Keep the ordering of system rules, project rules, agent definitions, and tool schemas stable too, and never pad in irrelevant content just to preserve a cache hit.

Cache economics has a priced example behind it. A support agent running 100,000 conversations a day behaved normally until an engineer added Current time: {{now}} to the system prompt so the model would "know" what time it was. The next day, time-to-first-token across every conversation went from 0.5 seconds to 3 or 5, and the monthly inference bill roughly doubled ai-agent-book. No logic changed and no model was swapped. The cache matches token byte sequences, so everything after the first differing token has to be recomputed — and the system prompt sits at the front, which made that recomputation nearly the whole input.

The same mechanism quietly decides two designs teams usually get wrong. Runtime conditions do not belong before the cache boundary. Content ahead of it is shared across users and sessions, so every binary condition doubles the key count: three of them (macOS or Linux, normal or debug, Chinese or English) already produce eight keys. Dynamic elements go after the boundary, which means prompt order is decided by cache economics first and by semantics second. Fix your few-shot sets per task type. Examples sit near the front, so retrieving the "most relevant" ones per request rewrites the prefix every time; two or three examples that cover edge cases beat ten near-duplicates, which additionally dilute attention on the rules themselves ai-agent-book. Sub-agents add one more rule: prompt, tool definitions, model config, message prefix, and thinking configuration must match the parent byte for byte, or none of that cache carries over.

Progressive disclosure is the same idea from the other side: use the rules file as a map pointing at structured docs read on demand, rather than unrolling them into context.

Three cases: the map, the catalog, the trio

OpenAI compressed AGENTS.md to about 100 lines, used purely as a table of contents pointing into structured docs: give Codex a map, not a thousand-page manual. Alongside it, background jobs scanned for codebase drift, and a doc-gardening agent retired stale documentation. openai-harness

Stripe's Kai platform hit the wall at scale. Once more than 150 skill descriptions sat in the system prompt, quality degraded even on frontier models. The fix was two-stage loading: let the LLM pick the needed skill from a catalog, then let that skill's allowedTools decide which tools load, with foundational skills still injected by default. One counterintuitive finding came with it: at their scale, plain LLM selection beat RAG prefiltering. langchain-stripe

Long-horizon work leans on durable memory in the filesystem: milestones, an execution runbook, and a decision audit log written out as files so the agent always knows where it stands. The 25-hour uninterrupted run OpenAI reported held together on exactly those three files — see Long Tasks Need Handoffs and End States. openai-long-horizon

The cost and how to check

The cost is infrastructure: a layered loader, required-read lists, contract versioning, and a way to retire stale docs. Small projects and single-file tasks do not justify it; overhead buries you first.

Two checks settle it: how many resident rules are irrelevant to the current task, and can the downstream phase name what the previous phase abandoned? Above a third, start layering. Blank on the second, you handed over nothing.