What Actually Goes Into Context

Context is not a chat log. It is a dozen kinds of sources competing for one budget, and each one needs a declared origin, trust level, and pruning rule.

Context is not a chat log. It is a dozen kinds of source competing for one budget, and each kind has to answer three questions: where did it come from, how much is it trusted, and when can it be cut. Without those answers you are not engineering anything; you are rephrasing sentences and hoping.

Context is not a chat log

Treating context as a transcript leads to two bad moves. You keep appending history and tool output without selection, and when the window fills up you reach for a larger model. The first manufactures noise. The second hides the cause. Both let you walk past the real problem repeatedly.

That is the difference between prompt management and context management. Prompt work asks how to phrase this sentence. Context work asks what this call is entitled to see, who is allowed to override whom, and where the rest went. The second is an engineering problem with a budget table, a lifecycle, and a pruning log.

Twelve sources, each with an owner

A modern agent carries at least twelve: system instructions, tool definitions, user input, session history, business state, retrieved data, files, memory, intermediate results, plan, budget, and environment feedback.

Declare the origin, trust level, retention, and pruning rule for each. This is not documentation overhead; it is the ability to do a postmortem. When the agent answers wrong in production you need to know what it saw, what got cut, and what triggered the cut. Whatever you cannot declare is what you will be guessing about at 2am.

Tool definitions are the most underestimated line item. A tool costs its resident schema tokens plus selection reasoning, argument construction, execution, result injection, and then the same history carried again on every later step. Connect enough MCP servers and schema alone eats a large share of the window, and you pay for it on every single call.

Structure and descriptions are not decoration

An ablation study run on Tau-Bench puts numbers on this. Leave every rule in place and change only the presentation — drop the heading levels, break an ordered procedure into an unordered pile of clauses — and task success falls by more than 30%. The agent starts violating the rules that matter: once "verify identity before issuing a refund" was scattered, it just issued the refund. Keep the function signatures and argument definitions but delete the descriptive text, and tool-call errors rise 45%, mostly invalid values and misread arguments ai-agent-book. In the same study, recasting the voice as a boastful showman or stuffing it with emoji barely moved completion at all.

Read the three results together: the model tolerates how you talk, not how you organize. A tool description is not documentation for humans; it is part of your argument accuracy. And when a team goes hunting for tokens, heading structure and description text are the first things on the block — which is why those are the first two things to break in production.

Six trust tiers

From highest to lowest: SYSTEM, ORGANIZATION, APPLICATION, USER, RETRIEVED, EXTERNAL_UNTRUSTED. One rule carries most of the weight: user input cannot change system permissions. It settles conflict resolution in prompt assembly. Not "whoever appears first wins," but "whoever ranks higher wins."

RETRIEVED and EXTERNAL_UNTRUSTED are often merged and should not be. The first comes from a knowledge base you operate, with versions you can trace. The second comes from web pages, email, and tool output, where the payload itself may be the attack, so it needs a source, a timestamp, and handling as untrusted. Organization policy ships only through admins, and project rules get versioned, because otherwise you cannot answer which rule version changed this behavior.

Subtract output and tool definitions first

Run a preflight before every call: model window - output reserve - tool definitions - system and org rules - current objective = allocatable context budget.

The common mistake is treating the whole window as room for history. Starve the output reserve and the model gets truncated mid-sentence. Squeeze the tool definitions and it starts calling tools that do not exist. Shrink in a fixed order when you are over: debug previews, then refetchable full data, then low-relevance summaries, then indirect dependencies, then history, then working memory. Objective, safety rules, approval state, and the current step go last. If you are still over after that, return CONTEXT_BUDGET_EXCEEDED rather than pretending it fits.

Also hard requirement: log what was pruned and how much. Without a pruning log you cannot tell whether the model missed the answer or saw it and ignored it, so you keep guessing.

System instructions do not just consume budget; they are dangerously sensitive. In one Anthropic incident, a single line asking for brevity cost two model generations 3% each on coding quality, and internal evals missed it because they did not cover the affected scenarios. anthropic-postmortem

A long window is not more usable capacity

This is the most expensive misconception. In one widely cited measurement, a model held roughly 90% accuracy at a full 256K context; its successor advertised a 1M window but accuracy fell to 36% past 500K, at a higher price redis-semantic-routing. Physical capacity is not usable capacity.

The Qwen platform team quantified it more directly. By step eight, 70% of the context was raw JSON from earlier tool calls, 20% was conversation history, and only 10% was instructions the current step actually needed. The same team spent two weeks evaluating a larger model with no metric movement, then one week on context management for a 40% improvement. qwen-harness

The cost is real engineering: external storage, pruning logic, pruning logs, per-source token accounting. Short single-step tasks do not justify it. The check takes minutes. Dump the prompt at step eight and count how much of it the current step actually uses. If it is under a third, fix the context before you shop for a model. How that 70/20/10 mix deteriorates on its own as steps pile up is what How Context Gets Dirty breaks down.