Attribute Cost Down to Every Step

Optimizing total tokens hides the problem. Break cost down by run, wave, agent, model, context source, and tool, and always report cost per successful task.

Watching total token spend hides the problem. Cost has to break down by run, wave, agent, model, context source, and tool, with cost per successful task alongside. Without that split you hold a number that moved and cannot say whether the agent improved or started retrying more.

Break it down before optimizing

The dimensions: project and task type, run, wave, step, agent, skill, model, context source, tool, cached versus uncached, success versus failure.

The point is not saving money, it is locating. Which part grew — a skill retrying in a loop, a tool called pointlessly, a context block loaded twice? Three rules go with it: baseline before optimizing, never claim a win from one end-to-end run, never copy a vendor's published reduction, because your cache hit rate, context composition, and retry rate are not theirs.

It rests on instrumentation: trace from the first version — runs, agent steps, model calls, tool calls, retrievals, context builds, plus model, prompt, tool, and config versions. Instrument later and there is no baseline.

Stable prefixes and a context ledger

Cache hits require a stable prefix and a dynamic suffix: system prompt, tool definitions, and fixed knowledge blocks in front, byte-stable, everything variable behind. Do not pad the prefix to lift the hit rate — you take precision and cost down with it.

Keep a context ledger: tokens by source, hunting three wastes — duplicate loads, where every sub-agent reads the same file; useless loads, retrieved but never used; unused loads, injected wholesale and never referenced.

One precision constraint on top: filling the window is not the way out. GPT-5 holds 90 percent accuracy at a full 256K; GPT-5.4 drops to 36 percent past 500K and costs more. Context is not free capacity.

Two switches, savings that do not add

Measure the prefix and the compaction switches together. The book's control run fixed one eight-turn customer-service refund task — look up the order, the shipment, the refund policy and the knowledge base, then run risk control, issue the refund, notify, close the ticket — called gpt-4o-mini, and toggled "stable prefix" and "compress history" independently ai-agent-book:

28.3 plus 17.5 is not 30. The two switches eat into each other instead of stacking: compacting the history also shortens the prefix the cache can hit. Make every optimization individually switchable, then re-measure the combination on the full task; adding up individual saving rates buys you savings on paper.

The same run shows where cost actually climbs. In the baseline, input grew from 1,113 tokens on the first turn to 3,668 on the last, and tool results riding along in history accounted for 9,544 cumulative input tokens across the eight turns, falling to 5,248 once both switches were on. A single web search returns 2,000 to 5,000 tokens that every later turn is billed for again as input. Turn one sends 1,000 tokens, turn two sends 2,000, turn three sends 3,000, and the total is 6,000, not 3,000. Agent cost does not grow linearly; it compounds.

Budgets, routing, a deterministic front layer

Resolve the budget when the run starts, update cost at every step and tool boundary. Soft limit triggers compaction, scope reduction, or routing; hard limit stops the run or requests approval; every change is recorded. A budget with no hard limit is a dashboard.

Route models by capability, risk, context length, and modality: classification, format conversion, and summarization go to a cheap model that is good enough, but define escalation conditions and prove with evals that the policy clears the quality bar. Picking by price alone is how cost optimization crashes.

"Looks simple" is not a routing criterion either. Questions such as which is larger, 9.9 or 9.11, or whether to walk or drive when the car wash is 50 metres from home, turn into wrong decisions the moment they are classified as simple and handed to a light model. Route on measurable task types and rules, and let evaluation show that the combination earns more than the system complexity it adds — including whether particular scenarios regress. ai-agent-book

Earlier still sits a deterministic front end: vector search in front of the LLM. Semantic routing matches known intents and calls the bound tool above a threshold; semantic caching returns on a similar request; semantic guardrails reject out-of-scope requests before any LLM cost. Redis measured a full pipeline at 13 seconds and roughly 400 tokens, becoming 345 milliseconds and zero tokens on a hit redis-semantic-routing.

Details decide whether it survives: calibrate thresholds on test data, chunk long requests by sentence, filter cache metadata per user so PII never crosses accounts, grade TTL by how fast meaning goes stale. Watch negation sensitivity — in general embeddings "what X is" sits close to "what X is not", and the cache returns the opposite answer. None of it is a silver bullet. Where that "deterministic first, model second" layer belongs on the complexity ladder, and the same measurement in full, is covered in Do You Need an Agent at All.

Cost per success is the number

Cost per successful task is total spend divided by tasks completed successfully, and it beats single-run cost because a task that fails ten times and succeeds twice is expensive however cheap each attempt looks — failed attempts burn tokens too.

Optimization is bounded by quality, safety, and stability: watch quality and cost per success together, never minimize tokens alone. Cursor tried a more expensive model for context summarization, ran an online A/B test, found almost no quality gain, and shelved it; offline evals had not shown it cursor-agent-harness. The mirror-image lesson: dropping default reasoning effort from high to medium cut latency and tokens, users reported the model getting dumber, and it was rolled back. Choosing a lower-intelligence default on someone's behalf is the wrong trade.

The hidden bills of multi-agent and setup

Multi-agent systems duplicate loads: every sub-agent reads the same files, tool definitions, and knowledge blocks, doubling tokens without adding information. Sharing a prefix and giving each sub-agent only what it needs beats switching to a cheaper model.

Setup is the other invisible line item. Coupled, every session pays full container setup — clone the repo, start processes, fetch pending events — even sessions that never touch the sandbox, and that dead time lands on time-to-first-token. Decoupled, the container is provisioned on demand: p50 TTFT fell about 60 percent, p95 over 90 percent anthropic-managed-agents. The environment-side of that bill — snapshots, splitting install from start — is what Environment Feedback Sets the Ceiling covers.

Attribution, budgets, and routing sit on traces, and that is not worth building for a script that runs a few dozen times. Take one high-frequency task and ask whether you can split its cost by skill, separate successful from failed spend, state cost per success, and baseline the cache hit rate. Any no means instrument first.