Turn Failures Into Engineering Assets

A failure is fixed only when it becomes a schema, a validator, a gate, a linter, or a regression test. A warning in a prompt just reassigns the constraint to next run's chance.

A failure is fixed only when it becomes a schema, a validator, a gate, a linter, or a regression test. Adding "be careful not to..." to a prompt fixes nothing — it hands the constraint to next run's probability.

A warning line is not a fix

The chain is fixed: Failure, Incident, Root Cause, Corrective Action, Test/Eval, then Rule, Gate, Binding, Tool, or Experience, then release and rollback.

Four rules: record the failing scenario, expected and actual behavior, and evidence; prefer changing system structure; convert the fix into a regression test or eval; never close a root-cause fix with a warning alone.

Why a warning does not count: no enforcement mechanism, no signal. It cannot block the next occurrence or tell you whether it ever worked. A gate and a validator do both — they block, and they count.

One trap, from four-layer skill validation: the result is correct but a critical verification step was skipped — it will fail next time. A correct result is not a fix; a correct path is.

Two shapes for a regression case

Production traces feed the regression set through three channels: a user who explicitly corrected the agent, a user who gave it a thumbs down, and cases found afterwards by state checks, rule validators, or an LLM review. All three pass through failure attribution before they become a case. This is the most expensive source and also the most accurate one, because it comes from problems users actually hit ai-agent-book.

A case then takes one of two shapes. End-to-end regression runs from the initial state to the final assertions: broad coverage, slow. Trajectory-prefix regression freezes the context, conversation, tool results, and environment state, and asks the agent for only the next observable action or two: cheaper, and it isolates a single policy or a single tool. For a production agent that must be highly reliable, the prefix set is often the one to build first — provided a failure taxonomy exists so cases can be sliced by class, which is what the four smoke-failure classes in What to Measure Before You Ship give you ai-agent-book.

The attribution record has to be structured rather than a one-line "reason for failure": JSON or YAML citing step numbers, tool names, and observed evidence, separating root cause from consequence, and recording recoverability and confidence. A typical case: edit_file returns an old_string match failure, and the agent retries three more times without ever writing the file. The primary cause is the file-editing and tool-call error; those three retries are consequence, not three independent causes. When several categories appear together, take the earliest one that explains the later failures as primary and keep the rest as secondary. At scale, let rules screen suspect traces first and the model localize afterwards — cheaper and more accurate than feeding every trace to an LLM. Three filters can be written as rules today: cross-check the completion claim against the commands actually executed; check whether a diff touches test assertions or skip markers; check whether a diff changes a public API or schema with no migration file ai-agent-book.

From lesson to active rule

The escalation chain runs Incident, Lesson, Cross-case Pattern, Candidate Rule, Regression Eval, Scoped Activation, Active Rule.

Two decisions decide whether this yields an asset. A pattern has to be validated across independent cases, never promoted after a fixed number of occurrences — the same root cause three times may mean the same task ran three times. P0/P1 promotion is human-approved, with every candidate rule carrying a scope, owner, false-block risk, and enforcement mechanism.

P0/P1 rules also need a stable ID and version, deterministic checks that outrank a grader, and a blocking gate wired into CI, guarded so the agent cannot skip it.

Anthropic's incident response had this shape: tighten the system prompt change process, run the full per-model eval suite plus ablation on every change, build a prompt-change audit tool, and give anything trading intelligence for performance a soak period and gradual rollout anthropic-postmortem.

A more extreme sample: Steve Yegge ran 50 to 60 agents as an organization and watched them build 450 "legal artifacts" — a constitution, case rulings, and mechanisms he calls fences, gates, and tripwires. A rule progresses from custom to advisory to written law to mechanical enforcement, tightening one level each time it is violated. His key finding: prune the rule system continuously or entropy wins yegge-fences.

The experience lifecycle

Trace capture, redaction and trajectory assembly, failure clustering, candidate, offline eval, shadow or canary, activation, recall, monitoring, then downweighting, pausing, or rollback.

Four boundaries: mined results start as CANDIDATE and reach ACTIVE only through an independent eval; production runs must not generate and activate an experience in the same breath; one-click pause and rollback are mandatory; an experience may never bypass security, permissions, approvals, or business rules.

Recall has an order: permission and scope first, relevance second, only a few executable experiences under a token budget. Never pour the full historical trajectory into the current context — an expensive way to manufacture noise.

Worth copying: SKILL.md self-evolution behind a four-layer gate — Target (a failing case improves), Guardrail (no passing case regresses), Holdout (a hidden set never used in diagnosis; reject any drop over 1 percent F1), Verify (text quality). Diagnosis is deterministic rules; the LLM has one job, turning it into a diff. A taboo blacklist carries the signature of every rejected patch across versions and branches alibaba-skill-evol. How those four gates and the three-way data split fit into an eval loop, and which constraints you cannot leave out, is the full subject of Online and Offline Evals.

Harness entropy and rule debt

With strong models, redundant instructions become quality loss: over-specification narrows the search space, gets followed literally, and over-triggers. Maintenance shifts from prescribing the path to defining success criteria — pin the goal, criteria, and constraints, leave the path to the model.

Check regularly: rule length, duplicate and conflicting rules, rules that never fire, dead commands and skills, gates always skipped or always passing.

Deletion needs the same discipline: cleanup produces a diff and evidence, and removing a rule or gate still runs the regression eval. A system that only grows becomes untouchable.

Where the flywheel sits

Self-evolution has three layers: artifact iteration, which compounds nothing; harness self-improvement — memory, skills, prompts, tool configuration — where one change pays off on every later run and rolls back; and model evolution, the most expensive and still mostly research. Most teams should invest in the middle one.

One claim governs the rest: the trustworthiness of your evals outranks the sophistication of your system. Bad positive feedback makes an agent learn the wrong thing faster, accelerating toward a cliff. Calibrate evals first, scale machine scoring once agreement with humans clears a threshold, then automate.

The flywheel costs infrastructure — traces, evals, canaries, rollback — plus pruning, and does not pay where output cannot be judged. Count how many of last month's failures became a gate, validator, or regression test. Near zero means the system is not getting better, just older.