Verification Must Keep Up With Generation

When output outruns verification throughput you have three options: expand verification, slow the agent down, or lower the bar. It must never happen by default.

When output volume exceeds your verification throughput, you have exactly three options: expand verification capacity, slow the agent down, or lower the quality bar. There is no fourth. The dangerous version is letting it happen by default: nobody measured throughput, nobody declared what was relaxed, and review turned from reading into skimming.

The gap is already here

The data points line up. Google's DORA research found heavier AI adoption correlates with lower delivery stability, and over a third of developers distrust AI-generated code. In METR's controlled experiment, experienced open-source developers on their own mature projects were 19% slower with AI assistance against an expected 25% speedup, the extra time going to prompting, waiting, reading, and correcting. Security testing across more than a hundred models found roughly 45% of AI-generated code introduces known security flaws; the ability to make code run has improved sharply while the ability to make code safe has stayed flat, and the gap is widening. bytebytego-verification

The cause is not a weak model but a reversal in the cost structure: writing code stopped being the slow step, so verification became the bottleneck. And the failure modes AI introduces, security flaws and duplicated code, are exactly what type checks and happy-path tests cannot see. There is a volume problem too: AI produces larger diffs, attention gets spread thin, and a 5,000-line pull request earns a single looks good to me.

Match verification layers to feedback latency

The full chain runs static, unit, integration, runtime, baseline, end-to-end, human, and the depth is set by risk, not a fixed number of layers. A low-risk change can stop after the first two; anything touching money, permissions, or irreversible side effects has to reach end-to-end plus a human.

The matching principle: feedback latency sets the autonomous iteration rate.

• Second-level feedback, meaning compilation, type checks, and unit tests, supports high-frequency iteration: an agent can try dozens of approaches in one loop only if each is judged within seconds.
• Minute-level feedback, meaning integration tests, contract tests, and browser automation, covers more realistic behavior and costs an order of magnitude in iteration frequency.
• Human judgment is reserved for what machines cannot decide: whether the intent is reasonable, whether the trade-off is acceptable. Spending humans on automatable checks is using the most expensive resource on the cheapest work.

Shift left, and check for fake fixes

Shift left: move each class of check as early as it can go. Finding a defect in the editor or at commit time costs far less than finding it in review or production. What can be checked statically should not wait for runtime; what CI can check should not wait for a canary.

Watch for fake fixes at the same time: verify the root cause is gone, not that the error signal disappeared. Agents are creative about turning tests green: deleting the log line that reported the error, dropping a log level to debug, swallowing the exception, loosening the assertion threshold. Look for those patterns in review, and when a test assertion was modified, ask why first.

Actionability

Report only problems a developer can act on. A check that is always red with no clear next step trains the team to ignore the dashboard, worse than no check. Track the false positive rate and tune thresholds, because once false alarms erode trust, real warnings are ignored too.

Speed, precision, and coverage cannot all be maximized, so state the trade-off explicitly instead of defaulting to all three, which produces something slow, leaky, and distrusted. Which you give up depends on risk: high-risk changes give up speed, prototypes can give up coverage.

The same test applies to the eval set itself, and there the saving is throughput rather than accuracy. When OpenAI curated SWE-bench Verified it drew 1,699 tasks from the original 2,294 and had 93 Python-fluent developers inspect them one by one against five questions: is the problem statement clear, do the tests cover boundary conditions, are the tests stable, does the reference patch introduce new errors, is the difficulty reasonable. Five hundred survived, about a 71% elimination rate among those reviewed. Cutting that hard buys signal and budget together, and evaluation cost fell roughly 80%. The cost side is not abstract: one complex agent task runs for minutes to hours, and a single full pass over an eval set on a frontier model is commonly a few thousand dollars of tokens ai-agent-book. Deleting cases you cannot judge is not the same as reviewing less. The first changes a case's acceptance definition and leaves a record; the second changes nothing and just looks at less.

The signal itself has noise

Two techniques help. Mutation testing: generate code variants and run the same suite, and if nothing fails the suite is missing what the agent introduced. Property testing: assert invariants rather than examples, since AI-introduced defects often slip past hand-written cases.

Then there is the reliability of the measurement. Anthropic quantified infrastructure noise in agentic coding evals: the gap between the most generous and the strictest resource configuration was 6 percentage points at p<0.01, while the top few leaderboard entries often differ by only a few points. Resource limits change what an eval actually measures, because tight limits reward lean strategies and generous limits reward brute force, and blending them into one score hides the difference. Specify both a guaranteed allocation and a hard kill threshold, treat resource configuration as a first-class recorded experiment variable, and view leaderboard gaps below three points with suspicion until the configuration is recorded and matched. anthropic-infra-noise

Hold your own score gaps to the same standard. A pass rate measured on n cases carries a standard error of roughly sqrt(p(1-p)/n). With 100 cases at 70%, the 95% interval is about 70 ± 9 percentage points, so "new model 73%, old model 70%" is not evidence for switching; both numbers sit inside the same noise band ai-agent-book.

Comparing two configurations calls for paired analysis. Record who wins on each case and test with McNemar or a paired bootstrap rather than subtracting two independently measured success rates; pairing means both sides share the same tasks and the same random conditions. Because a single agent run varies as well, give every configuration 3–5 random seeds and report the mean and the spread. A single run is only good for screening a direction. If the expected gain is 2–3 points and the set holds a few dozen cases, expand the sample first, since the standard error shrinks as 1/sqrt(n). When several hypotheses go at once, correct for multiple comparisons by tightening the threshold or re-running the positive results independently. A gap is worth acting on when it exceeds the noise, survives paired analysis, and reproduces ai-agent-book.

One cleanup rule is easy to forget: every hard constraint has to serve both quality and delivery flow, and a constraint serving neither should be removed. Constraints accumulate and rot on their own.