Autonomy Ends Where Verification Ends

How much autonomy you can grant an agent equals how cheaply and reliably you can verify it. For tasks you cannot check, build acceptance first.

How much autonomy you can grant an agent equals how cheaply and reliably you can verify it. For work you cannot check, build acceptance capability first and talk about autonomy second. Reverse the order and all you get is faster mistakes.

Finishing the loop beats per-step accuracy

Evaluating an agent, the instinct is to stare at "how accurate is each step." What decides whether it works in production is something else: can it carry the task through to the end on its own.

The difference is rework cost. In a multi-stage pipeline — identify the request, analyze dependencies, handle parameters, generate assertions, produce code — every stage can fail. If a failure anywhere requires a human to step in and debug, the time the agent saved gets eaten immediately. That is routinely how an efficiency gain disappears. So the design goal is not an optimal answer at every step; it is fault tolerance across the whole chain. Ambiguous or invalid model output gets handled by structural validation and discarding bad content so later stages can proceed. Even an imperfect first attempt should produce something runnable. bytedance-nl2test

One important corollary: "occasionally produces something impressive" and "reliably produces something usable" are different capabilities. The first looks better in an evaluation and the second is what can be depended on. An in-house eval that only measures per-step accuracy will systematically overrate the clever-but-unstable implementations.

Measure verification before you measure autonomy

Two rules hold. First, completion is decided by the environment, not by the model saying "done." That is the line between answering and owning: an agent reporting that it issued a request is not the same as that request producing exactly one side effect. Side-effecting actions need an idempotency key and a receipt, and if the process crashes before the result is confirmed, you reconcile actions with unknown outcomes — never blindly replay them, or you will double-charge, double-send, or double-create.

Second, do not outsource judgments your team cannot evaluate. If people cannot recognize the right answer, neither can the agent. Tasks that resist verification need a golden set, expert spot checks, or mechanically decidable criteria before you hand them over.

Criteria must be mechanically decidable

Write success criteria so a machine can decide them: tests go green, the artifact exists, the diff is empty, the command exits 0. If you cannot write that sentence, the task is undefined — not "let us run it and see."

Keep verification separate from execution. After the agent edits code, an independent step runs tests, type checks, inspects the diff, and validates output files; only then is the work complete. For high-value results, use a separate verifier or a different evaluation path, so the producer is not grading its own homework anthropic-evals.

"Mechanically decidable" contains its own cost ladder — do not skip rungs. Deterministic checks come first: schema, types, assertions, business rules. Classifier models handle structured risks such as harmful content, privacy leaks, and injection. An LLM judge stays last, used only when the verification itself needs semantic understanding and cannot be written as a deterministic rule agents-in-action. Skipping rungs costs you twice. Under high concurrency, guardrail spend can equal or exceed the agent's own; and verifying a model with a model shares its bias — the judge gets pulled along by the very prose it is supposed to assess.

What happens when generation outruns verification

The cost structure of production code has inverted. Writing code is no longer the slow part; verification is. The price is measurable: roughly 45% of AI-generated code carries known security weaknesses bytebytego-verification.

The quieter cost is cognitive debt. When generation consistently outruns verification, you accumulate code that passes review while nobody can say why it works. That is not a bug. It is an unowned liability that charges no interest until the day it settles in full.

When verification is short, you have three options

Once verification is the bottleneck, there are exactly three explicit options: expand verification capacity, slow the output, or lower the quality bar. There is no fourth, and none of them may happen by default. Quality erosion nobody chose is an incident. The same erosion, documented and signed off, is a decision.

Verification Must Keep Up With Generation covers how to turn verification throughput into a capacity you can operate. The only claim that belongs here: whether you can widen autonomy depends on which of those three you picked, and on whether you picked it on purpose.

Turn it around, and whether a decision mechanism exists at all decides whether a refactor can be handed to an agent. CodeScene's September 2026 case is the positive sample: 300,000 lines of C refactored in three weeks — 726 files, 2,903 commits, 252,055 lines changed, about $4,000 in tokens. Two mechanisms carried it, and they map onto this article's two lines. First, a CodeHealth MCP server supplied a deterministic score, so the agent knew where to improve and whether a transformation had actually helped. Second, a replay-trace harness compared rollback state hashes frame by frame, verifying after every change that behavior still matched. That mechanism works because a decompiled game offers deterministic frame-by-frame replay codescene-refactor.

The inverse is the part worth remembering: most legacy systems have no comparable decision mechanism, which is exactly why refactoring them is risky. The replay harness works because the artifact provides deterministic replay; swap in a business system with no equivalence check and the same autonomy does not transfer. The author's own conclusion lands there — automated tests and equivalence checks are non-negotiable safeguards, and that burden falls precisely on what unhealthy codebases tend to lack.

Push the task into the verifiable quadrant

Two axes put every task in one of four cells: is the goal clear, and can the result be verified automatically. The point of a harness is not to make the model smarter. It is to shove work toward the clear-goal, auto-verifiable cell ai-agent-book.

The other cells fail in different ways. Clear goal, human-only acceptance: the bottleneck is the price of checking, and you lower it grain by grain with golden sets, expert spot checks, and mechanically decidable proxy assertions. Vague goal that happens to have an automatic metric: the dangerous one. Once a working number exists, all optimization flows to it automatically. Optimize "code quality" as a linter-warning count and a few nights of effort yield an all-green project nobody dares to change. That is not failure — it is running off course efficiently, harder to spot than failure, because the dashboard kept getting better.

Leave the path to the model

Fix only three things: the target outcome, the success criteria, and the constraints. Let the model choose the path. Directing every step is a double waste: you pay for reasoning and then decline to use the judgment you paid for.

Recalibrate this boundary after every model upgrade. Requests that were unreasonable one generation ago — porting an entire large codebase, running long autonomous tasks — may now be viable, and delegation should shift from directing steps to describing the end state, letting the system plan the route and report the trade-offs.

One more way to widen the boundary is easy to miss: fix the environment instead of writing a better prompt. Taotian's team closed the verification loop inside real environments, and the pattern is consistent — problems with open, scalable feedback get solved first taotian-env-verification. The environment side has its own article: Environment Feedback Sets the Ceiling.

Three questions to check yourself: can you state success for this task as one machine-decidable sentence? What does verification cost relative to generation? If output doubled tomorrow, would verification keep up?