Loop Engineering: The Control Surface and Its Failure Modes

A loop's reliability comes from its control surface, not its degree of automation: six components, five ways to die, each with an engineering response.

Trusting a loop's reliability to its degree of automation is the most common misjudgment. Anyone can replay a prompt on a timer in an afternoon; keeping it trustworthy in week three depends on something else — how far you pushed state, verification, permissions, and stop conditions into the engineering process. Addy Osmani frames the shift as this: you are no longer the person who prompts the agent, you are the person who designs the system that prompts it addy-loop-engineering. This article covers what goes inside that system and how it breaks; when to build one and what counts as stopped live in Loop Engineering: Admission and Stopping.

Automation level is not the point; the control surface is

A working engineering loop has six moves: read external state, decide the next step, execute the task, verify the result, write conclusions outside the conversation, and decide whether to continue or wrap up taobao-loop-engineering. Drop the fifth and it is just a session; drop the sixth and it is a token-burning timer. You cannot judge a loop by counting how many steps it automated — only by whether each step has a control around it.

The four engineering disciplines stack in the same order: prompt engineering is about how to write one sentence, context engineering about what material to give the model, harness engineering about tools, permissions, and runtime environment, and loop engineering about how all of it keeps running taobao-loop-engineering. The Harness Around the Model governs the capabilities and constraints of a single run; the loop governs triggering, state, and verification across runs — the former is the control, the latter is the scheduling and the closed loop. Scheduling without controls is making mistakes on a schedule; controls without a loop is still a one-shot tool boeckeler-harness.

The engineer's position changes; the responsibility does not. You move from operating each round to designing the loop mechanism and owning it in the end. What is worth automating, which permissions can be relaxed, which diffs must be read by a human, and where stop conditions are written — all of it is still yours taobao-loop-engineering.

Six components, each guarding one failure mode

A loop that runs is assembled from six components, and each component guards against a distinct way it can leak taobao-loop-engineeringaddy-loop-engineering.

The wake-up component is a heartbeat, not a brain. Scheduled waking is the piece most often mistaken for "intelligence": it calls the task on time, but the judgment criteria have to be written into the prompt by you. A usable wake-up prompt carries two things — feedforward constraints (what must be known before starting: which PR to look at, which directories are off-limits, which risks must stop) and feedback sensors (what must be accepted after finishing: which test to run, which browser recording to attach, which reviewer reads the diff) boeckeler-harness. Without feedforward, the agent tries everything; without feedback, it treats "I think this is fine" as done. Between "read the CI failures of the last 24 hours, only handle ones tied to the current PR, build a minimal fix only for reproducible failures, touch at most three files per round, stop and hand to a human on permission or security changes" and "check the project and fix anything broken": the first is an operating procedure, the second is a wish list — the second hands the judgment to the model's guesswork.

The isolation component solves mechanical conflicts, not review bandwidth. When several agents edit one repo, the first failure is not a reasoning failure but a file conflict, and an independent working copy per task blocks that. But parallelism belongs to loosely coupled tasks only; merging still has to be throttled by human comprehension. Eight agents opening eight PRs a day when you can carefully read two diffs does not raise system throughput — it just moves the bottleneck from "writing code" to "understanding code," and that bottleneck does not raise an error: the PRs are small, the tests may all be green.

The knowledge component moves project knowledge outside the conversation. A loop without skill files wakes up each time like a new colleague: re-guessing the test command, re-learning the directory layout, rediscovering which patterns the team rejects. What it saves is not just tokens but misjudgment — when project conventions are not written down, the agent fills the blanks with common open-source patterns, and the more unusual your project, the more dangerous the fill-in taobao-loop-engineering. Skills stay small and explicit, because the agent decides whether to load one from its description.

The tool component connects real systems, and permissions tighten as capability grows. A filesystem-only loop runs out of road quickly; real engineering state is scattered across code hosting, tickets, alerts, logs, and CI. Three layers must be kept apart here: the access protocol, the connection to a specific system, and the mechanism that bundles skills and configuration for distribution — installing a bundle does not mean permissions, audit, and approval are designed taobao-loop-engineering. As capability grows, permission design has to get stricter: reading logs and modifying a production database are not the same permission, and opening a PR is not the same permission as merging one.

The check/write component separates the doer from the judge. The agent that writes the code is unfit to be the sole referee — it just spent several rounds convincing itself this route works, and it will keep believing its own result taobao-loop-engineering. A safe split gives exploration, modification, verification, and review their own agents, with security review as its own lane. But sub-agents are not a free lunch: they add token cost and coordination cost, and are worth it only for the checking stage and for parallel read-heavy tasks; when the task is small, one implementer plus one verifier is usually enough. The splitting criterion is in When Multi-Agent Actually Pays.

The state component keeps the system alive outside the conversation. Context gets compacted, threads get interrupted, models forget, and agents retry the same dead end. An external state layer is not decoration — it is the precondition for the loop to keep running: current goal, items handled, approaches tried, reasons for failure, verifications passed, questions awaiting a human, and the entry point for the next round all need a home outside the conversation taobao-loop-engineering. Hard constraints belong in repository documents rather than platform memory — team rules must be reviewable, versionable, and shareable. One sentence: the model may forget; the system state must not.

Five ways to die, each with an engineering response

The objective function is too coarse. "Improve quality" and "optimize the experience" get translated by the agent into moves it can make: refactoring, copy edits, more tests, deleting duplicate code. Every move looks reasonable, but none necessarily solves your problem. Objectives must be written as verifiable conditions — the shape of "make these failing cases pass, do not touch the assertions, do not modify this piece of production logic"; the criterion is in Define Done Before You Write Code.

Verification gets swallowed by the agent itself. When logs run long, the agent summarizes into "most tests passed, only a few unrelated issues remain" — in a loop, that sentence should fail outright. Verification results must be structured: at minimum record which command ran, the exit code, which cases failed, and whether the stop condition is satisfied taobao-loop-engineering. One move from Amplitude's Ralph loop experiment is worth borrowing: have the agent instrument the features it ships, then use data next round to see which paths were used and where users got stuck, with a browser recording attached to every PR as verification evidence amplitude-ralph. Verification rests on evidence, not self-report.

Nobody reads the state file. Writing state is only the first step; the next prompt must explicitly require reading the state file first and deciding from it whether to continue. Otherwise it is a daily report nobody opens taobao-loop-engineering.

Parallelism manufactures understanding debt. Parallel agents easily create the illusion of doubled capacity. Whether you still understand the merged system is the harder question — reading summaries instead of diffs, this debt compounds fast taobao-loop-engineering. It is the same phenomenon as the scissors gap in Verification Must Keep Up With Generation: when output outruns verification throughput nothing errors, but you can no longer say what happened inside the system.

Cost has no ceiling. A loop's token cost is not a billing problem, it is an architecture problem — a loop with no ceiling will eventually be stopped by cost, noise, or a misoperation taobao-loop-engineering. Every loop needs a budget: at most how many attempts per round, at most how many PRs per day, how long without progress before stopping, and when it must be handed to a human. The attribution method for budgets is in Attribute Cost Down to Every Step.

Release permissions one level at a time

Layer permissions by risk instead of going straight to full autonomy: level one is read-only, allowing summarizing and classifying; level two allows generating a patch or PR but not merging; level three can take external actions with human approval at key steps; level four is reserved for low-risk, reversible, well-verified tasks taobao-loop-engineering. Most teams should not start at level four — doing the first two levels solidly already pays off, and the cost of failure is a patch awaiting review, not a production incident. The approval request itself must be verifiable (which action, which resources are affected, when it expires); details are in Sandboxes, Credentials, and Human Boundaries.

The EurekAgent paper makes the same point as environment engineering: permissions, artifacts, budgets, and human-in-the-loop are not supporting facilities — they directly change how an agent works. Bounded execution and isolated evaluation produce a different species of behavior than a hands-off setup eurekagent.

Let the harness grow

There is one kind of loop that ships no features yet makes every feature loop after it steadier: each week, read the recently merged agent PRs and review comments, and find the failure patterns that repeat — accidentally changing public interfaces, skipping migrations, tests that only cover the happy path, over-refactoring. Sort what you find into two kinds: what deterministic tools can block goes first into lint, type checks, architecture rules, or test fixtures; only what needs semantic judgment goes into updating the review skill or verifier prompt. Change one control at a time, and record what that control blocked and whether it produced false positives taobao-loop-engineering.

Many teams fall for "make the agent write more code" first, but the better first question is how to make the agent stop making the same kind of mistake. That judgment also adds one admission condition — whether this loop's output can be sedimented: a one-off conversation that fails leaves only a chat log; a good loop that fails should leave better fixtures, clearer rules, smaller skills, or a process you can reuse taobao-loop-engineering. Otherwise you are just buying the same improvisation over and over. Sedimenting reliable processes into reusable, constrained workflows instead of improvising each time is also the core argument of the AI Workflow Store paper workflow-store.

When not to build a loop

Prompting the agent directly is still useful; many tasks should be solved in a single conversation. Do not turn everything into a loop to look advanced — engineering tasks that recur, have clear boundaries, clear verification, and controllable risk are worth the engineering cost taobao-loop-engineering.

One plain standard can close the loop: does this loop leave you understanding the system more, or reading less code, judging less, and carrying less responsibility? The first is leverage; the second is just going into debt faster taobao-loop-engineering.