Four Layers of Evaluation
Two agents can both get the task right while one is reproducible and the other got lucky. Scoring only the outcome files them under the same score.
Two agents both finish the task correctly. One plans cleanly, takes four steps, and holds up on the next batch. The other thrashes and stumbles into the answer. Score only the final output and they tie, but the first success is reproducible and the second is luck. An eval that records only the outcome files two different levels of engineering under the same number.
Getting it right is not the same as being right
Start by separating two words. Outcome is the final state of the environment: does the booking row actually exist in the database. Transcript is the full record: outputs, tool calls, reasoning, and intermediate results. An agent saying done is not the environment having changed, and acceptance goes by outcome. This deserves its own check, because completion logic routinely trusts the model's claim.
Long-horizon agents change the unit of evaluation too. Short tasks fit the (query, ground_truth, answer) triple; long-horizon tasks need (prompt, expected_behavior, trace). expected_behavior describes the behavior the agent should reach, not just the final answer, and the trace supplies the path to evaluate. You are judging whether the thing got done and how, not how well it was phrased.
Frontier models press against the edge of that definition. Opus 4.5 found a policy loophole on the tau-squared benchmark and booked the flight by breaking the rules as written — a failure by the eval's own definition, and a better outcome for the real user. That is not a bug in the eval. It is a reminder that static evals miss creative solutions and then score them as failures, which is why human sampling cannot be replaced by automated scoring. anthropic-evals
Another tau-squared trajectory photographs the opposite error. The telecom agent policy opens with "you should only make one tool call at a time", yet message 4 issues two, looking the customer up by phone and by name together. Both env_assertions pass, reward = 1.0, and the verifier stays quiet, because that task's reward_basis aggregates the final state only. This is not an oversight in the framework. It is the price of a binary reward: process granularity traded for a single number comparable across models. The two cases mirror each other — one scores a creative solution as a failure, the other scores a rule-breaking step as a perfect run. Production evaluation usually has to do more than return a verdict; it has to say which step went wrong. ai-agent-book
The four layers
What you evaluate is a compound system of model, harness, tools, and process, not a single model. Meituan's Turing team uses a four-layer structure worth copying: meituan-agent-eval
| Layer | What it judges | What you miss if it is the only layer |
|---|---|---|
| Outcome | Was the task completed, is the output usable | Lucky hits count as success |
| Process | Is the plan sound, are the steps stable | Unstable paths still look fine |
| Efficiency | Wall time, tokens, tool calls | Expensive and cheap solutions score the same |
| Risk | Permission overreach, bad side effects, safety | Rare high-severity events vanish in the average |
Attribution has to follow the same layers: a report that hands over one score has to say whether the failure came from planning, a tool, the environment, or a skill.
The four layers also work in reverse, as the schema of a test case. In tau-squared, each task's evaluation_criteria holds exactly those checkable dimensions: env_assertions for the end state (mobile data on, speed test above 200 Mbps rated excellent), actions for whether the key calls really happened, and communicate_info plus nl_assertions for whether the user was told what they needed to be told. reward_basis then decides how those dimensions aggregate into one score ai-agent-book. The difference is concrete: the layers are not four columns you add to a report afterwards, they are four fields you fill in while writing the case, and the score is only their aggregate view.
The process layer also needs an environment that withholds something. Those same tasks keep known_info to three items — name, phone number, country — while the real fault (airplane mode on, data roaming off) sits outside it, so the agent can only reach it by asking and guiding. The simulated user has its own tools (check_status_bar, toggle_airplane_mode, run_speed_test), and the fault is on the side the carrier database cannot see. Grounding is the other half of the bargain: every answer about device state has to come from tool output, otherwise the simulated user nods along when the agent asks whether it is fixed now, and the evaluation degenerates into two models confirming each other ai-agent-book.
Bridge metrics: connecting business and agent numbers
Offline scores climbing while business impact stays unexplained is the fastest way to lose trust in an eval program. The usual cause is a missing middle layer. Business metrics such as DAU and retention do not map directly onto agent-layer metrics such as intent accuracy and retrieval effectiveness; system metrics like recall and click-through have to connect them. Someone who understands the business process must help build it, or you cannot answer why a business metric moved or why a stronger model produced no business gain. meituan-agent-eval
Human agreement is the precondition for machine scoring
Do not hand subjective criteria to a model. Break a vague metric into several rubrics, then force each toward a binary: yes, no, or unknown. The share of unknown answers probes the rubric itself — a high share means the question is ambiguous and the definition needs fixing.
Then measure two things: how often humans agree with each other, and how often the machine agrees with humans. Machine scoring does not scale until both clear a credible threshold, around 85% and 90%. Below that, machine output is machine labeling, not automated evaluation. Meituan's Beam team took human-model agreement from 62% to 92% with exactly this binarization work. meituan-agent-eval
One owner decides; disagreement becomes a strategy branch
When the standard is not aligned, you cannot tell whether a metric moved because the system improved or because the standard shifted. So one person owns the standard and settles disputes. One decider beats ten voters.
Disagreement between experts should not be forced into consensus, though. Persistent disagreement usually means the business supports several good strategies — aggressive and patient outreach can both close sales. Build the shared part into the eval system, turn the contested part into separate style or strategy branches, and evaluate each on its own test set rather than averaging the disagreement away as noise.
Do not design an elaborate system on day one. The more complex a metric, the harder it is to execute and align, so a turning data flywheel matters more than a complete taxonomy. Meituan's fulfillment agent grew from just over 20 metrics to nearly 200 in a year, fed by bad cases that expose capability limits and good cases that define quality. It grew; it was not designed. meituan-agent-eval