06 Apr 2025 AI Engineering 5 min read

Evaluating Agents: Judge the Trajectory, Not the Answer

Agent evaluation in 2025 means judging trajectories and tool calls, not just final answers. We define step-level metrics, per-task success predicates, and reliability measurement with pass^k, drawing on τ-bench, WebArena, BFCL V3, and SWE-bench Verified — and state plainly what trajectory evaluation does not deliver and why it is harder than RAG evaluation.

The final answer is the wrong unit of evaluation

An agent, in Anthropic's December 2024 definition, is a system in which a language model dynamically directs its own process and tool usage. Concretely it is a loop: select a tool, fill in arguments, observe the result, decide again. A production run contains five to thirty such steps. An evaluation that reads only the last message ignores most of the behavior — and most of the failure surface.

The gap between answer quality and task success is measurable. WebArena (July 2023) put agents into 812 realistic web tasks and checked functional outcomes, not transcripts: the best GPT-4-based agent completed 14.41% of tasks end to end; humans completed 78.24%. An agent can produce a fluent final message and a wrong environment state at the same time. Only one of the two matters.

A change lands — prompt, model or retrieval. 1/4

Why agent evals are harder than RAG evals

A RAG pipeline has a fixed shape: query, retrieval, generation, one pass. Its standard metrics — context precision, context recall, faithfulness, answer relevance — are computable from four static artifacts: the question, the retrieved chunks, the answer, and a reference. Nothing in the pipeline changes state. Every run of the same input is structurally identical.

Agents break every one of these assumptions. Step counts vary. Several distinct trajectories can be correct for the same task. Tool calls have side effects, and a wrong write is not graded like a wrong sentence — it may be irreversible. Sampling makes runs non-repeatable. An agent eval must therefore score sequences and end states, accept path variance, and quantify run-to-run variance. A RAG scorecard has none of these obligations.

Trajectories as first-class evaluation objects

A trajectory is the complete ordered record of one run: every model turn, every tool call with its arguments, every tool result, and the final environment state. It must be persisted in full. By March 2025 the tooling assumes this: OpenAI's Agents SDK (released 11 March 2025) traces runs by default, LangSmith stores them as trees, and the Model Context Protocol (November 2024) gives tool calls a uniform shape worth logging.

Scoring a trajectory means comparing it to expectations at a chosen strictness: exact match against a reference sequence, in-order match that tolerates extra steps, any-order match, or precision and recall over the set of tool calls. Strictness is a design decision with known trade-offs. Exact match penalizes legitimate detours. Set-based matching misses ordering bugs — a write before a validation passes the set check and fails in production.

Step-level metrics for tool calls

Below the trajectory sits the individual step, and the useful step metrics are countable rather than judged: correct tool selected, arguments valid, call succeeded, call redundant, error recovered. These counts localize failures. τ-bench's failure analysis is a model here: GPT-4o made 0.46 tool calls with non-existent IDs per retail task; GPT-3.5-Turbo made 2.08. That single number separates an argument-grounding problem from a policy problem.

BFCL V3 (19 September 2024) shows that step and outcome checks can stay mechanical even in multi-turn settings. It runs two checks at the end of every turn: a state-based comparison of the backend after execution, and a subset-matched response check for read-only calls that leave no state trace. No judge model is involved. Deterministic checks first, LLM judges only where no predicate exists — that ordering keeps evals cheap and debuggable.

Level	Question	Example check
Final answer	Is the reply correct?	String match or LLM-judged comparison
End state	Did the environment reach the goal?	Database diff (τ-bench), unit tests (SWE-bench Verified)
Trajectory	Was the path valid and efficient?	In-order tool-call match, step count
Single step	Was this tool call right?	Tool selection, argument validation, error handling

Success criteria belong to the task

Generic scores do not transfer between tasks; success must be defined per task, as an executable predicate. τ-bench annotates each task with a goal database state and compares end states, which stays robust when the simulated user rephrases the request. SWE-bench Verified (13 August 2024) attaches human-validated unit tests to each of its 500 issues. Both reduce the question of success to a mechanical check.

We apply the same rule in our own agent projects: the success predicate is written before the agent, is part of the task definition, and runs without a judge model. Forbidden actions — destructive writes, policy violations — are listed alongside it and count as hard failures regardless of the final state. If success cannot be stated as a predicate, the task is not ready for an agent.

Measuring reliability with pass^k

pass@k asks whether at least one of k trials succeeds; it rewards lucky runs. τ-bench (June 2024) proposed pass^k: the probability that all k independent trials succeed. The difference between the two is the reliability gap. GPT-4o with function calling reaches pass^1 of 61.2% on τ-retail and 35.2% on τ-airline; on τ-retail its pass^8 drops below 25%.

A customer-facing agent meets the same request many times, so pass^k is the honest production metric. It also sets the eval budget: each task must run several times. The τ-bench authors report roughly 200 US dollars in API fees for a single trial pass over the retail domain. Reliability measurement is a line item, not a by-product.

What trajectory evaluation does not do

Trajectory evaluation samples behavior; it does not bound it. A passing suite is evidence, not a safety certificate. It does not remove the judge problem: where an LLM grades trajectories, its verdicts must be calibrated against human labels and disagreement rates reported. It does not solve distribution shift: offline task sets age, and production traffic drifts away from them.

It is also not cheap. Multiple trials times long trajectories multiply token costs, and state-based checks require a resettable environment — seeded databases, mock APIs, fixtures — whose engineering effort is comparable to the agent itself. Teams should budget for this explicitly rather than discover it mid-project.

Outlook from March 2025

Three developments seem likely from where we stand. First, trace formats converge: MCP is spreading — OpenAI added support to its Agents SDK in late March 2025 — and OpenTelemetry's generative-AI semantic conventions are maturing. Standardized traces make trajectory evals portable across frameworks.

Second, step-level checks move into CI. Once trajectories are structured data, asserting on them is a unit test, and we expect agent regressions to be caught by pipelines rather than by users. Third, reliability metrics enter contracts: a vendor claiming an agent works will be asked for pass^k at a stated k, not for a demo. The durable engineering artifact of this period will be the eval harness, not the prompt.