Traditional LLM evals check one input-output pair. An agent runs many steps, calls tools, and can succeed at the end while taking a badly inefficient or unsafe path there, so evaluating it needs a different frame entirely.
A single correct answer proves nothing about an agent. What matters is whether the same path holds up across a hundred slightly different runs.
What actually gets measured in an AI agent eval?
Three levels, not one. End to end asks the blunt question, did the task succeed, judged against a labeled outcome or a rubric. Trajectory level asks whether the path there was sound: did the agent call the right tool, in a reasonable number of steps, without looping or wandering off task. Component level isolates a single piece, one retriever, one subagent, one tool, so a failure traces back to the part that actually broke rather than a vague verdict that the agent got it wrong.
Code-based checks or an LLM as judge, which one actually works?
Both, for different things. A code-based evaluator is fast and exactly reproducible, good for anything with a checkable ground truth: did the agent call the right API, does the output match a schema. It penalizes a correct answer that took a different, equally valid path, which is where an LLM-as-judge evaluator earns its cost: it can score nuance, tone, or reasoning quality, at the price of extra inference and a prompt that itself needs care. Most reliable eval suites combine both rather than picking one.
Why do evals matter more for an agent than for a chatbot?
A chatbot’s failure mode is a bad answer a person reads and discards. An AI agent failure mode is a wrong action already taken, a bad reconciliation written to a record, a message already sent, so the cost of an unmeasured regression is categorically higher. Evals are what let a team ship a change to an agent’s prompt or its toolset with actual evidence it did not quietly break something, instead of a demo that happened to work once. This is also the layer that decides where a human approval gate actually needs to sit: a step the evals show fails often enough is exactly the step that should not run unattended, alongside guardrails that catch at runtime what evals alone would only find after the fact.
What does an actual eval suite look like in production?
A dataset of representative tasks, a mix of scripted and judge-based scorers, and a way to trace every step of a run back to the decision that produced it, so a regression is diagnosable rather than just visible. LangSmith is a common choice for exactly this: tracing, offline datasets, and online evaluators built specifically for agentic AI systems rather than single-turn chat. Running evals once before launch and never again is the most common mistake an otherwise careful team makes; an agent’s behavior drifts as the model, the prompt, or the underlying data changes, so the suite needs to run on every meaningful change, not just the first one.