This matters most for agents. A final answer can be correct while the agent searched the wrong source, repeated a costly tool call, or attempted a forbidden action before recovering. The trace supplies the evidence needed to evaluate those hidden steps.
What does a trace-based evaluator inspect?
It reads the spans from LLM tracing: inputs, model calls, retrieval results, tool arguments, timing, errors, and outputs. The evaluator asks whether the execution path met the contract, not merely whether the final text sounded good. Agent trajectory is the path being assessed.
How does it fit into an eval suite?
Use exact checks where a correct path is known, such as a mandatory policy check before a tool call. Use a rubric when several paths can succeed. Trace-based evaluation turns a vague failed run into a concrete regression case. A trace turns a final-answer verdict into evidence about the steps that caused it. AI agent evals combine outcome, trajectory, and component measures, while LLM observability preserves the production evidence that feeds the suite.