A trace is the full record of one request. Each operation inside it becomes a span carrying its own inputs, outputs, timing, and token usage, nested under the step that triggered it. When an answer is wrong, the span tree shows which step introduced the error, whether retrieval pulled weak sources or a tool returned stale data.
How is LLM tracing different from logging?
Logs record flat events and confirm that something failed. Tracing preserves the causal path behind an output, which is what makes LLM observability possible: dashboards, evaluations, and alerts all read from the same structured spans. Dedicated platforms such as Langfuse, LangSmith, and Arize Phoenix exist to store and query these traces at scale.
Why do multi-step agents depend on it?
An AI agent can make many model and tool calls per request, and reasoning backward from one bad final answer gives no visibility into which call broke. With tracing, engineers walk the span tree in execution order, compare runs before and after a fix, and feed failed traces into AI agent evals as regression test cases. Reading that sequence is how we found the failures in Open Loyalty’s SDR, where odd answers turned out to be retrieval and CRM problems.