Technical Guide

What Is Agent Observability? Traces, Spans, and Governance Evidence

Agent observability is the practice of capturing structured, correlated records of what an AI agent did at runtime, including which tools it invoked, under what identity, with what permissions, and whether the action passed policy checks. It builds on distributed tracing concepts (traces and spans) but requires additional attributes to function as governance evidence rather than only debugging telemetry.

Explore Runtime Governance Talk to an Expert
In short

Agent observability applies distributed tracing's trace-and-span model to AI agents, but a trace only counts as governance evidence when it also records identity, delegated permission scope, tool-call arguments, and policy-decision outcomes at the point each action occurred.

What a Governance-Grade Trace Requires

OpenTelemetry spans support arbitrary key-value attributes beyond the core specification's default fields. Closing the gap between operational telemetry and governance evidence is largely a question of which custom attributes a team chooses to capture at instrumentation time.

  1. 1

    Identity Linkage

    A deterministic link between the span and the identity or credential under which the action executed, not reconstructed after the fact.

  2. 2

    Permission Scope

    The delegated permission scope in effect at the time of the tool call, attached as a first-class attribute rather than embedded in free-text logs.

  3. 3

    Tool-Call Arguments

    The tool name and arguments passed at the MCP client-server boundary, captured at the point of invocation.

  4. 4

    Policy Decision Outcome

    The allow or deny outcome of any policy check, attached to the specific span where the decision occurred.

  5. 5

    Reasoning Step Context

    Nested spans representing planning, tool selection, execution, and result interpretation, since standard semantic conventions do not yet fully specify multi-step reasoning.

Evaluation Questions for Platform Teams

  • Does the platform capture identity and delegated permission scope as first-class span attributes, not only as free-text log entries?
  • How does the platform correlate MCP or equivalent tool-call invocations with the underlying distributed trace context?
  • Can span-level data be retained and exported in a format suitable for compliance review, separate from short-lived debugging telemetry?
  • Does the solution support redaction or field-level access control on sensitive data captured within spans?
  • Can policy-decision outcomes be attached to the specific span where the decision occurred, rather than logged separately?

Agent Observability Defined

Distributed tracing was built to answer a specific question: what happened, in what order, across which services, during a single request. OpenTelemetry formalizes this as a trace, a directed representation of a request's execution path composed of spans. Each span captures an operation name, start and end timestamps, and its relationship to parent and child spans, allowing engineers to reconstruct how a request moved through a distributed system. The W3C Trace Context specification extends this by standardizing how trace-id and span-id values propagate across service boundaries through HTTP headers, so spans generated by different services can be correlated into one coherent trace.

Agent observability applies the same structural idea, a trace composed of correlated spans, to AI agent execution. Instead of representing only service calls, an agent trace needs to represent planning steps, tool invocations, the identity under which each action ran, and the outcome of any policy check applied to that action. The mechanics are unchanged. What changes is what the spans are required to contain.

Why Application Observability Was Not Built for Agent Execution

Conventional application tracing was designed around service calls and database queries, not around tool invocation by a semi-autonomous system. The Model Context Protocol (MCP), published by Anthropic as an open specification, gives agent tool calls a defined structure to work with. MCP separates hosts, clients, and servers, and defines tool invocation and resource access as explicit protocol-level messages rather than opaque application logic. That structure gives implementers a defined point at which to capture what tool was called, with what arguments, and under whose authority.

The gap is that MCP does not natively embed W3C Trace Context headers, so correlating a tool-call event with the broader distributed trace requires deliberate instrumentation at the client-server boundary. Left uninstrumented, a tool call executed by an agent can become an orphaned event: technically logged, but not connected to the trace that would explain why it happened, who initiated it, or under what delegated permission it ran.

Instrumentation and Retention Decisions

Traces as Governance Evidence, Not Just Debugging Telemetry

NIST's AI Risk Management Framework identifies documentation and traceability of AI system behavior as components supporting the Govern and Map functions, treating them as accountability mechanisms rather than pure debugging aids. That framing matters for how agent traces are designed. A trace built only to help an engineer diagnose a failed tool call will not necessarily contain the identity, permission scope, and policy outcome a security or compliance reviewer needs to reconstruct what an agent did and whether it was authorized to do it.

No single agreed standard yet exists for agent-specific trace schemas. OpenTelemetry's generative AI semantic conventions remain under active development, and field-level implementations vary across tooling. Teams building toward auditability today are, in effect, making architectural choices ahead of a settled standard, which creates some interoperability risk when trace data is later used as cross-vendor audit evidence.

Core Concepts

Four terms recur throughout agent observability work; each carries a distinct meaning in this context.

Traces

A directed record of an execution path composed of correlated spans.

Spans

Individual operations with timestamps, parent-child relationships, and attributes.

Tool Calls

Protocol-level operations, as defined by MCP, that need explicit trace correlation.

Governance Evidence

Trace data structured to support audit, security review, and accountability.

Turn Agent Telemetry Into Governance Evidence

Trussed AI provides runtime governance for AI agents, including agent identity, permission enforcement, and audit logging designed to produce evidence suitable for security and compliance review.

Explore Runtime Governance