Why Standardized Telemetry Matters for Agent Systems
Enterprises running LLM-based agents in production face a common problem: each vendor SDK, orchestration framework, and model provider tends to log activity differently. A tool call in one framework produces a different trace structure than the same operation in another. This fragmentation makes it difficult to reconstruct what an agent did, in what order, and why, particularly when a single user request triggers multiple model calls and tool invocations across services.
OpenTelemetry addresses this at the infrastructure level. It is a CNCF-hosted project that already provides a mature distributed tracing model (spans, span attributes, span events, and trace context propagation) used broadly across cloud-native systems. The OpenTelemetry GenAI working group has extended this model with semantic conventions specific to generative AI, giving teams a common vocabulary for representing model invocations and, increasingly, agent tool calls.
What the GenAI Semantic Conventions Define
The current specification defines attributes for LLM client operations, including gen_ai.system (identifying the model provider), gen_ai.request.model, and token usage fields such as gen_ai.usage.input_tokens and gen_ai.usage.output_tokens. These are span-level attributes intended to be structured, low-cardinality metadata suitable for dashboards and cost tracking.
Separately, prompts and completions are captured as span events rather than attributes. This distinction matters operationally: event-based capture allows large or sensitive content to be handled differently from structured metadata, which is relevant when deciding what gets logged, retained, or redacted.
The working group has also published guidance for instrumenting tool and function calls within agent workflows, extending the conventions beyond single LLM requests toward multi-step agent behavior. This is a newer and less mature part of the specification compared to the core LLM request attributes.
Important Caveat: Everything Is Still Experimental
OpenTelemetry documentation explicitly marks the GenAI semantic conventions as experimental. This means attribute names, span structures, and event formats can change without the backward-compatibility guarantees that apply to stable OpenTelemetry components. Teams instrumenting production systems today should expect some rework as the specification matures toward general availability.
This has practical consequences. Platform teams should document which schema version they instrument against, monitor specification updates, and avoid hardcoding vendor-specific field names directly into downstream analysis logic.
Practical safeguard
An abstraction layer between raw telemetry attributes and internal dashboards or governance tooling reduces the cost of future schema changes.
A Practical OpenTelemetry-Based Observability Architecture
A workable architecture separates concerns into three layers so that backend or vendor changes do not require re-instrumenting applications.
Where OpenTelemetry Falls Short of Governance Requirements
Operational observability and governance-grade auditability are related but distinct problems. OpenTelemetry's GenAI conventions were designed to help engineers debug and monitor systems, not to produce a standardized audit trail suitable for regulatory or compliance evidence. There is currently no defined format within the specification for immutable, tamper-evident audit records of agent actions.
This creates a gap enterprises need to plan for directly. Capturing full prompts and completions for audit purposes may conflict with data minimization requirements, so the decision about what content-level telemetry to retain has to be made explicitly as policy, not left to default SDK behavior. Additionally, because different vendors implement the GenAI conventions with varying completeness, an agent workflow that spans tools from multiple providers can produce inconsistent or incomplete traces, undermining end-to-end auditability exactly where it matters most: multi-tool, multi-vendor agent chains.
This is where runtime governance controls become necessary as a complement to telemetry, not a replacement for it. Standardized spans tell you what happened technically; policy enforcement and audit logging at the point of tool invocation determine whether that action was authorized and how it is recorded for compliance purposes. Trussed AI's runtime governance and audit logging capabilities are built to operate at this layer, enforcing agent permissions and tool approval workflows at execution time and producing audit records independent of whether every downstream vendor has fully implemented the same telemetry schema.