Assembling an AI control plane with OPA, LiteLLM, and OpenTelemetry: what it enforces and where gaps remain

Separate decision, enforcement, and evidence

The assembled stack is easier to reason about when three questions are kept apart. Where is a policy decision computed? Where is it enforced? Where is the outcome recorded?

OPA answers the first. It takes structured input, such as the user, the model, the request attributes, and the data classification, evaluates Rego policy, and returns a decision. It does not sit in the traffic. A decision that nothing acts on is advice, not control.

LiteLLM answers the second, for a defined scope. It is a model routing proxy with virtual keys, budgets, and callback hooks. Because it sees requests and responses, it is the natural place to call OPA and act on the answer. That glue is code your team writes and owns.

OpenTelemetry answers the third, in a limited sense. It standardizes how traces, metrics, and logs are emitted and transported. Telemetry describes behavior. Evidence is a record that a third party can rely on. The two are not the same thing, and the difference matters most in a regulated enterprise.

Which control functions the stack covers

Coverage differs by function. Allow and block are the cleanest fit. Redact, rate limit, and escalate each depend on what your team builds around the components.

• Allow: OPA decides. The proxy hook permits the request to proceed. Covered for traffic through the proxy.
• Block: OPA decides. The proxy hook rejects the request or response. Covered for traffic through the proxy.
• Redact: OPA can signal that redaction is required, but the detection and transformation logic must be written and tested by your team.
• Rate limit: Key-level budgets and limits come from the proxy. Limits tied to user, data class, or tool need custom policy inputs.
• Escalate: No component supplies a human approval workflow. OWASP notes approval can live in a downstream system outside the LLM application, so it must be built there.

Where coverage ends

The first boundary is the proxy itself. A proxy governs only what is sent to it. Coding assistants, browser-based AI sites, and applications holding their own provider credentials can bypass it, and nothing in the proxy or OPA will see that traffic. Closing this requires network and credential controls, plus a way to detect unmanaged clients.

The second boundary is agency. OWASP describes excessive agency as arising when a model can call functions or interface with other systems through tools, skills, or plugins. A policy check on the prompt and response does not govern what a tool then does downstream. OWASP's mitigations operate at the tool and downstream layers: granular tools instead of open-ended ones, execution in the context of the specific user with minimum privileges, and human approval of high-impact actions. That means identity propagation and per-action authorization, which a model-routing layer does not provide.

MCP servers are a third surface. OWASP publishes separate guidance for securing MCP use and for vetting third-party MCP servers, which supports treating MCP traffic as its own control point with its own policy and review process.

Agentic risks such as cascading failures, where false signals spread through automated pipelines, also point past per-request checks. Per-request decisions are necessary but do not describe a multi-step run.

Telemetry is not yet evidence

OpenTelemetry gives teams a consistent way to see requests, latency, and model usage. OWASP's Agent Control Standard says agents must be inspectable, traceable, and instrumentable, meaning visibility into what they can access and what they did and why. Telemetry contributes to that. It does not complete it.

An auditor will ask whether the record is complete, whether it was altered, how long it is kept, and where it is stored. Standard telemetry pipelines are built for operations, so sampling, dropped data, and mutable storage are normal. To turn them into evidence, a team has to define which events must never be sampled, make records tamper-evident, set retention periods, and control where data resides. It must also record the policy version behind each decision and map the records to the frameworks it answers to, such as NIST AI RMF, HIPAA, or the EU AI Act (Regulation (EU) 2024/1689, adopted 13 June 2024).

Evidence is strongest when the enforcement point generates it, because the record then shows what was actually allowed or blocked rather than a reconstruction from separate systems.

How purpose-built runtime enforcement differs

Trussed is one example of the purpose-built approach. It enforces policy in the execution path across models, agents, tools, MCP servers, AI sites, and coding assistants, with actions that include allow, redact, block, rate limit, and escalate for human approval. Enforcement applies to individual interactions and tool calls. It generates continuous audit-ready evidence from that enforcement, mapped to HIPAA, NAIC, NYDFS, the EU AI Act, and NIST. It is deployed self-managed in the customer cloud, with configurable retention and data residency.

The comparison is not open source against commercial. It is about which team owns the gaps above. Teams that choose to extend the assembled stack should plan for bypass detection, tool and MCP enforcement, evidence integrity, and framework mapping as ongoing engineering work, not as a one-time integration.