Retrofitting LLM Governance onto LLM Applications Already in Production

Why review alone does not govern a live application

Many regulated enterprises have LLM applications that shipped before formal governance requirements existed. They call models, tools and data sources directly. Controls differ from one application to the next, and there is rarely a durable record of what each application was allowed to do or did.

A documentation or committee review can describe intended behavior. It cannot constrain behavior on the next request. Policy that is enforced at runtime, not reviewed after the fact, sits in the path of each interaction and tool call and decides what happens to it. For a system already carrying traffic, that distinction shapes the whole plan: the goal is to add enforcement and evidence without rewriting applications or interrupting users.

NIST AI RMF 1.0 describes governance as a continual requirement over a system's lifespan, and Measure 2.4 expects the functionality and behavior of an AI system and its components to be monitored in production. A one-time review does not meet that description. Continuous enforcement and continuous evidence do.

Rolling out without causing an outage

Start with observe-only mode. The enforcement point evaluates every interaction against policy and records the decision it would have made, but changes nothing. This produces the first real picture of production behavior, including tool calls and data flows nobody documented. Expect the inventory to grow during this phase. Common surprises are unmanaged API keys, applications calling tools or MCP servers directly, and data sources that appear in no design document.

Use observed decisions to tune policy before enforcing it. Review what would have been blocked or redacted and separate true violations from false positives. Run adversarial tests against the policy set before enabling stricter actions on a live application, as OWASP guidance advises for agents before production.

Then enforce selectively. Begin with the highest-risk tier and the clearest rules, such as redacting sensitive fields or blocking a tool an application should never call. Use escalation for human approval where a call is legitimate but consequential. Define an exception process with an owner, a scope and an expiry date, so exceptions do not become permanent gaps. Keep a tested path to disable or revert enforcement per application. NIST AI RMF 1.0 says deployment should cease safely where unacceptable risk is present, and a working block or disable path supports that.

Promote one application at a time and hold each stage long enough to see normal and peak traffic. Watch latency added by the enforcement point and the rate of blocked interactions against the observe-only baseline.

Designing audit-ready evidence

Evidence should be a product of enforcement, not a separate reconstruction effort. Each decision record should identify the application, the model or tool involved, the policy version applied, the action taken (allow, redact, block or escalate) and the time. Retention and data residency should be configurable, because evidence about regulated data is itself regulated.

Map records to the frameworks that apply to your organization. NIST AI RMF Measure 2.4 aligns with production monitoring records. For EU AI Act high-risk systems, recitals call for deployer impact assessments before deployment, updated when relevant factors change, so live systems may need assessment after the fact. Whether a given application is high-risk is a classification your team must determine. The risk tier and policy rationale recorded in earlier phases support the documentation of risk-management choices that the Act expects. As of October 2026, confirm current obligations and dates against the consolidated text before committing to a timeline.

Trussed is built around this model. It enforces policy in the execution path across models, agents, tools and MCP servers, and generates continuous audit-ready evidence mapped to HIPAA, NAIC, NYDFS, the EU AI Act and NIST. It can be self-managed in the customer cloud, with configurable retention and data residency. It has run in continuous production in a regulated enterprise for more than 18 months, at 99.95% uptime on live production traffic.

Failure points and the limits of interaction-level enforcement

Retrofits most often fail in four places. Inventory is treated as complete after the first pass, so bypass paths remain. Policy is written broadly and then loosened under pressure from false positives, leaving nothing enforceable. Observe-only mode becomes permanent because no one owns the promotion decision. Evidence is retained but not tied to policy versions, so it cannot show what an application was allowed to do at a given time.

Scope matters as well. Enforcement at the level of individual interactions and tool calls tells you what each call was allowed to do and what happened to it. It does not provide transaction-level governance across tasks and agents, and it does not provide end-to-end tracing of agentic workflows. If an audit question concerns a business transaction that spans several agents, you will need other records alongside interaction-level evidence.

Nor does runtime enforcement replace testing, impact assessment or ownership. It makes the decisions made in those activities enforceable and provable on live traffic.