LLM Governance Baseline: What It Is and How to Set One

What a baseline is, and what it is not

A baseline is a floor. It lists the controls that no LLM use may fall below, and it is written so that adherence can be checked for any single interaction. A governance program is broader. It covers roles, risk assessment, review boards, training, vendor management and ongoing oversight. The baseline is the part of that program that can be turned into rules a system applies.

The distinction matters because many organizations already have policy documents but no agreed answer to a simple question: which controls apply to every LLM interaction? Without one, controls differ by team, exceptions go untracked, and nobody can show what an LLM was allowed to do and what it did.

The baseline should not be built in isolation. The NIST AI RMF Govern playbook and the OWASP LLM governance checklist both advise connecting AI governance to existing governance, privacy, security and risk practices. Treat the baseline as an extension of those, with a named owner and documented responsibility for risk assessment.

Step 1 and 2: Define scope and inventory LLM usage

Start by writing down what the baseline covers. A usable scope names the interaction types (chat, API calls, agent tool calls, coding assistants), the models and providers in use, and the data classes those interactions may touch. Decide explicitly whether internally built applications and third-party AI sites are in scope. Anything left out of scope will need a separate control path.

Then inventory actual usage rather than approved usage. Review which teams call which models, which tools and MCP servers agents can reach, and which coding assistants developers have installed. The inventory shows where the baseline will bind first and where the largest gaps sit. Record an owner for each entry, since the baseline needs someone accountable for exceptions.

• Interaction types: chat, API, agent tool calls, coding assistants
• Models and providers, including versions in use
• Tools and MCP servers reachable by agents
• Data classes each interaction may touch
• Owning team and accountable individual for each use

Step 3 and 4: Set minimum controls and map them to NIST AI RMF

Choose controls by risk tolerance, not by what is easiest to implement. AI RMF Govern 1.3 calls for processes that determine the level of risk management activity from the organization's risk tolerance, and Govern 1.4 expects outcomes to be established through transparent policies, procedures and other controls based on risk priorities. A baseline therefore needs a recorded rationale for each control, not just a list.

NIST describes outcomes. Govern 1 says policies and practices should be in place, transparent and implemented effectively, and Govern 1.1 requires legal and regulatory requirements to be understood, managed and documented. Neither prescribes per-interaction technical controls. Translating them into allow, redact, block, rate limit and escalate actions is a design decision your team makes, and the mapping document should say so.

The NIST Generative AI Profile adds a related idea: minimum thresholds for performance or assurance criteria, reviewed as part of go/no-go deployment approval. That is a pre-deployment gate. It complements runtime controls but does not replace them, because approval at launch says nothing about what an individual request or tool call does afterward. The profile also notes that governance tools applied to other AI systems can be applied to generative AI, which supports defining one baseline across models rather than one per team.

Step 5: Enforce at runtime, version the baseline and handle exceptions

A control that exists only in a policy document cannot be shown to have operated. Enforcement in the execution path, where each prompt, response and tool call is evaluated against policy before it proceeds, is what makes a baseline verifiable. The decision and its inputs are then a by-product of the control itself, not something reconstructed later.

Version the baseline like any other controlled artifact. Each change should carry an identifier, an approver and an effective date, so any interaction can be tied to the policy version that governed it. Also record the model version in use. When a model is updated or replaced, you need to know which interactions ran under which combination.

Exception handling is where baselines usually erode. Define who can approve a deviation, what scope it covers, when it expires and how it is recorded. An exception that is not time-limited becomes a second, undocumented baseline.

Trussed enforces policy in the execution path across models, agents, tools, MCP servers, AI sites and coding assistants, at the level of individual interactions and tool calls, with allow, redact, block, rate limit and escalate for human approval. It runs 100+ models under one policy plane and can be self-managed in the customer cloud with configurable retention and data residency. Enforced at runtime, not reviewed after the fact, the baseline applies the same way regardless of which team or model is involved.