LLM Governance Controls Matrix: Design to Runtime
AI Governance Controls

LLM Governance Controls Matrix: Design, Deploy, Runtime

An LLM governance controls matrix maps specific governance and security controls to each stage of the model lifecycle: design, deployment, and runtime. Design-stage controls address data governance, model selection, and threat modeling. Deployment-stage controls cover access provisioning, agent identity assignment, and environment hardening. Runtime controls, the stage most often left unaddressed, enforce policy, validate agent identity per request, govern tool calls, and log activity while the system is operating. Governance gaps in production LLM applications typically occur because design and deployment controls are treated as sufficient on their own, without dedicated runtime enforcement.

Explore Runtime Governance Talk to an Expert
Lifecycle Overview

The Three-Stage Controls Matrix

Stage Focus Areas
Design Data governance, model selection criteria, threat modeling
Deploy Access provisioning, agent identity assignment, environment hardening
Runtime Policy enforcement, tool-call governance, least-privilege authorization, audit logging
Governance Framework

LLM Governance Controls by Lifecycle Stage

Design-Stage Controls: Governance Assumptions Before Deployment

Design-stage governance establishes the assumptions an LLM application will operate under before any code reaches production. This includes data governance decisions about what training, fine-tuning, or retrieval-augmented data sources are permitted, model selection criteria that weigh capability against known risk categories, and threat modeling that anticipates how the system could be misused. NIST's AI Risk Management Framework treats this work as part of its Map function, which establishes context and identifies risks before deeper measurement or ongoing management begins. OWASP's Top 10 for LLM Applications lists prompt injection as a top-ranked risk category, and design-stage threat modeling is typically where organizations first account for it. These controls are necessary. They are also, by definition, based on anticipated behavior rather than observed behavior, since the system has not yet run in production.

Deployment-Stage Controls: Provisioning Before Production

Deployment-stage controls convert design assumptions into a running system. This stage includes access provisioning, agent identity assignment, and environment hardening: the practices that determine what an agent or service account is permitted to touch once it goes live. NIST's AI RMF frames this work as part of its Measure function, and joint guidance from CISA and NSA, published in April 2024, addresses secure deployment and the operational practices that should accompany it. Least-privilege access, as defined in NIST SP 800-53's AC-6 control, is typically applied at this stage, granting accounts and processes only the access needed to perform assigned tasks. The limitation is structural rather than a failure of diligence. Identity and permissions assigned at deployment time are usually static. They describe what an agent is allowed to do in general, not what it should be allowed to do for a specific tool call made hours or weeks later.

Runtime Controls: Where Enforcement Actually Happens

Runtime is where governance responsibility shifts from provisioning to enforcement, and it is the stage most enterprise LLM governance programs leave incomplete. NIST's AI RMF calls this the Manage function, addressing the ongoing treatment of risk once a system is operating rather than the pre-deployment work of mapping and measuring it. OWASP's Excessive Agency risk category exists specifically because permissions granted at deploy time do not automatically constrain what an agent actually does at runtime; an agent with broad service-account access can still act outside its intended scope unless each action is checked as it happens. MITRE