Applying the NIST AI Risk Management Framework to Agentic AI

The Framework, and Why Agentic AI Stresses It

NIST released the AI Risk Management Framework (AI RMF 1.0) in January 2023 as a voluntary framework to help organizations manage risk and build trustworthiness into AI systems. The framework is structured around four core functions: Govern, Map, Measure, and Manage. In July 2024, NIST published a Generative AI Profile (NIST AI 600-1) extending guidance to risks specific to generative AI systems. NIST has continued to signal further work toward autonomous and agentic systems as adoption accelerates, though organizations should treat agentic-specific implementation as an active application of the existing framework rather than wait for a dedicated agentic profile to be finalized.

The reason agentic AI stresses the framework is straightforward: the AI RMF was written with a model that primarily generates content or predictions in mind. An agent that can call tools, modify records, and trigger downstream workflows introduces a category of risk the original framework's authors were addressing at a higher level of abstraction. Applying the framework well means being deliberate about extending each function to cover what an agent does, not only what a model says.

The Four Functions Applied to Agents

The table below provides an orientation to each function and how its scope changes when the AI system in question is an autonomous agent rather than a model that only produces outputs.

Function	Standard AI System	Agentic AI Extension
GovernAccountability	Who owns the model and its outputs.	Who owns the specific tools and permissions the agent has been granted, and who can approve expanding its scope.
MapContext	Training data, intended use, affected stakeholders.	Every tool, API, MCP server, and downstream system the agent can reach.
MeasureEvaluation	Pre-deployment benchmarking and testing.	Continuous runtime telemetry capturing actual actions taken in production.
ManageResponse	Flagging or filtering a bad output before or after delivery.	Containing, reversing, and communicating about an action the agent has already taken.

Govern: Accountability for Autonomous Action

The Govern function is about establishing the organizational structures, policies, and accountability needed to manage AI risk. For agentic systems, this means defining explicitly who is accountable for an agent's actions, not just who owns the model, but who owns the specific tools and permissions the agent has been granted, and who has authority to approve expanding an agent's scope.

Organizations that treat agent governance as a direct extension of existing model governance committees, without adding action-specific accountability, frequently find that no one can answer who signed off on a given agent's ability to take a specific action once an incident occurs. The policy question is not "who approved this model for deployment" but "who approved this agent to write to this system, and under what conditions."

Governance for multi-agent systems requires an additional layer. When one agent delegates a subtask to another agent, the delegation chain itself must be documented and governed. An authorization granted to a top-level orchestrating agent does not automatically transfer to every sub-agent it spawns. Organizations should define explicitly whether delegated permissions are inherited, scoped-down, or blocked entirely.

Map: Inventorying What an Agent Can Actually Reach

The Map function calls for understanding the context an AI system operates in, including the data, use cases, and stakeholders involved. For an agent, this inventory must extend to the tools it can invoke, the MCP servers or APIs it connects to, and the downstream systems those tools can affect.

A model risk assessment that documents training data and intended use, but does not document that the agent built on top of that model can also write to a production database through a connected tool, has not accurately mapped the system's real risk surface. The risk surface of an agentic system is defined not by the model itself, but by the combination of the model and every tool it has been given access to.

This has practical implications for how organizations structure their AI inventories. Rather than cataloging models alone, a complete inventory for an agentic system should capture:

• The model or models the agent relies on
• Every tool the agent has been authorized to call
• The MCP servers, APIs, or services those tools connect to
• The downstream systems and data stores those services can read from or write to
• The conditions under which each tool is permitted to be used

Without this inventory, it is not possible to conduct a meaningful risk assessment, because the scope of what the agent can do is unknown.

Measure: Continuous Evaluation, Not Just Pre-Deployment Testing

The Measure function is about assessing, benchmarking, and monitoring AI risks. For a static model, this can be substantially satisfied through pre-deployment evaluation: benchmark the model against relevant tasks, document its limitations, and re-evaluate when the model changes.

For an agent whose behavior depends on the specific inputs, tools, and context it encounters in production, pre-deployment testing alone is insufficient. An agent may behave appropriately across every test case and still encounter edge cases in production that cause it to take actions outside its intended scope. The combination of inputs and tool states the agent encounters in the real world will never be fully anticipated in a test environment.

This means agentic systems require continuous runtime telemetry that captures:

• What actions were actually taken by the agent in production
• How often policy checks were triggered and what the outcomes were
• Where the agent's real behavior diverged from its expected task boundaries
• Which tools were invoked and with what parameters

This is the function most commonly under-implemented for agentic systems, because it requires runtime instrumentation that many organizations have not yet built. Pre-deployment testing remains necessary, but it is not a substitute for runtime measurement when the system takes actions with real-world consequences.

Manage: Incident Response Built for Actions, Not Just Outputs

The Manage function covers responding to, recovering from, and communicating about AI risk incidents. For most AI systems, an incident is a bad response that a user can choose to disregard or that a human reviewer can flag before it causes harm. The consequences are often bounded by the fact that the model only produced text or a prediction.

For agentic AI, an incident is an action already taken: a record already modified, a message already sent, a transaction already initiated. The response is fundamentally different because the harm is often not preventable once the action has occurred. Incident response plans need to account for:

• Containment: revoking a specific agent's access without disrupting other agents or systems that share the same infrastructure
• Remediation: reversing or correcting an action already taken, where that is technically possible
• Communication: notifying affected parties or systems, which may carry different obligations than a model-output incident
• Investigation: reconstructing exactly what the agent did, in what sequence, using the runtime telemetry captured under the Measure function

Organizations that apply the AI RMF to static models often have incident response plans built around output monitoring and human-in-the-loop review. Those plans need to be revisited for agentic systems, because the intervention window is different: by the time an agentic action is detected, it may already have had effects that cannot be undone simply by filtering a response.

A Realistic Enterprise Scenario

Consider an enterprise that deploys an AI agent to handle procurement-related tasks: looking up vendor information, drafting purchase order requests, and submitting approved requests through a connected procurement system. The agent has access to an internal database, an email tool, and a procurement API.

Under the AI RMF, a complete implementation would require:

• Govern: a named accountable owner for the agent's access to the procurement API, separate from the team that owns the underlying model; a policy defining which dollar thresholds require human approval before the agent may submit
• Map: an inventory documenting that the agent has access to the procurement API, what that API can do, and which downstream financial systems it connects to
• Measure: runtime telemetry capturing every API call the agent makes, the parameters it passes, and whether any policy check was triggered before submission
• Manage: a documented procedure for revoking the agent's API access if anomalous submission behavior is detected, a process for reversing incorrectly submitted orders if the procurement system supports reversal, and a communication path to notify procurement stakeholders

An implementation that covers only the model-level elements — documenting training data and running pre-deployment tests — would leave the majority of the actual risk surface unaddressed.

Audit Readiness

For organizations preparing to demonstrate AI RMF alignment to internal audit, a board, or a regulator, the strongest evidence is not a policy document alone. The most defensible position pairs a policy document with runtime records showing the policy was actually enforced: which agents exist, what they were approved to do, what they actually did, and what happened when an action fell outside approved scope.

That pairing is what turns a governance framework from a documentation exercise into something that holds up under scrutiny. A policy stating that agents require human approval above a certain threshold is not the same as a log demonstrating that human approval was required and obtained in every such case.

Organizations should consider what evidence they would need to produce if asked to demonstrate compliance, and instrument their agentic systems accordingly before an audit or incident makes that question urgent.

---