NYDFS Part 500 Risk Assessment for Multi-Agent Systems Built with CrewAI and the OpenAI Agents SDK

What the Part 500 risk assessment requires

Section 500.9 requires each covered entity to conduct a periodic risk assessment of its information systems, sufficient to inform the design of the cybersecurity program. As of November 2026, the assessment must be reviewed and updated as reasonably necessary, at a minimum annually, and whenever a change in the business or technology causes a material change to cyber risk. It must consider the risks of the entity's business operations, the nonpublic information (NPI) collected or stored, the information systems used, and the availability and effectiveness of controls.

NYDFS's industry letter on AI-related cybersecurity risks adds no new requirements. It explains how to use the Part 500 framework to assess AI threats. That shapes the approach: do not build a separate AI governance document. Extend the existing 500.9 process so that a multi-agent system is scored the way any other information system is scored.

NYDFS guidance also expects written policies and procedures that include criteria for evaluating and categorizing risks. In its GEICO consent order, NYDFS directed an assessment covering changes to systems, NPI or operations, the criteria used, the risks identified, and how each risk would be mitigated or accepted. Your rubric should produce those same elements: criteria, inherent rating, controls considered, and a recorded treatment decision.

Score the architecture as configured, not the framework name

CrewAI organizes work as crews, roles and tasks. The OpenAI Agents SDK organizes it as agents with handoffs, tools and framework-provided checks. The vocabulary differs, but the risk questions are the same. Who can act, on what authority, through which tools, against which NPI stores, and who can pass work to whom?

Map each framework's constructs to the four dimensions. A role or agent definition informs autonomy. A tool attached to an agent informs tool access. A data store, retrieval source or memory the agent can reach informs data exposure. A task assigned to another role in a crew, or a handoff between agents, informs delegation depth. Score what is deployed in a given environment. Two deployments on the same framework can differ widely, and a framework default is not a control until you have confirmed it is enabled and effective in your configuration.

Because 500.9 calls for reassessment on material technology change, define adding an agent, tool, data source or delegation path as a re-scoring trigger in your written procedure. Align the language with NIST AI RMF and NIST AI 600-1, which support likelihood and severity as base axes and treat dependencies between AI and other systems as a tiering factor. NYDFS does not mandate either reference, so cite them as alignment, not as requirements.

A scoring rubric for agentic systems

Score each dimension from 1 to 5, combining likelihood of misuse and severity of impact, with anchors written down before scoring begins. The weights below are a starting proposal, not a regulatory standard. State your rationale for any weighting in the procedure, since no source prescribes one.

The inherent rating is the weighted sum before controls. For residual scoring, reduce a dimension only for implemented controls: up to two points for a control enforced at runtime with evidence, up to one point for a control reviewed after the fact, and none for planned controls. Do not go below 1. Suggested bands: up to 2.0 low, 2.1 to 3.0 moderate, 3.1 to 4.0 high, above 4.0 critical.

• Data exposure (30%): 1 means no NPI reachable. 5 means broad read access to NPI stores, or NPI persisting in agent memory or outputs.
• Tool access (25%): 1 means read-only, low-privilege tools. 5 means tools that write, transact, change access or reach external services.
• Autonomy (25%): 1 means every action is human-approved. 5 means unattended execution of consequential actions.
• Delegation depth (20%): 1 means a single agent. 5 means multi-hop delegation in which downstream agents inherit or widen access.

Illustrative scoring of two deployments

The following hypothetical examples show how the arithmetic works and how control evidence changes the outcome. Deployment A is a crew in which a research role delegates to an analyst role that reads customer records. Deployment B is an Agents SDK deployment with a triage agent handing off to a servicing agent that holds several tools.

• Deployment A, inherent: Autonomy 4, tool access 3, data exposure 5, delegation 4. Weighted score 4.05, critical.
• Deployment A, controls: Runtime redaction of NPI (data -2), runtime human approval for high-impact calls (autonomy -1), after-the-fact tool review (tool -1). Delegation controls are planned only, so no credit.
• Deployment A, residual: Autonomy 3, tool access 2, data exposure 3, delegation 4. Weighted score 2.95, moderate.
• Deployment B, inherent: Autonomy 3, tool access 4, data exposure 4, delegation 2. Weighted score 3.35, high.
• Deployment B, controls: Runtime tool allow-list (tool -2), after-the-fact review of data access (data -1).
• Deployment B, residual: Autonomy 3, tool access 2, data exposure 3, delegation 2. Weighted score 2.55, moderate.

Runtime enforcement versus after-the-fact review

The distinction between these two control types drives how much credit a control earns. A review of transcripts or tool-call records after the fact can detect misuse and supports accountability, but it does not prevent the action. A control in the execution path decides, for an individual interaction or tool call, whether to allow, redact, block or escalate for human approval before the action completes. Residual risk claims are stronger when a high-severity dimension is reduced by the second kind of control, and when the control can show it operated.

Each control should be tied to evidence an examiner can inspect: the policy in force, the decision taken on each call, and for escalations, who approved and when. That evidence also supports model risk management, because it shows what the system was permitted to do and what it did. Map these artifacts to the program areas your entity already maintains, such as access privileges, third-party service provider oversight and audit trail, so the rubric connects to the controls Part 500 expects.

Trussed enforces policy in the execution path across models, agents, tools and MCP servers, with allow, redact, block, rate limit and human-approval escalation applied to individual interactions and tool calls. The audit-ready evidence that enforcement produces is mapped to NYDFS and NIST. Trussed is deepest in insurance, a regulated financial sector. Scope matters when scoring: enforcement operates per interaction and per tool call, so reconstructing a full delegation chain is a separate design decision, and delegation depth should not be credited as reduced on enforcement alone.