Azure OpenAI request path controls: a checklist for banks under the 2026 model risk guidance
Treat the request path between application and model as the unit of control. Work through seven stages: ingress and identity, network and residency, pre-model policy, tool calls, rate limiting and escalation, response handling, and evidence capture. For each, confirm where it is enforced and what record it produces for every interaction.
Minimum evidence record for each interaction
- Caller identity: the authenticated user or service, and the application that made the request.
- Policy applied: the rule that matched and the policy version in force at that moment.
- Decision and action: allow, redact, block, rate limit or escalate, and what actually happened.
- Timestamp and correlation: a request identifier linking prompt, tool calls and response.
- Tool call detail: tool name, parameters, target system and authorization outcome.
- Storage and retention: where the record is held, in which region and for how long.
Why the request path is the unit of control
Bank platform teams that expose Azure OpenAI to internal applications usually end up with controls spread across API management, application code and Azure-native settings. Each layer does something useful, but none of them owns the full decision. When an examiner or model risk reviewer asks what the AI was permitted to do and what it did, the answer is assembled from several systems after the fact.
Organizing controls by stage along the request path fixes the unit of review. For every stage you can ask three questions: what should be verified, where is it enforced, and what evidence does it produce. The checklist below follows that structure.
The supervisory context has also changed. On 17 April 2026 the OCC, in coordination with the Federal Reserve Board and the FDIC, issued updated model risk management guidance. It rescinds prior model risk management guidance and other issuances. It covers model development and use, validation and monitoring, governance and controls, and vendor and other third-party products, and it is risk-based: practices suitable for one bank may not suit another with a different risk profile or model purpose. The agencies have also said they plan to issue a request for information on model risk management that considers banks' use of AI, including generative and agentic AI. As of November 2026, document why each control level fits the use case instead of applying one standard everywhere.
Stages 1 to 3: identity, network and pre-model policy
Stage 1, ingress and identity. Verify that every call reaches the model through one defined entry point, that the caller is authenticated, and that the identity of the end user or calling service is carried into the policy decision instead of being lost behind a shared service credential. Enforcement is split between Azure identity configuration, the gateway and whatever layer evaluates policy. The evidence is the authenticated principal, the calling application, a timestamp and a request identifier that can be joined to later records.
Stage 2, network and data residency. Verify that the model endpoint is reachable only over private networking, that no public path bypasses the entry point, and that the deployment region matches the bank's residency requirements. Microsoft describes private networking and regional availability as Azure OpenAI capabilities, so this stage is largely Azure-native configuration. The evidence is the configuration state, the network path used and the region that served each request.
Stage 3, pre-model policy. Verify that prompts are evaluated before they reach the model and that each one receives an explicit outcome: allow, redact or block. Content filtering configuration is part of this stage, but the bank's own rules on sensitive data, restricted topics and injection patterns have to be defined by the bank and enforced in the path. The evidence is the decision, the rule that matched, the policy version in force and the category of data redacted, recorded without keeping the sensitive content itself.
Stages 4 to 7: tool calls, limits, responses and evidence
Stage 4, tool calls and downstream access. Verify that authorization and privilege checks happen outside the model. OWASP guidance on system prompt leakage is explicit that controls such as privilege separation and authorization bounds checks must not be delegated to the LLM, and that they need to be deterministic and auditable. Training a model not to reveal or exceed its instructions is not a guarantee. Related OWASP research supports enforcing scope through controls the agent cannot rewrite, isolating tools and credentials, reducing outbound request paths and validating external content. Test the real permissions and network paths of each tool integration. The evidence is the tool requested, its parameters, the acting identity, the decision and the action actually taken.
Stage 5, rate limiting and escalation. Verify limits per caller, application and tool, and define which actions require human approval before they proceed. The evidence is the limit that applied, whether it was reached, who approved or rejected an escalation and the outcome.
Stage 6, post-model response handling. Verify that model output passes through policy before it returns to the application, and that any tool call the model proposes is checked before execution. Apply the same allow, redact or block outcomes as on the input side. The evidence mirrors stage 3 for the response.
Stage 7, evidence capture and retention. Verify that the record is written by the enforcement point at decision time and is not reconstructed later. Retention periods and storage location are the bank's decisions, and the storage region should be consistent with the residency requirement from stage 2. Monitor actions actually taken, not only configured permissions.
Separating Azure-native controls from runtime enforcement
Use the checklist as a gap assessment. For each stage, mark whether the control is Azure-native configuration, gateway behaviour, application code or a runtime policy layer, and name the owner. Gaps tend to appear where a control is real but has no consistent enforcement point, or where it produces no per-request evidence. Redaction rules, tool-call authorization and escalation are the stages most often left to application code, which makes them hard to evidence uniformly.
Keep posture management and runtime enforcement distinct. Tools that identify misconfigurations in generative AI applications on Azure OpenAI describe a different capability from enforcing policy on each request. Both matter, but only the second decides what a given prompt or tool call is allowed to do. AI gateways are a market category that route model traffic and apply input and output controls, which shows the enforcement point is a recognized architectural need.
For governance, map each stage to the themes in the 2026 interagency guidance: governance and controls, validation and monitoring, and vendor and third-party products. Policies with named roles, monitoring that reflects actual behaviour and clear treatment of the Azure service as a third-party product all rely on the evidence record above. Tracking the planned request for information on AI is sensible, since it may bring further expectations.
Where Trussed fits
Trussed enforces policy in the execution path across models, agents, tools and MCP servers. It applies allow, redact, block, rate limit and escalate-for-human-approval decisions to individual interactions and tool calls, enforced at runtime, not reviewed after the fact. That enforcement generates audit-ready evidence continuously, mapped to frameworks including NIST. Trussed deploys self-managed in the customer cloud, with configurable retention and data residency.
Trussed runs 100+ models under one policy plane and has run continuously in production in a regulated enterprise for more than 18 months, at 99.95% uptime on live production traffic. It serves customers in four regulated verticals: insurance, pharma, healthcare and education. Insurance, a regulated financial sector, is where it is deepest. Trussed does not claim bank deployments.
The checklist above is useful without it. Where stages 3 to 7 are the gaps, a policy layer in the path gives those stages a single enforcement point and a single evidence record.
The request path, stage by stage
Ingress and identity
Authenticated callers with identity carried into policy decisions.
Network and residency
Private paths only, in approved regions.
Pre-model policy
Allow, redact or block prompts before the model sees them.
Tool calls
Authorization and scope enforced outside the model.
Limits and response handling
Rate limits, human escalation and output policy.
Evidence and retention
A per-interaction record written by the enforcement point.
Assess your request path against the checklist
Walk through the seven stages with an expert and see how enforcement in the execution path produces per-interaction evidence.
Talk to an ExpertHow does your AI governance program compare?
See where your program has gaps in less than 2 minutes.