Implementation Guide | Financial Services

    When to send bank traffic to OpenAI vs Azure OpenAI: a data classification routing policy

    Route by data classification and destination together. Public and internal-only content can reasonably go to either approved endpoint. Confidential content should reach only the endpoint whose contract, residency, network and identity controls satisfy your requirements. Customer NPI/PII, confidential supervisory information and MNPI should be redacted, escalated or blocked by default. Enforce the mapping on every request in the execution path, and record each decision as evidence for model risk and examiner review.

    Evidence a routing decision should produce

    • Classification outcome for each part of the request, and how it was determined
    • Policy version in force at the time of the decision
    • Destination endpoint requested and destination actually used
    • Action taken: allow, redact, block, rate limit or escalate
    • Redaction applied, including what category of data was removed
    • Approver, reason and expiry where the request was escalated

    Why routing needs a rule, not a preference

    Many banks use both OpenAI's direct API and Azure OpenAI Service. Without a written mapping, each team decides which endpoint to call, and customer PII, confidential supervisory information, material non-public information and internal content can end up at destinations with different contractual terms, residency options, network controls and audit evidence.

    The underlying control is information flow enforcement. NIST SP 800-53 Rev. 5 describes it as comparing the security and privacy attributes of the information, including its content and structure, with those of the source and destination. It responds when a flow is not explicitly allowed by policy. For LLM traffic, the destination endpoint is therefore an attribute of the decision, next to the data classification.

    A policy document alone does not meet that standard. Model risk management teams and examiners will ask whether the decision was applied to each interaction, and what the system did when a request fell outside the rule.

    Decision matrix: classification tier to permitted endpoint

    The matrix below is a starting position, not a verdict on either provider. Adjust each row after your legal, security and model risk reviewers have assessed the agreements and controls in force for each endpoint. Anything not explicitly allowed is blocked or escalated.

    • Public: Endpoint: OpenAI direct or Azure OpenAI. Action: allow. Still subject to rate limits and the same evidence record.
    • Internal only: Endpoint: either, if the provider terms are approved for internal content. Action: allow, with redaction of embedded identifiers.
    • Confidential: Endpoint: only the one whose contract, residency, network and identity controls meet your requirements. Action: allow to that endpoint, block to the other.
    • Restricted: Endpoint: neither by default. Action: block, or escalate for human approval with a documented business reason.
    • Regulated: NPI/PII: Endpoint: neither until identifiers are removed. Action: redact, then re-evaluate the residual content; block if redaction is incomplete.
    • Regulated: MNPI and confidential supervisory information: Endpoint: neither. Action: block, with escalation only through a named approver and a recorded decision.

    What separates the two endpoints in practice

    Both services give access to OpenAI models, but the routing decision depends on the terms and controls around the model, not the model itself. Record these attributes for each endpoint and review them whenever a provider changes its terms:

    Data use and retention: what is retained, for how long, and whether customer data is used for training. An FTC report notes that data sent through an API may be governed by the provider's policies and any business agreements, so the contract is a routing input.

    Abuse monitoring: whether prompts may be reviewed, by whom, and whether a configuration exists to change that.

    Regional processing and residency: where requests are processed and stored.

    Network and identity: whether private networking and enterprise identity integration are available. Microsoft describes Azure OpenAI as offering private networking, regional availability and responsible AI content filtering. Treat that as a vendor characterization to confirm against your own tenant configuration.

    Content filtering and attestations: which filters apply, and which certifications and compliance attestations your reviewers will accept.

    The result is a documented profile per endpoint. Each tier in the matrix points to a profile, and each profile has a named owner who approves changes.

    Expressing the matrix as enforceable rules

    Define the tiers and the handling ruleset first, then write each rule as an action by destination. NCCoE data classification guidance describes labels that carry provenance, sensitivity and releasability, paired with handling rules that can be communicated to other systems and audited across the information lifecycle.

    Four actions cover most cases. Allow when the tier is permitted for the destination. Redact when the content can be made permissible by removing identifiers. Block when no permitted destination exists. Escalate when a human with delegated authority must approve an exception before the request proceeds.

    Two design points matter. First, inspect parts separately. NIST notes that decomposing information into policy-relevant parts before transfer supports decisions on classification, attachments and destination, and a prompt, a pasted table and an uploaded file may sit in different tiers. Second, do not rely on the caller's self-declared label. Carry classification to the enforcement point and let content inspection raise the tier, never lower it.

    Trussed enforces policy in the execution path across models, agents, tools and MCP servers, applying allow, redact, block, rate limit and escalate-for-approval actions to individual interactions. Policy is enforced at runtime, not reviewed after the fact. It does this across more than 100 models under one policy plane, so one tier-to-endpoint rule set can cover both OpenAI endpoints and others.

    Failure modes to design against

    Misclassification is the most common. A document labeled internal may contain customer account details, and a prompt may not carry any label at all. Content-based checks and a default of the more restrictive tier when signals conflict reduce this risk.

    Shadow API keys and bypass of approved routes are the second. If a team can call either provider with its own credentials from outside the enforcement path, the matrix only governs well-behaved traffic. Pair the policy with credential management and egress controls so the approved route is the only route.

    The third is policy drift. Providers change terms, and endpoints change configuration. Tie each endpoint profile to a review date and require re-approval of affected tiers when it changes.

    The fourth is escalation without discipline. If approvals are informal, exceptions become the real policy. Escalation should name the approver, the reason and the scope, and expire.

    Routing decision per request

    Classify

    Determine the tier of content, structure and attachments before transfer.

    Match destination

    Compare the tier with the permitted endpoint: OpenAI direct, Azure OpenAI or neither.

    Act

    Allow, redact, block or escalate for approval.

    Record

    Keep the classification, policy version, destination and action as evidence.

    Turn the routing matrix into runtime enforcement

    Review how tier-to-endpoint rules can be enforced on individual interactions and how the resulting evidence supports model risk and examiner review.

    Talk to an Expert