Implementation Guide | Financial Services

    Bringing Shadow AI Under Policy Behind the Bank Firewall Without Sending Prompts to a Vendor

    Shadow AI in banking can be brought under enforceable policy by running the enforcement layer inside the institution's own network and cloud boundary, so prompts are inspected where they originate and are never relayed to a vendor. The path is: discover AI traffic, write policy, apply allow, redact, block, rate limit or escalate decisions at runtime, set residency and retention yourself, and keep the resulting evidence in your own environment for examiners and model risk reviewers.

    Evaluating vendors against a no-prompt-egress requirement

    • Does any prompt, response or derived evidence leave the network or cloud boundary, and what telemetry is sent to the vendor by default?
    • Which components inspect prompts, and does each one run inside infrastructure you operate?
    • Where is data stored, and which retention and deletion settings do you control rather than the vendor?
    • How are agents and non-human actors identified and authorized, and are prompts and data inputs visible for risk decisions?
    • What evidence is produced, in what format, and can a model risk or examination team review it without vendor involvement?
    • Can the vendor demonstrate through network egress testing in your environment that no prompts or evidence are sent outside?

    Why a third-party filter does not solve the problem

    Shadow AI is AI use that happens without the knowledge or oversight of central IT or a governance body. In a bank that typically means employees pasting text into browser-based AI sites, developers using coding assistants, staff switching on model features embedded in SaaS tools, and teams calling model APIs directly. Each path moves prompts and data outside the institution's control.

    The obvious response is to route that traffic to a cloud filter that inspects it. That creates a second exposure: the same prompts now reach another vendor. For a CISO this is an architectural requirement, not a legal mandate. The institution decides that prompt content, response content and derived evidence must not leave its boundary, and any candidate design is tested against that rule.

    There is also a governance consequence. A vendor that receives prompts for inspection is a third-party relationship that needs due diligence, and its handling of that data becomes something the institution must explain. Keeping inspection inside the boundary removes that relationship from the prompt path altogether.

    Three enforcement architectures and where prompts go

    The useful question for any design is where prompt data travels between the user and the model, and when a decision is made. The comparison below is an editorial framing of the common patterns, not a ranking of specific products.

    • Runtime enforcement in your boundary: Policy is applied in the execution path, before the prompt reaches the model or tool. Prompts and evidence stay in the institution's network and cloud. Decisions are enforced at runtime, not reviewed after the fact.
    • After-the-fact review: Interactions complete first and are examined later. The data has already left, so review can describe an exposure but cannot prevent it.
    • External relay for inspection: Prompts are sent to a vendor service that inspects and returns a verdict. Enforcement may be timely, but the prompt has crossed the boundary and a new third party holds it.

    Choosing runtime actions and enforcement points

    Each shadow AI channel is reached at a different point. Browser-based sites are typically seen at the browser or network edge. Coding assistants are seen at the developer endpoint or the path to the assistant's service. Embedded SaaS features are harder, because the model call happens on the provider's side, so the institution often can only govern whether the feature is enabled and what data feeds it. Direct API use is visible where outbound calls can be identified and attributed. Most institutions will need more than one enforcement point, and each must be deployable inside the boundary.

    The five runtime actions suit different cases. Allow is the default for sanctioned use. Redact fits interactions that are legitimate but contain sensitive fields. Block applies to unsanctioned tools or prohibited data. Rate limit contains volume risk, such as bulk extraction through an approved tool. Escalate for human approval suits high-consequence actions, particularly tool calls by agents, where an automatic answer is not defensible.

    The tradeoff is friction. Blocking everything unsanctioned pushes use to personal devices, where visibility is lower. Offering a sanctioned alternative, and using redaction and escalation instead of blanket blocking, usually keeps usage in view.

    Evidence for examiners and model risk management

    In April 2026 the OCC, in coordination with the Federal Reserve Board and the FDIC, issued updated model risk management guidance, and the Federal Reserve transmitted revised guidance as SR 26-2. As of November 2026, the OCC release describes the guidance as covering model risk factors, model development and use, validation and monitoring, and governance and controls. The Federal Reserve letter says it is expected to be most relevant to banking organizations with over $30 billion in total assets, and that practices appropriately vary with risk profile and model usage. Confirm with counsel which guidance applies to your charter.

    The materials emphasize clear policies and effective controls. For shadow AI, that points to evidence showing both halves: the written policy, and proof that a control applied it to real interactions. Whether a particular tool is treated as a model in the institution's inventory is a determination for the model risk function, and enforcement records give that function something concrete to review.

    Evidence is only useful if it is durable and queryable. Records should show the policy version, the identity involved, the action taken and the outcome, and they should live where the institution controls access and retention. Evidence that sits with a vendor reintroduces the exposure the architecture was meant to remove.

    Policy enforcement inside the institution's boundary

    Discover

    Inventory AI sites, coding assistants, embedded features and direct API use.

    Decide

    Apply written policy to each interaction: allow, redact, block, rate limit or escalate.

    Contain

    Prompts, responses and evidence stay in the institution's network and cloud.

    Evidence

    Retain records that a model risk function or examiner can review.

    Test your shadow AI controls against the boundary requirement

    Talk through discovery, runtime policy and evidence design for a self-managed deployment with a Trussed specialist.

    Talk to an Expert